GET /api/v1/models
Self-hosted inference behind an OpenAI-compatible adapter. Swap models without touching orchestration.
Primary coder
vllm → http://gpu-01.internal:8000/v1
p50 latency
820ms
throughput
38tok/s
context
32ktokens
tool-call validity
98.7%
temperature
0.2
max output
4,096tok
target validity
≥ 99%
adapter
openaicompat
Local laptop
ollama → http://localhost:11434/v1
p50 latency
210ms
throughput
62tok/s
context
32ktokens
tool-call validity
94.1%
temperature
0.2
max output
2,048tok
target validity
≥ 99%
adapter
openaicompat
Candidate (benchmarking)
sglang → http://gpu-02.internal:30000/v1
Under evaluation — not yet selected for production.
p50 latency
1430ms
throughput
29tok/s
context
64ktokens
tool-call validity
96.2%
temperature
0.2
max output
8,192tok
target validity
≥ 99%
adapter
openaicompat
Candidate (not benchmarked)
vllm → http://gpu-03.internal:8000/v1
openai/gpt-oss-20b (21B MoE, 3.6B active) — not deployed or benchmarked yet.
p50 latency
0ms
throughput
0tok/s
context
128ktokens
tool-call validity
0.0%
temperature
1
max output
8,192tok
target validity
≥ 99%
adapter
openaicompat