Reference: HTTP¶
This page lists every route, every header and every error of the engine's server. TandemApp's proxy speaks the same API for clients and has its own routes, headers and errors, and so does its extension inside the engine: TandemApp's docs lists them. The tables are written in tools/http_ref.py, and tools/gen_reference.py --check compares them with the code in the tests: a route a handler tests, a header name or an error type the code uses that is missing here, or a row the code no longer has, fails the check.
Who may call what¶
The admin password is the admin token, QSE_ADMIN_TOKEN in secrets.env. It is not set by default. Then every route below answers anyone who reaches the port, and only /metrics keeps the metrics token when one is set. With a password set, an admin route needs it as a bearer header or a dashboard session (a cookie the token buys). A request from the machine itself (loopback, no proxy headers) needs neither, because the watchdog and the measurement tools rely on that (--trust-loopback).
The OpenAI routes are open on the engine, with or without a password. The table below names the labels the route tables use.
Who may call a route: the labels of the routes' who column¶
| label | who may call |
|---|---|
open |
anyone who reaches the port |
admin |
the admin rule: anyone while no admin password is set; with one, the admin token, a dashboard session, or a tool on the machine itself |
admin-bearer |
as admin, but the token only as a bearer header, never the session cookie |
dash |
the dashboard rule: anyone unless a password is set and the dashboard login is on |
metrics |
the metrics token when one is set, else as admin |
session |
only under a session address, /s/<token>/... |
key |
anyone until the first user key exists; then a user key, except from the machine itself |
collect |
the collect token, from every address |
The engine's routes¶
the table below lists the engine server's routes. A route the server does not know answers 404. A route TandemApp serves answers 404 not_on_engine on a headless engine, and the page 404 no_ui; in chat mode the extension serves them. The proxy's routes without --backends answer 404 with a message that names the flag, and the decision routes without decision models as an unknown route. The table below adds the status responder, which answers while the engine is stopped.
The engine server's routes¶
| route | who | what |
|---|---|---|
POST /v1/chat/completions |
open | A chat completion, streamed as server-sent events or not. Body: OpenAI's chat body: model, messages, tools, stream, max_tokens and the other fields of the API fields. Answer: chat.completion, or chat.completion.chunk events and [DONE]; usage carries prompt_tokens_details.hidden_tokens for the conductor's notes. |
POST /v1/completions |
open | A plain text completion. Body: prompt, max_tokens and the sampling fields. Answer: text_completion, or its chunks and [DONE]. |
GET /v1/models |
open | The model list: TandemLLM alone with the conductor on, else the served name. Answer: {object: list, data: [...]}, and decision_models beside data when decision models are served. |
GET /v1/models/{id} |
open | One model card: the engine's or a decision model's. Answer: the card, or 404 model_not_found. |
GET /v1/replica/identity |
open | What two engines compare before they serve one model as replicas: the code, the family, the weights, drafters, rows and tokenizer by content, and the decode settings (SRV-741). Follows the caller rule of /v1/models. Answer: {identity: 1, served_model, base_model, version, code, family, layers, profile, weights, drafters, rows, tokenizer, decode, max_len, stage}; 503 identity_pending until the first call's computation ends. |
GET /health |
open | Whether the server takes work. The full body follows the admin rule; anyone else gets the status alone. Also /healthz and /v1/health. Answer: {status: ok\|draining}; the full body adds model, uptime, inflight, the queue, limits, sampling, cache, memory and parallel; 503 while draining. |
GET /metrics |
metrics | The Prometheus families of the metrics reference. Answer: Prometheus text format. |
GET /metrics/up |
open | One gauge, qse_up, so a scraper can tell an engine that is down from a wrong token. Answer: qse_up 1, or 0 while draining. |
GET /v1/cache/stats |
admin | Every cache's counters. Answer: the cache report /health carries. |
GET /v1/cache/memory?pools=1 |
admin | The engine's memory by owner against what it held once loaded: each slot's anchors, end anchor, stash, pin and park, the store, the KV pool's pages and what no cache holds (SPD-257). pools=1 adds the allocator's segments by pool; it walks every block. Answer: {allocated, reserved, load, caches, by_owner, unattributed, limits, kv_pool, allocator_pools?}. |
POST /v1/cache/peek |
admin | Where a chat's rows are before it is sent: the body is rendered as a request would be, the session's router notes are replayed, and the cache tiers are asked without running the model (SRV-349). Body: a chat body (messages, tools, the effort fields) and session. Answer: {tier: resident\|ram_park\|ram_snapshot\|nvme_spill\|disk_tier\|none, cached_tokens, reread_tokens, reread_eta_ms, tokens, restore_ms, session}; 409 while a request holds the engine. |
POST /v1/cache/prewarm |
admin | Reads a conversation the peek finds on the NVMe back into the engine while nothing runs or waits, so its next request resumes at the end anchor. Body: as /v1/cache/peek. Answer: {prewarmed, why?, restore?} and the peek's fields; 409 while busy. |
POST /v1/tokenize |
open | Exact token counts with the model's own chat template, split by part; nothing runs on the GPU (SRV-347). With ids: true also the ids, the chat prompt's or the text's as /v1/completions reads it (SPD-249). 240 a minute from one address. Body: {messages, tools?, ids?} or {text, ids?}. Answer: {tokens, parts: {system, tools, messages}, ids?}. |
POST /v1/cache/clear?tier=ram\|disk\|all |
admin-bearer | Empties the caches in memory (ram, the default), the disk tier and the session spill (disk), or both (all). Nothing else is reset. Answer: the cache report after the clear; 400 for another tier. |
POST /v1/cache/release |
admin-bearer | Gives back memory the engine holds idle, cheapest first: free allocator blocks and caches while slot 0 is idle, the small models, idle side conversations, slot 0's idle conversation (to the NVMe first). chatd asks it before a sandbox waits (SRV-483). Body: {bytes}. Answer: what each step gave back and the total. |
GET /dashboard/ |
open | No page: every engine answers 404 no_ui under /dashboard (engine API 1.8); TandemApp's controller serves the page. Answer: 404 no_ui. |
GET /v1/dashboard/summary?from=&to=&tz=&model=&client=&backend=&user=&internal= |
dash | The Usage view's totals for a range. Answer: summary.schema.json in contracts/engine-api-v1/dashboard. |
GET /v1/dashboard/usage?bucket=hour\|day&from=&to=&tz=&model=&client=&backend=&user=&internal= |
dash | Tokens, requests and cost per hour or day. Answer: usage.schema.json. |
GET /v1/dashboard/requests?limit=&before=&finish=&model=&client=&backend=&user=&internal= |
dash | Recent request rows of the usage ledger, newest first. Answer: requests.schema.json. |
GET /v1/dashboard/system |
dash | The machine, the engine, the memory plan and the slots; the proxy's mode is always off. Answer: system.schema.json. |
GET /v1/dashboard/live |
dash | What each request is doing now, as server-sent events. Answer: live.schema.json events. |
GET /v1/dashboard/logs?level=&grep=&backlog=&since=&follow= |
dash | The log: a backlog of up to 5,000 lines, then live lines as server-sent events; Last-Event-ID resumes; follow=0 answers JSON once. Answer: logs-line.schema.json events, or logs-json.schema.json; 429 past four streams at once. |
GET /v1/dashboard/logs/export |
dash | The whole log ring as one text file: a header with the version, the commit, the uptime, the time of export and the line count, then every line the ring holds, at all levels, oldest first. It takes no stream place. Answer: text/plain as an attachment named tandemllm-<version>-<time>.log. |
GET /v1/dashboard/metrics |
dash | The /metrics page under the dashboard's rule, for the Performance view. Answer: Prometheus text format. |
GET /v1/dashboard/parallel?since= |
dash | Every request in flight on the engine. Answer: parallel.schema.json. |
GET /v1/dashboard/session |
open | Whether a sign-in exists and is needed. Answer: {authenticated, login, password_set, expires_at}. |
POST /v1/dashboard/session |
open | The sign-in, with a password set and the login on: sets the qse_dash cookie. Five failures a minute from one address and the sixth gets a 429. Body: {token}. Answer: the session; 401 for a wrong token. |
DELETE /v1/dashboard/session |
open | Signs this session out. Answer: the session, signed out. |
* /v1/conductor/* |
admin | The conductor's documents, journal, sessions and memory, served by the extension in chat mode (TandemApp's docs lists them). Reads follow the dashboard rule, writes and every memory route the admin rule. 404 not_on_engine on a headless engine. |
* /v1/setup/* |
admin | The Setup Guide's API (docs/setup.md): the model registry, the box, the memory estimate, the review plan, the apply and reset jobs and Wren. Every POST must also be same-origin JSON: 415 for another content type, 403 for another origin. In setup mode (--setup) these, the session and /health are the only routes that answer. |
* /v1/config/* |
open | 404 not_on_engine on every engine (engine API 1.8): other backends sit behind TandemApp's controller. Answer: 404 not_on_engine. |
POST /v1/collect/{backend} |
open | 404 not_on_engine on every engine (engine API 1.8): the host collector is TandemApp's controller's. Answer: 404 not_on_engine. |
* /v1/chat-app |
open | 404 not_on_engine on every engine (engine API 1.8), and every path under it: TandemApp's controller serves the Chat tab's service. The body is not read. Answer: 404 not_on_engine. |
POST /v1/systemone |
admin | Typed decisions from the decision models, in the engine's idle window. /v1/predict is the same route. Only with QWEN38_DECIDER_MODELS. Body: {state, model?, questions: {id: {type, instructions, criteria?}}, task?, lang?}. Answer: {model, answers, usage, routing, latency_ms, path}. |
POST /v1/predict |
admin | The same as /v1/systemone. Body: as /v1/systemone. Answer: as /v1/systemone. |
GET /readyz |
open | Whether a decision model is loaded and warm. Answer: 200, or 503 with the reason. |
GET /v1/laya |
open | What the Laya route takes and answers, its limits, and whether it is on. The Laya, library and session memory routes below are the extension's in chat mode (TandemApp's docs has their shapes); a headless engine answers them 404 not_on_engine. Answer: a description of the route. |
POST /v1/laya/ask |
open | A typed answer from Laya on one text or many, for the model's own scripts; rate-limited. Body: {question, choices \| scale \| neither, text \| texts, threshold?, model?, deadline_ms?}. Answer: {label, p, sure, probs, model, ms, ...}, or {answers: [...]} for texts. |
POST /v1/laya/drive/end |
open | Ends a playtest driver session and writes its journal row. Body: the session's summary. Answer: the row. |
GET /v1/laya/kit/{file} |
open | A file of the playtest kit the model copies into a project. Answer: the file. |
OPTIONS /v1/laya/* |
open | A browser's CORS preflight for the Laya routes, which the engine asks the extension for. Every other path answers OPTIONS with 501. Answer: 204 with the extension's CORS headers. |
GET /v1/library |
open | The package library: every enabled skill's name, description and classes. Answer: the list and the two routes below. |
GET /v1/library/search?q=&k=&class= |
open | BM25 search over the skills, best first (k from 1 to 20, default 5). Answer: [{name, description, score}]. |
GET /v1/library/{name} |
open | One skill's text and its reference. Answer: {name, description, classes, body, reference}. |
GET /s/{token}/v1/memory |
session | The memory index this session sees. The /s/{token}/ prefix also serves the library and Laya routes, filed to the session. Answer: the notes' index lines. |
GET /s/{token}/v1/memory/search?q=&k= |
session | Memory notes that match, best first (k from 1 to 10). Answer: [{id, kind, scope, status, title, score}]. |
GET /s/{token}/v1/memory/{id} |
session | One note. Answer: {id, title, kind, scope, status, body, ...}. |
POST /s/{token}/v1/memory |
session | Writes, changes or archives a note, with the fields of the hidden memory block. Body: {op, scope, kind, title, tags, body, id, reason}. Answer: {op, id, title, ...}, or 422 with the reason as the type. |
The status responder beside the engine¶
| server | route | answer |
|---|---|---|
ops/engine_status.py |
GET /v1/models |
200 with the engine's last model list and its state, while the engine is away |
ops/engine_status.py |
GET /dashboard/ |
503 with a page that says why the engine is away and opens the dashboard when it is back |
ops/engine_status.py |
* any other path |
503 provider_unavailable with Retry-After and the state as engine_<state> |
Request fields of the engine's own¶
Two fields of a chat or completion request concern the wait for memory (waiting for memory). The engine reads them; What the API supports lists the OpenAI fields.
tandem.priority:"high","normal"or"low", the chat's place in the line for memory and in the order the KV pool's reclaim takes idle conversations. Any other value, or none, is normal (0.15.3).wait_for_memory: true, in the body or in itstandemobject: a request without a stream waits for memory up toQSE_MEM_WAIT_Sinstead of--queue-timeout, as the headerX-Tandem-Wait-For-Memory: 1asks (0.15.3).
Headers¶
the table below lists every header the engine reads or writes beyond the ones HTTP itself needs. in is a header the server reads, out one it writes.
Every header the engine reads or writes (both: TandemApp's controller too)¶
| header | direction, where | meaning |
|---|---|---|
X-Tandem |
both, both | raw, shadow or on: lowers the conductor's mode for one request; the response says which mode ran |
X-Tandem-Profile |
both, both | picks a conductor profile; the response names the profile that ran; an unknown name is a 400 |
X-Tandem-Memory |
in, engine | off: no memory note is read or written; read: reads only, a block the model writes is hidden and dropped |
X-Tandem-Probe |
in, engine | off takes the request out of the probe snapshots |
X-Tandem-User |
in, engine | a proxy's user name for the memory notes, honoured from loopback or QWEN38_MEMORY_PROXIES |
X-Tandem-Relay-Id |
in, engine | a proxy's id for a relayed request, for the live counts; a relayed request without X-Tandem-User touches no memory |
X-Tandem-Parent |
in, engine | the request a call of TandemLLM to itself serves; the row becomes client kind internal |
X-Tandem-Worker |
in, engine | 1 on an agent's worker request: its session comes back, so it is never a guest, and its rows go to the NVMe rather than away while it waits |
X-Tandem-Parent-Session |
in, engine | a worker's parent session (the chat's): for 600 s after the worker's last request the parent's rows go to the NVMe rather than away, and its return tries slot 0 first |
X-Tandem-Regenerate |
in, engine | 1 serves the request as a regenerate (sampled, a fresh seed); 0 turns the detection off for it |
X-Tandem-Images-Left-Out |
out, engine | how many earlier images were left out of the prompt |
X-Tandem-Wait |
out, engine | memory, on a response that waited for memory before its prefill; its stream says so every 15 s |
X-Tandem-Wait-For-Memory |
in, engine | 1: a request not streamed waits for memory up to QSE_MEM_WAIT_S, as wait_for_memory: true in the body does, instead of a 503 after the queue timeout |
X-Engine-Stop |
out, engine | pattern-stop(<label>) on a response without a stream that a stop pattern ended |
x-session-id |
in, both | opencode's session id: the router's session and the caches' session key |
X-Conversation-Id |
in, both | a conversation id for clients that send no session field; read after the body's own ids |
X-OpenWebUI-Chat-Id |
in, both | Open WebUI's chat id (a connection header with {{CHAT_ID}}): the router's session for that chat |
X-QSE-Client |
in, engine | dashboard marks the dashboard's own requests (client kind dashboard) |
X-Real-IP |
in, both | set by a reverse proxy; a request that carries it is never trusted as local; the sign-in's rate limit counts per address |
X-Forwarded-For |
in, both | as X-Real-IP: a request that carries it is never trusted as local |
X-Forwarded-Proto |
in, both | https makes the session cookie Secure, and the notes' {tandem_url} https |
X-Forwarded-Host |
in, engine | the host the notes' {tandem_url} names, unless QWEN38_TANDEM_URL sets it |
X-Decider-Deadline-Ms |
in, engine | how long a decision request waits for the idle window (default 30,000, at most 120,000) |
X-Decider-Window |
in, engine | in_decode is refused: decisions run in the idle window only |
X-Accel-Buffering |
out, both | no, on every stream, so nginx sends each event at once |
X-Content-Type-Options |
out, both | nosniff on the dashboard's static files |
Authorization |
in, both | Bearer: the admin token, the metrics token, the collect token or a proxy user key, by route; the engine's OpenAI routes ignore it and file the request under a hash of it |
User-Agent |
in, both | the client kind in the ledger: open-webui, opencode, openai-sdk, curl, other (a Hermes-shaped body: hermes) |
Last-Event-ID |
in, both | resumes the log stream after that line |
Retry-After |
out, both | with a 503 (queue full, memory gate, draining, a full backend), a 429, or a decision model that is loading |
WWW-Authenticate |
out, both | Bearer, with a 401 |
Set-Cookie |
out, both | qse_dash, the dashboard session: HttpOnly, SameSite=Strict, Secure behind https |
X-Tandem-Pair-Key |
in, engine | the pairing window key (run/remote/pair-local.key): tandem pair opens and closes a window with it, from the machine itself |
Errors¶
Errors are OpenAI-shaped:
{"error": {"message": "max_tokens must be at least 1", "type": "invalid_request_error",
"param": "max_tokens", "code": null}}
A 400 names the field or the message part it refuses in param. A refusal the client can retry (a full queue, the memory gate, a server that drains) has the type server_busy, the status in code, and a Retry-After header. A refusal after a stream opened (a queue timeout, a prompt too long, the memory gate) ends the stream with a chunk whose finish_reason is "error" and which carries the error object, then [DONE]. The decision routes keep their own body, {"type": "error", "error": {"type", "message"}}. The tables below list every status and type: the first explains the common ones, the second lists every other type the code answers, with the code's own message.
The errors explained¶
| status | error.type or code | when |
|---|---|---|
400 |
invalid_request_error |
a request the engine will not serve as asked: error.param names the field (messages, a role, max_tokens, tools, an image part, a prompt too long, a body that is not JSON or not an object) |
400 |
bad_request |
a dashboard or conductor query that does not parse (a level, a range, a number) |
400 |
stage_text_only |
a request with images on the tail stage of a split model (--stage), as error.code beside invalid_request_error |
400 |
confirm_required |
a setup reset without the confirmation its level asks for (true, or the typed word) |
401 |
unauthorized |
a guarded route without the token or the session it needs, once an admin password is set; a wrong collect token |
403 |
local_only, window_key |
opening or closing a pairing window (POST /v1/engine/pair/open, /close) from another machine, or without the window key |
404 |
not_found |
no such route, or a route whose part is off on this server (the conductor, the proxy, the decision models) |
404 |
not_on_engine |
a route TandemApp serves (the conductor, its library, memory and Laya, the backends) on an engine where no extension serves it |
404 |
no_pairing |
a pairing route on a process that serves no pairing |
404 |
no_ui |
the page, on every engine: TandemApp's controller serves it |
404 |
model_not_found |
a model id no backend serves, or one two backends serve (the message names the prefixed forms) (code model_not_found) |
409 |
busy, not_loaded, role_off |
an action that needs a role or a decision model in another state, or a role the roles document switches off (role_off) |
409 |
stage_front |
any route but /health on the front stage of a split model (--stage): requests go to the tail stage |
422 |
invalid_request_error |
a value the schema refuses: a config field (error.field), a field an environment variable locks (error.variable), a decision request's fields |
429 with Retry-After |
server_busy |
a request without a stream that waited --queue-timeout for the engine |
429 with Retry-After |
too_many |
a fifth log or live stream at once; a sixth failed sign-in in a minute |
429 with Retry-After |
rate_limited |
/v1/tokenize past its 240 calls a minute from one address; a pairing route past 20 a minute from one address |
500 |
internal_error |
a bug: the message is the exception's; the request is in the log |
500 |
internal |
a PATCH /v1/engine/settings that raised: the message is the exception's; the call is journaled |
500 |
extension_error |
a route of the extension that raised: the extension's bug, in the engine's log |
501 |
none | OPTIONS on any path but the Laya routes |
503 with Retry-After |
server_busy |
the queue is full, the server is draining, or the memory gate or the admission by bytes cannot make room |
503 with Retry-After |
memory_wait_timeout |
a request that waited for memory to its bound: --queue-timeout without a stream, QSE_MEM_WAIT_S streamed or with wait_for_memory; a stream that opened while it waited ends with it as an error event (type server_busy, Retry-After: 30) |
503 with Retry-After |
setup_mode |
any route but the dashboard, /health, /v1/models and /v1/setup/* on a box in setup mode (Retry-After: 60) |
503 with Retry-After |
warming, loading, refused, failed, not_loaded, busy_decoding |
a decision request while the model loads, was refused memory, failed, or the engine kept decoding past the deadline |
503 with Retry-After |
identity_pending |
the replica identity while the engine computes it, once after the start (Retry-After: 1) |
503 with Retry-After |
stage_down |
a completion on the tail stage of a split model while its link to the front stage is down (Retry-After: 5); a request in flight when the link fails ends with 500 and the same word in its message |
503 with Retry-After |
engine_failed |
a completion after the engine's CUDA context was lost: the sentence says so, the watchdog restarts the engine (Retry-After: 60) |
503 with Retry-After |
provider_unavailable |
the status responder while the engine is stopped, starting or held (code engine_<state>) |
529 |
overloaded_error |
a decision request the runtime could not take |
Every other error type the code answers, with its status and the code's own message ({...} is a value)¶
| error.type or code | status | message | where |
|---|---|---|---|
active_profile |
409 | {...} is the running setup: switch to another model first, or remove it after the switch | server/setup/remove.py |
active_unknown |
409 | . | server/setup_api.py |
browser_only |
400 | level 1 (the setup progress and Wren's chat) is the browser's own: the page clears it | server/setup_api.py |
invalid_combination |
422 | server/setup/estimate.py |
|
invalid_level |
422 | level is 2, 3, 4 or 5 | server/setup_api.py |
invalid_request |
422 | remove takes profiles, confirm and dry_run | server/setup/remove.py, server/setup_api.py |
invalid_selection |
422 | the body must be a JSON object | server/setup/estimate.py, server/setup_api.py |
invalid_split |
422 | split.{...}: | server/setup/split.py |
invalid_value |
422 | {...}.{...}: | server/setup/estimate.py |
job_running |
409 | job {...} runs; one setup job at a time | server/setup_api.py |
not_installed |
404 | {...} has no weights on this box | server/setup/remove.py |
not_retryable |
409 | job {...} is {...}: only a failed or rolled back job is tried again | server/setup_api.py |
not_running |
409 | server/engine_routes.py |
|
off_step |
422 | {...}: a multiple of {...}, not {...} | server/setup/estimate.py |
out_of_range |
422 | {...}: a whole number from {...} to {...} | server/setup/estimate.py |
shared |
409 | {...}'s folder is also {...}'s; remove them together | server/setup/remove.py |
timeout |
408 | the request body stopped arriving | server/app.py |
too_large |
413 | a setup request body is at most {...} KiB | server/api/http_body.py, server/setup_api.py |
unknown_component |
400 | no tool {...} for {...} | server/setup/estimate.py |
unknown_device |
400 | no device preset {...} for {...} | server/setup/estimate.py |
unknown_field |
422 | unknown field {...}; reset takes level, confirm and dry_run | server/setup/estimate.py, server/setup_api.py |
unknown_model |
400, 404 | no model {...} in models/ | server/setup/estimate.py, server/setup_api.py |
unknown_profile |
400 | no weight profile {...} for {...} | server/setup/estimate.py |
unsafe |
409 | {...} is not removed: | server/setup/remove.py |