Documentation¶
TandemEngine is an inference engine, and we measure its speed on raw /v1/completions (measurement.md, the rows in CHANGELOG.md). These pages are the whole documentation; each covers one part of the engine and names the files that implement it. Start with overview.md, then install.md.
Start here¶
| page | what it covers |
|---|---|
| overview.md | what the engine is, the test board, what runs today, the two rules, the headline numbers, the paper, and one request followed end to end |
| install.md | the setups that work, what the engine needs, a new box in setup mode, the installer and its steps, the first request, access, the small models, profiles, upgrading and uninstalling |
| faq.md | short answers to the questions people ask first |
Installing and running¶
| page | what it covers |
|---|---|
| setup.md | a new box from the command line, setup mode and the Setup API behind the Setup Guide, the model registry, and Wren |
| operations.md | running a profile, memory rules, holds, deploys, troubleshooting |
| community-weights.md | packs other people published: running one, adding yours with one JSON file, how a maintainer verifies it |
| remote-control.md | tandem remote: the control verbs over an SSH key that runs nothing else, the settings overlay and its versions, the journal |
| pairing.md | pairing TandemApp by a 6-digit code (tandem pair, the end of every install), and the engine host on the LAN (_tandem._tcp) |
| design-simple-setup.md | the design of the Setup Guide's simple mode: three pages over the same plan, estimate and install job |
How the engine works¶
| page | what it covers |
|---|---|
| architecture.md | the 27B and Flash-Next, the byte budget, the decode loop, the layers every model plugs into, a map of the code |
| quantisation.md | the checkpoint's FP8, NVFP4, the clip search, GPTQ from the BF16 release, the quality gate, the three profiles, Flash-Next's EXL3 packs |
| kernels.md | row invariance, the 4-bit and 8-bit matrix products, the recurrent-layer kernels, graphs |
| speculative-decoding.md | the block drafter, the lookup drafter, StairCut, the older length router, tree verify through recurrent layers, Flash-Next's MTP drafter |
| exactness.md | the promise about the text, the near-tie exception, and the checks behind it |
| caches.md | the resident prefix, the state store, the persistent suffix store, the swap of long chats to the SSD (session spill) |
| server.md | the OpenAI API, tool calls, structured outputs, streaming, admission, chat mode and the extension it loads |
| parallel-requests.md | serving several requests at once (--parallel N): switching it on, waiting for memory and chat priority, speed, the text, Flash-Next |
| measurement.md | the benchmark row, the teacher-forced bench, noise, the release gate |
| benchmark-report.md | the benchmark report of a running server, on your GPU: how to run it, how long it takes, what each section measures |
| quality-gate.md | the report a pull request carries: the command, what a diff needs, the suites, the verdicts, the check |
| adding-a-model.md | the adapter path a new model takes, the seams, and the tests it must pass |
| shadow-reader.md | the small model that follows each session and answers questions about it |
The models¶
| page | what it covers |
|---|---|
| flash-next.md | Qwen3.8-Flash-Next: the model, its EXL3 packs and the window each fits, loading, kernels, speculation, exactness, caches, the e4m3 KV cache, memory, serving, what is not measured |
| lfm2.5-24b-a2b.md | LiquidAI's LFM2.5-24B-A2B: the model, its profiles, its tool-call format, how the engine runs it |
| nemotron-h.md | NVIDIA's NemotronH (Nemotron 3.5 Bolt, Super VL, Lightning): the models, the EXL3, GGUF and NVFP4 profiles, LiLiCorr, how the engine runs them |
| ornith.md | Ornith-1.5-122B: the model, its pack, drafting, serving, the e4m3 KV cache, what is not measured |
| flint.md | Recursive's Flint: the model, its EXL3 and BF16 profiles, reasoning and tools, the lookup at long context, how the engine runs it |
Reference¶
| page | what it covers |
|---|---|
| reference/commands.md | every flag of the engine's server and of install.sh, the service scripts, the measurement tools |
| reference/variables.md | ops/serve.env, every QWEN38_* switch with its default and served value, the QSE_* and TANDEM_* variables, the conductor's documents |
| reference/http.md | every route of the engine's server, who may call it, every header and every error |
| reference/metrics.md | every Prometheus family, and the usage ledger's columns |
| troubleshooting.md | symptoms, causes and what to do: installing, the engine, memory, the caches |
| glossary.md | the engine's terms: anchor, round, StairCut and the rest |
| benchmarks.md | tool-eval-bench against TandemLLM: the raw and conductor paths, the settings a score must name, results on a DGX Spark |
| tool-eval-bench.md | tool-eval-bench results of 2026-10-19: the raw path at five thinking settings, four conductor profiles, hard mode |
| roadmap.md | what Flash-Next still lacks, parallel requests, other GPUs |
| threat-model.md | where untrusted input enters, the parts that matter most for security |
| labels.md | the GitHub labels of issues and pull requests, by group |
| gallery/README.md | the dashboard's pictures this README and the site use |
| CHANGELOG.md | every release |
Three more references live next to the code. contracts/engine-api-v1 is the engine's API: every route, the shapes of its reads and every Prometheus metric. contracts/engine-ext-v1 is the interface of the extension the engine loads in chat mode, and contracts/engine-ctl-v1 the control verbs of tandem remote.
The dashboard, the Chat tab, the conductor, its packages and memory notes are TandemApp's, a separate project with its own docs: TandemApp's documentation.
tools/gen_reference.py writes the reference pages' tables, and the verify price tables of speculative-decoding.md, from the code between marked blocks. python3 tools/gen_reference.py --check, which tests/test_docs_reference.py runs, fails while a table is stale or the HTTP reference lacks a route or header the code has.
Cite the paper for the decoding method and its numbers (overview.md), and the repository for the software; CITATION.cff has both.
Every number on these pages was measured on one DGX Spark, unless the page calls it arithmetic or an estimate. measurement.md says how.
This documentation is licensed under CC BY 4.0 (LICENSE). The code is licensed separately (see the README).