scripts/demo.sh is a real walkthrough: it starts the gateway, routes model:auto,
resolves four aliases to four different models, shows a cache hit costing zero
quota, recalls a name across a server-side session, 404s an unknown model with
suggestions, and prints where the tokens went. Nothing is staged — every line is
output the gateway actually produced against a live provider.
scripts/record-demo.html turns a captured transcript into video frames. ffmpeg
on macOS ships without freetype, so it can encode frames but cannot draw text;
the browser can do both, so frames are rasterised on a canvas and posted to a
local server for ffmpeg to assemble.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The dashboard now answers "where did my tokens go" without opening the log:
totals, per-provider and per-model breakdown, and cache hits shown as the
requests that cost nothing. Each log row expands to the actual prompt and
response, truncated to keep the ring buffer flat.
CI's Docker smoke step supplied no virtual keys, so the container correctly
refused to bind 0.0.0.0 and the job failed. The failsafe was right and the
workflow was wrong; it now supplies keys and additionally asserts that an
unauthenticated call gets 401 and an inference key gets 403 on /api, so an
image that lost its auth boundaries cannot be published green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
One endpoint in front of every free LLM tier, with automatic failover and no
provider key ever leaving the box.
Providers are data, not code: adding one is a JSON file plus an env var, never
a TypeScript change. Ships 12 providers pre-written; `verify-provider --apply`
derives the live model list from your own key.
- core: registry, bucketed quota meter (rolling + UTC-calendar windows),
key-pool store, 6-step model router with diagnostic rejection reasons
- gateway: node:http, failover that commits to a stream only after 200 headers,
SSE passthrough with usage sniffing, virtual keys with admin/inference roles
- context engine: sqlite sessions and a 5-rung refit ladder, so a 128k thread
keeps working when it fails over to an 8k model
- cache: exact (sha256, safe-by-default) + opt-in semantic (flat Float32Array)
- guard: inbound limits, quota forecasting, limit calibration from upstream
headers, key health, secret/PII/injection scanning
- dashboard: one self-contained HTML file, live SSE request log, token usage
- sdk: zero-dep typed client, drop-in for the OpenAI SDK
Zero runtime dependencies. 326 tests, 27-check smoke suite against a scripted
mock upstream so routing is testable without spending quota.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
