6 Commits

This Branch

This Branch

All Branches

• Stop running the full pipeline on documentation commits

  A README change was building a multi-arch Docker image. Both workflows now
  ignore markdown, docs/, demo/, .gitignore and LICENSE. providers/*.json is
  deliberately not ignored: CI fails if one stops parsing, which is the one data
  change that can break a running gateway.
  
  arm64 is cross-built under QEMU and dominates the Docker job, so it is now only
  built for tagged releases; main and PRs build native amd64 only. QEMU setup is
  skipped entirely when it is not needed.
  
  Anchors would have kept the paths-ignore lists DRY, but Actions has been
  unreliable with them and a broken workflow costs more than five duplicated
  lines.
  
  Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

  Aryan SinghandClaude Opus 5 2026-08-16 23:44:01 -05:00
  e7b51c6fc9
• README: lock the tagline, and document the cache properly

  The caching section was four paragraphs that said it existed. It now explains
  what it actually is: a canonical SHA-256 key over the fields that can change an
  answer, keyed on the resolution so aliases share entries; why `safe` is the
  default and refuses to replay a temperature 0.9 call; the semantic tier's flat
  Float32Array slab, 0.92 threshold and system-prompt scoping; SSE replay of
  cached streams; why hits are never metered; and the LRU counter that replaced
  millisecond timestamps.
  
  Also states plainly what it is not — shared across virtual keys, single-process,
  no distributed tier — since that is the trade a reader needs to make a decision.
  
  Every number and behaviour in the section was checked against the built code
  before writing it down.
  
  Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

  Aryan SinghandClaude Opus 5 2026-08-16 23:41:39 -05:00
  8180cec073
• Record the dashboard, and make the page capturable

  node scripts/record-dashboard.mjs drives the gateway with real requests and
  screenshots the real dashboard between them with headless Chrome. Screen
  recording was the obvious route and the wrong one: it would photograph whatever
  else happened to be on screen and needs a permission grant. Headless captures
  only the page, needs no permission, and writes frames straight to disk.
  
  The dashboard never finished loading under a headless browser, because it holds
  an SSE connection open and polls on an interval — so ?nolive=1 now renders once
  and goes quiet. That also makes the page screenshot-testable in general.
  
  Chrome does not reliably exit after --screenshot, so the recorder waits for the
  PNG to appear and stop growing rather than guessing a sleep, and treats profile
  cleanup as best-effort since Chrome keeps flushing after SIGKILL.
  
  Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

  Aryan SinghandClaude Opus 5 2026-08-16 23:18:22 -05:00
  6c41b6d804
• Add a recorded demo, and the script that produces it

  scripts/demo.sh is a real walkthrough: it starts the gateway, routes model:auto,
  resolves four aliases to four different models, shows a cache hit costing zero
  quota, recalls a name across a server-side session, 404s an unknown model with
  suggestions, and prints where the tokens went. Nothing is staged — every line is
  output the gateway actually produced against a live provider.
  
  scripts/record-demo.html turns a captured transcript into video frames. ffmpeg
  on macOS ships without freetype, so it can encode frames but cannot draw text;
  the browser can do both, so frames are rasterised on a canvas and posted to a
  local server for ffmpeg to assemble.
  
  Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

  Aryan SinghandClaude Opus 5 2026-08-16 22:53:45 -05:00
  eeb5cad4ce
• Dashboard token usage and response inspection; fix Docker CI smoke

  The dashboard now answers "where did my tokens go" without opening the log:
  totals, per-provider and per-model breakdown, and cache hits shown as the
  requests that cost nothing. Each log row expands to the actual prompt and
  response, truncated to keep the ring buffer flat.
  
  CI's Docker smoke step supplied no virtual keys, so the container correctly
  refused to bind 0.0.0.0 and the job failed. The failsafe was right and the
  workflow was wrong; it now supplies keys and additionally asserts that an
  unauthenticated call gets 401 and an inference key gets 403 on /api, so an
  image that lost its auth boundaries cannot be published green.
  
  Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

  Aryan SinghandClaude Opus 5 2026-08-16 22:41:49 -05:00
  b85542abc6
• Freeway: self-hosted OpenAI-compatible gateway for free-tier providers

  One endpoint in front of every free LLM tier, with automatic failover and no
  provider key ever leaving the box.
  
  Providers are data, not code: adding one is a JSON file plus an env var, never
  a TypeScript change. Ships 12 providers pre-written; `verify-provider --apply`
  derives the live model list from your own key.
  
  - core: registry, bucketed quota meter (rolling + UTC-calendar windows),
    key-pool store, 6-step model router with diagnostic rejection reasons
  - gateway: node:http, failover that commits to a stream only after 200 headers,
    SSE passthrough with usage sniffing, virtual keys with admin/inference roles
  - context engine: sqlite sessions and a 5-rung refit ladder, so a 128k thread
    keeps working when it fails over to an 8k model
  - cache: exact (sha256, safe-by-default) + opt-in semantic (flat Float32Array)
  - guard: inbound limits, quota forecasting, limit calibration from upstream
    headers, key health, secret/PII/injection scanning
  - dashboard: one self-contained HTML file, live SSE request log, token usage
  - sdk: zero-dep typed client, drop-in for the OpenAI SDK
  
  Zero runtime dependencies. 326 tests, 27-check smoke suite against a scripted
  mock upstream so routing is testable without spending quota.
  
  Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

  Aryan SinghandClaude Opus 5 2026-08-16 22:41:49 -05:00
  cb985dcf9c