freeway/CLAUDE.md

Freeway — notes for whoever works on this next

Self-hosted, OpenAI-compatible gateway in front of free-tier LLM providers. One endpoint, automatic failover, live quota visibility, and no real provider key ever leaving the box.

The one rule

Providers are data, not code. Adding a provider is one JSON file in providers/ plus one env var. It must never require a TypeScript edit.

If you are adding a provider, read docs/ADDING-A-PROVIDER.md and use the tools — do not hand-write the file:

node scripts/new-provider.mjs <id> --url <baseUrl> --env <ENV_VAR>
node scripts/verify-provider.mjs <id>            # your feedback loop; --json for machine output

The schema is the contract: providers/_schema.json.

Layout

packages/core/      registry, meter, store, router, guard, context, cache, probe
packages/gateway/   node:http server + public/index.html dashboard
packages/sdk/       @freeway/sdk client
providers/          the registry — one JSON file per provider, _-prefixed files skipped
scripts/            mock-upstream.js, verify-provider.mjs, new-provider.mjs

Hard constraints

• Node 22+, TypeScript, ESM, npm workspaces. Zero runtime dependencies. Dev deps are typescript and @types/node only. If you think you need a runtime dep, stop and ask.
• No TypeScript parameter properties. constructor(private readonly x: T) throws ERR_UNSUPPORTED_TYPESCRIPT_SYNTAX under Node's strip-only type stripping, which is how the tests and verify-provider.mjs run .ts sources directly. Declare fields, assign in the constructor.
• strict + noUncheckedIndexedAccess: indexing returns T | undefined. Guard or ?? fallback; never !. No any without a comment justifying it.
• Tests are node:test only. Run them against scripts/mock-upstream.js, never a real provider.
• Never log a key. Everything goes through mask() in packages/core/src/util.ts. There are tests asserting raw keys cannot appear in errors or /api/*.
• Comments explain why. Skip comments that restate the code.

Things that look wrong but are deliberate

• null limits. null means "unknown or unlimited" and beats a guessed number, because the guard layer overwrites nulls with values observed from upstream rate-limit headers. A wrong number never self-corrects. Mistral ships with null rps/tpm because Mistral stopped publishing them.
• Day and month limits are calendar windows, not rolling. Free tiers reset at 00:00 UTC. A rolling 24h window would block for hours after quota refilled.
• Model ids are split on the first slash only. cloudflare/@cf/meta/llama-3.1-8b-instruct is provider cloudflare, model @cf/meta/llama-3.1-8b-instruct.
• Failover commits after 200 headers, before piping the body. Once a byte of a stream reaches the client you cannot fail over, so the commit point sits exactly there.
• A 400 is never retried and benches nothing. It fails identically everywhere. A 401/403/429 benches the key; a 5xx or network error benches the provider.
• Per-model credit rates, not one per provider. Output tokens cost ~8x input and a 70B ~10x an 8B, so a single flat number is wrong by an order of magnitude.

Commands

npm run build                       # tsc --build, must be clean
npm test                            # node:test across all workspaces
npm run smoke                       # gateway against the mock upstream, every route
node scripts/mock-upstream.js --port 9101 --name alpha   # a fake provider you control

Where things live

Want to change…	File
provider file format	packages/core/src/spec.ts + providers/_schema.json
quota accounting	packages/core/src/meter.ts
which provider serves a request	packages/core/src/router.ts
failover / streaming behaviour	packages/gateway/src/proxy.ts
routes	packages/gateway/src/server.ts
the dashboard	packages/gateway/public/index.html (one file, no build step, no CDN)