One endpoint in front of every free LLM tier, with automatic failover and no provider key ever leaving the box. Providers are data, not code: adding one is a JSON file plus an env var, never a TypeScript change. Ships 12 providers pre-written; `verify-provider --apply` derives the live model list from your own key. - core: registry, bucketed quota meter (rolling + UTC-calendar windows), key-pool store, 6-step model router with diagnostic rejection reasons - gateway: node:http, failover that commits to a stream only after 200 headers, SSE passthrough with usage sniffing, virtual keys with admin/inference roles - context engine: sqlite sessions and a 5-rung refit ladder, so a 128k thread keeps working when it fails over to an 8k model - cache: exact (sha256, safe-by-default) + opt-in semantic (flat Float32Array) - guard: inbound limits, quota forecasting, limit calibration from upstream headers, key health, secret/PII/injection scanning - dashboard: one self-contained HTML file, live SSE request log, token usage - sdk: zero-dep typed client, drop-in for the OpenAI SDK Zero runtime dependencies. 326 tests, 27-check smoke suite against a scripted mock upstream so routing is testable without spending quota. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
4.4 KiB
Exposing Freeway safely
The gateway holds every provider key you own. Treat it as a credential store that happens to speak OpenAI, not as a stateless proxy.
Before you expose it
# 1. Inference keys — one per app, so you can revoke them individually.
node -e "console.log('fw-'+crypto.randomUUID())"
# 2. Admin key — separate, and never given to an app.
node -e "console.log('fw-admin-'+crypto.randomUUID())"
FREEWAY_KEYS=fw-app-one,fw-app-two # inference only: /v1/*
FREEWAY_ADMIN_KEY=fw-admin-… # everything: /api/*
FREEWAY_CORS_ORIGINS= # leave empty unless a browser app needs it
Then put it behind TLS. Freeway speaks plain HTTP by design — terminate with Caddy, nginx or a tunnel. A virtual key sent over cleartext to a remote host is a virtual key you have published.
The permission model
| Credential | Reaches | Notes |
|---|---|---|
| none | /healthz, / |
dashboard shell only; it still needs a key for data |
FREEWAY_KEYS entry |
/v1/* |
inference only — cannot read sessions or touch config |
FREEWAY_ADMIN_KEY |
/v1/* and /api/* |
reads stored conversations, writes provider keys |
/api/* is administrative in full: it reads conversation history, writes
provider credentials to disk, spends quota via probes, and can disable
providers. An app key must never reach it, which is why the roles are split.
With no admin key configured, /api/* is allowed on loopback (where you are
the only caller) and refused entirely on any other interface.
Defaults that change when you expose it
Binding a non-loopback address turns on inbound limits automatically, because
null everywhere is only safe when the caller can only be you:
default on 0.0.0.0 |
|
|---|---|
| per key | 120 req/min, 8 concurrent |
| per IP | 240 req/min, 16 concurrent |
Override in freeway.config.json under guard. Tighten them if one app is
meant to be small.
Two failsafes you cannot turn off: the gateway refuses to bind a public
interface with no FREEWAY_KEYS, and the runtime key-paste endpoint refuses to
accept credentials over a non-loopback socket without an admin key.
What is protected
- Provider keys never leave the process. Masked in logs,
/api/*, the dashboard and error bodies; tests assert a raw key cannot appear in any of them. - Sessions are owned. A session is claimed by a fingerprint of the key that
created it; another key gets
403 session_forbidden. - CORS is an allowlist. Empty by default,
*rejected in config. - Caller identity is a hash, not a mask. Masks are lossy and two keys can render identically — that must never decide authorization.
- Constant-time key comparison over hashes, so neither content nor length is observable by timing.
- Data at rest is 0600:
keys.json,freeway.db(conversations),cache.db(responses). - Live-log subscriptions are capped at 16 concurrent.
What is not protected — decide for yourself
- No TLS. Terminate it in front.
- The cache is shared across keys. Two keys asking the same question share an
entry. Not a content leak, but
x-freeway-cache: exactreveals that someone asked it before. Setcache.mode: "off"if that matters. /healthzis unauthenticated and reports provider counts and uptime.- Content scanning is
flag, notblock. It records secrets and PII found in prompts rather than refusing them. Setguard.scan.mode: "block"to refuse on high-severity findings. - Prompts and replies go to third-party free tiers. Nothing here changes what those providers do with them. Do not send anything through a free tier that you would not be comfortable handing to that vendor.
- The probe endpoint amplifies. One admin request can trigger dozens of upstream calls against a provider.
Reviewed and found clean
SQL injection (all statements are prepared with bound parameters) · dashboard
XSS (every interpolation escaped) · SSRF (baseUrl comes from disk, never from
a request) · prototype pollution · path traversal (ids are database keys, never
paths) · header injection (all header values sanitised to printable ASCII).
If a key leaks
- Remove it from
FREEWAY_KEYSand restart. Sessions owned by it become unreachable rather than exposed. - If it was the admin key, assume every provider key was readable — rotate them at each provider, not just here.
data/usage.jsonand the dashboard log show what it spent.