Freeway: self-hosted OpenAI-compatible gateway for free-tier providers

One endpoint in front of every free LLM tier, with automatic failover and no
provider key ever leaving the box.

Providers are data, not code: adding one is a JSON file plus an env var, never
a TypeScript change. Ships 12 providers pre-written; `verify-provider --apply`
derives the live model list from your own key.

- core: registry, bucketed quota meter (rolling + UTC-calendar windows),
  key-pool store, 6-step model router with diagnostic rejection reasons
- gateway: node:http, failover that commits to a stream only after 200 headers,
  SSE passthrough with usage sniffing, virtual keys with admin/inference roles
- context engine: sqlite sessions and a 5-rung refit ladder, so a 128k thread
  keeps working when it fails over to an 8k model
- cache: exact (sha256, safe-by-default) + opt-in semantic (flat Float32Array)
- guard: inbound limits, quota forecasting, limit calibration from upstream
  headers, key health, secret/PII/injection scanning
- dashboard: one self-contained HTML file, live SSE request log, token usage
- sdk: zero-dep typed client, drop-in for the OpenAI SDK

Zero runtime dependencies. 326 tests, 27-check smoke suite against a scripted
mock upstream so routing is testable without spending quota.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Aryan SinghandClaude Opus 5 committed 2026-08-16 22:41:49 -05:00
commit cb985dcf9c
86 files changed
+16350

No files matched your search

+17
View File
@@ -0,0 +1,17 @@
{
"$schema": "./_schema.json",
"id": "together",
"label": "Together AI",
"docs": "https://docs.together.ai/docs/inference-models",
"console": "https://api.together.xyz/settings/api-keys",
"priority": 45,
"baseUrl": "https://api.together.xyz/v1",
"auth": { "type": "bearer", "envKeys": ["TOGETHER_API_KEY"] },
"limits": { "rpm": 60 },
"limitsSource": "unverified",
"verifiedOn": "2026-08-16",
"modelsEndpoint": "/models",
"modelsPath": "[].id",
"notes": "Free tier is limited to models with a -Free suffix; everything else bills. Leaving models empty on purpose so discovery reports what your key can actually reach rather than guessing at a moving list.",
"models": []
}