freeway/docs/ADDING-A-PROVIDER.md

Adding a provider

You do not edit TypeScript to add a provider. You write one JSON file and set one environment variable. If you find yourself opening a .ts file, something has gone wrong — see When you think you need code.

This document is written to be usable by a person or by an AI agent working unattended, including for a provider nobody here has ever heard of.

---

The 60-second version

node scripts/new-provider.mjs acme --url https://api.acme.ai/v1 --env ACME_API_KEY
export ACME_API_KEY=sk-...
node scripts/verify-provider.mjs acme

The verifier tells you what is wrong, what models it found, and what the real rate limits are. Paste its suggestion into the file, run it again until it says PASS, and the provider is live.

---

The loop

  new-provider.mjs  ──▶  providers/acme.json  ──▶  verify-provider.mjs
                              ▲                            │
                              └────── paste suggestion ◀────┘

verify-provider.mjs is the feedback signal. It checks five things in order and stops at the first that fails, so you always get one actionable problem rather than a wall of noise:

Step	Checks	Typical failure
1. file	parses, matches the schema	limits.rpm: expected a positive number or null
2. environment	${ENV} resolves, a key exists	no API key found (set one of: ACME_API_KEY)
3. live probe	auth accepted, /models readable	auth invalid — GET /models → 401
4. limits	headers vs declared limits	rpm: file says 60, provider reports 30
5. suggestion	prints paste-ready JSON	—

Flags: --validate also sends a one-token request per model (spends real quota, tells you which models the key can genuinely reach). --json for machine output — use this if you are an agent.

---

The fields that actually matter

Everything else has a working default. Full contract: providers/_schema.json.

baseUrl — with ${ENV_VAR} interpolation

"baseUrl": "https://api.groq.com/openai/v1"

// Some providers put an account id in the path. Interpolation is why that
// stays data instead of becoming a special case in code:
"baseUrl": "https://api.cloudflare.com/client/v4/accounts/${CLOUDFLARE_ACCOUNT_ID}/ai/v1"

A missing variable is not a crash. The provider renders on the dashboard as a card reading missing environment variable: CLOUDFLARE_ACCOUNT_ID.

auth — four shapes cover everything so far

{ "type": "bearer", "envKeys": ["ACME_API_KEY"] }                     // Authorization: Bearer <key>
{ "type": "header", "header": "x-api-key", "envKeys": ["ACME_KEY"] }  // a named header
{ "type": "query",  "query": "key",        "envKeys": ["ACME_KEY"] }  // ?key=<key>
{ "type": "none",   "envKeys": [] }                                   // local Ollama, no credential
{ "type": "bearer", "prefix": "", "envKeys": ["ACME_KEY"] }            // bare token, no "Bearer "

Key pools. One variable may hold several comma-separated keys:

ACME_API_KEY=sk-aaa,sk-bbb,sk-ccc

Quota is tracked per key. A 429 on one benches only that key and the request retries on the next; a 5xx benches the whole provider because every key would fail identically. Commas are the grammar precisely so this works from a plain docker run -e with no extra syntax.

limits — null is a real answer

"limits": { "rps": 1, "tpm": 500000, "tpmo": 1000000000 }

Key	Window
rps rpm tpm	rolling
rpd tpd creditsPerDay	calendar day, resets 00:00 UTC
tpmo	calendar month

Do not guess a number. null means "unknown or unlimited" and is always the right answer when you have not verified it. The guard layer overwrites nulls with values read from the provider's own rate-limit response headers, so an honest null becomes accurate on its own. A wrong number does not — it silently misroutes, and nothing ever corrects it.

Set "limitsSource": "unverified" when you are unsure. The dashboard badges it and everything still works. Mistral's file does exactly this, because Mistral stopped publishing free-tier figures.

models — a few verified beats a long guess

{
  "id": "acme-turbo",              // exactly as the provider spells it; slashes are fine
  "alias": ["fast", "cheap"],      // several providers sharing an alias is what enables failover
  "context": 128000,               // null if unpublished
  "caps": ["chat", "tools", "json"],
  "priority": 10,                  // lower is tried first
  "credits": { "perMTokIn": 2457, "perMTokOut": 18252 }  // only for credit-metered providers
}

caps drives routing and is not decorative:

Cap	Effect
chat	eligible for /v1/chat/completions
embed	eligible for /v1/embeddings — and never for chat
tools	required when the request body has tools
json	required when response_format.type is json_object
vision	required when a message contains an image part
reasoning code long-context	informational, filterable in the dashboard

Leave models: [] and let discovery populate it. That is better than a stale list — see together.json and github-models.json.

modelsPath — for non-OpenAI response shapes

"modelsEndpoint": "/models",
"modelsPath": "data[].id"     // {"data":[{"id":"..."}]}   ← OpenAI standard
"modelsPath": "[].id"         // [{"id":"..."}]            ← GitHub Models
"modelsPath": "models[].name" // {"models":[{"name":...}]}
"modelsPath": "result[].name" // {"result":[{"name":...}]}

Set "modelsEndpoint": null when a provider has no catalog endpoint — Cloudflare's OpenAI-compatible surface exposes /chat/completions and /embeddings but not /models.

dropParams — params a provider rejects

"dropParams": ["frequency_penalty", "presence_penalty"]

Find them empirically: send a request, read the 400, add the offending field.

---

Worked example: a provider that does not exist

Say Acme AI offers a free tier. OpenAI-compatible, key in x-api-key, account id in the URL, and a /models endpoint returning {"result":[...]}.

1. Scaffold.

node scripts/new-provider.mjs acme \
  --url 'https://api.acme.ai/v1/org/${ACME_ORG}' \
  --auth header --header x-api-key --env ACME_API_KEY

2. Fix the response shape in providers/acme.json:

"modelsPath": "result[].name"

3. Verify.

export ACME_API_KEY=sk-live-xxxx
export ACME_ORG=org_123
node scripts/verify-provider.mjs acme

1. file
  ✓ providers/acme.json parses
2. environment
  ✓ baseUrl resolves to https://api.acme.ai/v1/org/org_123
  ✓ 1 key in the pool: sk-l…xxxx (ACME_API_KEY)
3. live probe
  ✓ auth accepted (142ms)
  ✓ discovery found 3 models via /models → result[].name
4. limits
  ✓ observed from response headers: rpm=60, tpm=40000
  + rpm: file says null, provider reports 60 — consider recording it
5. suggested edits
  { "models": [...], "limits": {...}, "limitsSource": "observed" }

PASS — 0 problems, 0 warnings

4. Paste the suggestion in, add aliases and capabilities the verifier could not know, and re-run with --validate to confirm each model really answers.

5. Use it. No restart needed:

curl -X POST localhost:8787/api/reload
curl localhost:8787/v1/chat/completions -H 'content-type: application/json' \
  -d '{"model":"acme/acme-turbo","messages":[{"role":"user","content":"hi"}]}'

---

Common failures

Symptom	Cause
Provider missing from the dashboard entirely	filename starts with _ (reserved), or the JSON failed to parse — check the red banner, it names the file and the syntax error
unknown field — did you mean "baseUrl"?	a typo like baseURL. Warnings are surfaced, never silently ignored
Card shows no API key found	env var unset, or set in a shell the gateway did not inherit
Card shows missing environment variable: X	a ${X} in baseUrl/headers did not resolve. Pasting a key will not fix this
modelsPath matched nothing	wrong response shape — curl the endpoint and read it
Model never gets routed to	missing a required cap, or context too small for minContext. The 429 body's blocked[] says which
Everything 429s immediately	a limit is set too low. Set it to null and let calibration find the truth

Every rejection reason appears in the blocked array of a 429 response and in the dashboard. You should never have to guess why a provider was skipped.

---

When you think you need code

Nearly always, you do not:

"I need code because…"	Actually
the URL contains an account id	${ENV_VAR} in baseUrl
auth is a custom header	auth.type: "header"
the token has no Bearer prefix	auth.prefix: ""
/models is shaped differently	modelsPath
there is no /models	modelsEndpoint: null
it rejects a parameter	dropParams
it needs an extra header	headers: {} (supports ${ENV} too)
it bills in credits, not tokens	limits.creditsPerDay + per-model credits
I have several keys	comma-separate them in one env var

A genuinely new wire format — not OpenAI-compatible at all — is the one case that needs a new adapter. That field exists and is validated, but only "openai" is implemented. Adding one is a real code change; everything above is not.

---

Repo conventions that will bite you

• No TypeScript parameter properties. constructor(private readonly x: T) throws ERR_UNSUPPORTED_TYPESCRIPT_SYNTAX under Node's strip-only type stripping. Declare fields explicitly and assign in the constructor.
• Zero runtime dependencies. Dev deps are typescript and @types/node. Stop and ask before adding anything.
• noUncheckedIndexedAccess is on: indexing returns T | undefined. Use guards or ?? fallback, never !.
• Never log a key. Everything goes through mask() in packages/core/src/util.ts.
• Tests are node:test. Point them at scripts/mock-upstream.js, never at a real provider — that is what it is for.