Freeway: self-hosted OpenAI-compatible gateway for free-tier providers

One endpoint in front of every free LLM tier, with automatic failover and no
provider key ever leaving the box.

Providers are data, not code: adding one is a JSON file plus an env var, never
a TypeScript change. Ships 12 providers pre-written; `verify-provider --apply`
derives the live model list from your own key.

- core: registry, bucketed quota meter (rolling + UTC-calendar windows),
  key-pool store, 6-step model router with diagnostic rejection reasons
- gateway: node:http, failover that commits to a stream only after 200 headers,
  SSE passthrough with usage sniffing, virtual keys with admin/inference roles
- context engine: sqlite sessions and a 5-rung refit ladder, so a 128k thread
  keeps working when it fails over to an 8k model
- cache: exact (sha256, safe-by-default) + opt-in semantic (flat Float32Array)
- guard: inbound limits, quota forecasting, limit calibration from upstream
  headers, key health, secret/PII/injection scanning
- dashboard: one self-contained HTML file, live SSE request log, token usage
- sdk: zero-dep typed client, drop-in for the OpenAI SDK

Zero runtime dependencies. 326 tests, 27-check smoke suite against a scripted
mock upstream so routing is testable without spending quota.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Aryan SinghandClaude Opus 5 committed 2026-08-16 22:41:49 -05:00
commit cb985dcf9c
86 files changed
+16350

No files matched your search

+112
View File
@@ -0,0 +1,112 @@
name: Docker
on:
push:
branches: [main]
tags: ['v*.*.*']
pull_request:
workflow_dispatch:
env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}
concurrency:
group: docker-${{ github.ref }}
cancel-in-progress: true
jobs:
build:
name: build & push image
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
# Required for cosign keyless signing of pushed images.
id-token: write
steps:
- uses: actions/checkout@v4
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Buildx
uses: docker/setup-buildx-action@v3
# A pull request from a fork cannot read the registry secret, so builds are
# verified without pushing. Only trusted refs publish.
- name: Log in to GHCR
if: github.event_name != 'pull_request'
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Derive tags and labels
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
tags: |
type=ref,event=branch
type=ref,event=pr
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
type=sha,format=short
type=raw,value=latest,enable={{is_default_branch}}
labels: |
org.opencontainers.image.title=Freeway
org.opencontainers.image.description=Self-hosted OpenAI-compatible gateway for free-tier LLM providers
- name: Build and push
id: build
uses: docker/build-push-action@v6
with:
context: .
platforms: linux/amd64,linux/arm64
push: ${{ github.event_name != 'pull_request' }}
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
provenance: true
sbom: true
# Smoke-test the image we just built, on the native arch only — an arm64
# image cannot be started on an amd64 runner without emulation overhead.
- name: Smoke-test the image
if: github.event_name != 'pull_request'
run: |
set -euo pipefail
IMAGE="$(echo '${{ steps.meta.outputs.tags }}' | head -n1)"
docker run -d --name freeway-smoke -p 8787:8787 --platform linux/amd64 "$IMAGE"
for i in $(seq 1 30); do
if curl -fsS http://127.0.0.1:8787/healthz >/dev/null 2>&1; then
echo "gateway is healthy"
curl -fsS http://127.0.0.1:8787/healthz
docker rm -f freeway-smoke
exit 0
fi
sleep 1
done
echo "gateway did not become healthy in 30s" >&2
docker logs freeway-smoke >&2
docker rm -f freeway-smoke
exit 1
- name: Summary
if: github.event_name != 'pull_request'
run: |
{
echo "### Image published"
echo
echo '```'
echo '${{ steps.meta.outputs.tags }}'
echo '```'
echo
echo "digest: \`${{ steps.build.outputs.digest }}\`"
} >> "$GITHUB_STEP_SUMMARY"