AI Gateway

A single-binary LLM gateway that exposes a unified OpenAI-compatible and Anthropic-compatible API in front of multiple upstream providers. Ship one endpoint; route to any backend.

Client (OpenAI SDK / Anthropic SDK / curl)
        │
        ▼
  AI Gateway :8080
  ├── /v1/chat/completions   ← OpenAI protocol
  ├── /v1/messages           ← Anthropic protocol
  ├── /v1/models             ← unified model list
  ├── /admin/                ← web admin panel
  └── /metrics               ← Prometheus scrape
        │
        ▼
  Upstream providers (OpenAI, Anthropic, custom endpoints, …)

Features

• Dual protocol — accepts both OpenAI (/v1/chat/completions) and Anthropic (/v1/messages) requests; auto-converts between the two when the upstream speaks a different protocol.
• Multi-provider routing — route by model name across any number of configured providers.
• Web admin panel — add, edit, and test providers; view request logs and statistics; no restart required.
• Hot config reload — edit config.yaml; changes apply within ~200 ms without restarting.
• Prometheus metrics — request counts, latency histograms, token counters at /metrics.
• SQLite storage — request logs and aggregate stats stored locally; zero external dependencies.
• Single binary — the Next.js admin panel is embedded; deploy one file.

Quick start

# 1. Set API keys
export OPENAI_API_KEY=sk-...
export ANTHROPIC_API_KEY=sk-ant-...

# 2. Build everything (frontend + Go binary)
make all

# 3. Run
./gateway --config config/config.example.yaml

# Admin panel → http://localhost:8080/admin/
# API        → http://localhost:8080/v1/

Building

Command	What it does
make all	Build frontend + copy assets + compile binary (full build)
make build	Compile Go binary only (reuses last frontend build)
make web-build	Build Next.js frontend only
make run	Build + run with config/config.yaml
make run-dev	go run without a pre-build step
make test	Run all Go tests
make docker-build	Build Docker image

Prerequisites: Go 1.24+, Node.js 18+.

Configuration

Copy the example and edit:

cp config/config.example.yaml config/config.yaml

Minimal config:

server:
  port: 8080
  admin_key: "change-me"

providers:
  - name: openai
    type: openai_compatible
    base_url: https://api.openai.com
    api_key: "${OPENAI_API_KEY}"
    timeout: 120
    models:
      - id: gpt-4o
        capabilities: [chat]

Provider types

Type	Protocol sent upstream
openai_compatible	OpenAI
anthropic_compatible	Anthropic
custom_openai	OpenAI (explicit protocol: openai)
custom_anthropic	Anthropic (explicit protocol: anthropic)

API key formats

api_key: "sk-hard-coded"           # literal (avoid in production)
api_key: "${OPENAI_API_KEY}"       # env var reference
api_key: "${file:/run/secrets/oai}"# file contents
api_key_env: "OPENAI_API_KEY"      # alternative: name-only reference

Env-var and file references are re-resolved every 60 seconds — Kubernetes Secret updates are picked up automatically.

CLI flags

--config <path>     Config file path (default: config/config.yaml)
--validate          Validate config and exit (0 = ok, 1 = errors)
--print-config      Print resolved config with secrets masked, then exit

Endpoints

Upstream (client-facing)

Method	Path	Description
POST	/v1/chat/completions	OpenAI-format chat
POST	/v1/messages	Anthropic-format messages
GET	/v1/models	List all configured models

Admin (requires Authorization: Bearer <admin_key>)

Method	Path	Description
GET/POST	/admin/api/providers	List / create providers
GET/PUT/DELETE	/admin/api/providers/{name}	Read / update / delete
POST	/admin/api/provider-test	Test provider reachability
GET	/admin/api/stats/overview	Aggregate stats
GET	/admin/api/stats/providers	Per-provider stats
GET	/admin/api/stats/requests	Request log
GET/POST	/admin/api/logs	Log config / cleanup
GET	/admin/api/audit	Audit log

Operational (unauthenticated)

Path	Description
/healthz	Liveness probe
/readyz	Readiness probe (503 until SQLite is ready)
/metrics	Prometheus scrape target

Docker

docker run -d --name gateway \
  -p 8080:8080 \
  -v $PWD/config:/app/config:ro \
  -v gateway-data:/app/data \
  -e OPENAI_API_KEY \
  -e GATEWAY_ADMIN_KEY \
  ai-gateway --config /app/config/config.yaml

Documentation

Doc	Contents
docs/usage.md	Sending requests, client setup, admin panel walkthrough
docs/api.md	Full API reference with request/response shapes
docs/deployment.md	Bare-metal, Docker, Kubernetes, TLS
docs/design.md	Architecture and design decisions

License

MIT