pm-aigateway

Self-hosted AI gateway powered by agentgateway — a unified, OpenAI-compatible proxy for LLM providers, MCP servers, and A2A agents.

Project goal

Provide a single API endpoint that routes LLM requests to multiple providers (OpenAI, Anthropic, etc.) with centralized auth and observability. Stateless to start; extend to MCP and A2A as needed.

Deployment targets

Target	Status
Docker Compose	current — single stateless container
Kubernetes	planned

Stack

agentgateway — the sole service; stateless, single binary in a container

No database or cache needed for the current scope.

Repository layout

.
├── CLAUDE.md
├── compose/
│   ├── docker-compose.yml
│   └── .env.example
└── config/
    └── config.yaml          # model list and routing rules

Key configuration

All provider API keys are supplied via environment variables (never committed).
Copy compose/.env.example → compose/.env and fill in values.

Required env vars:

OPENAI_API_KEY
ANTHROPIC_API_KEY

agentgateway reads config/config.yaml on startup. Restart the container to pick up changes.

Common commands

# Start
docker compose -f compose/docker-compose.yml --env-file compose/.env up -d

# Tail logs
docker compose -f compose/docker-compose.yml logs -f

# Restart after config change
docker compose -f compose/docker-compose.yml restart

# Stop
docker compose -f compose/docker-compose.yml down

Ports

Endpoint	Port
OpenAI-compatible API	4000
agentgateway UI	15000

Test

curl -s http://localhost:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o-mini",
    "messages": [{"role": "user", "content": "Hello"}]
  }' | jq .

Coding conventions

Secrets in compose/.env only — gitignored. compose/.env.example is committed.
Pin the agentgateway image tag; avoid latest in production.
Keep config/config.yaml as the single source of truth for model/routing config.

Future extensions

MCP gateway: add mcp.servers block to config.yaml to expose tools
A2A gateway: add a2a block for agent-to-agent routing
Persistence: add Postgres if spend tracking or virtual keys are needed
Kubernetes: translate compose to k8s manifests under k8s/

2.4 KiB Raw Permalink Blame History