Stéphane Tailland
91 lines
2.4 KiB
Markdown
91 lines
2.4 KiB
Markdown
# pm-aigateway
|
|
|
|
Self-hosted AI gateway powered by [agentgateway](https://agentgateway.dev) — a unified, OpenAI-compatible proxy for LLM providers, MCP servers, and A2A agents.
|
|
|
|
## Project goal
|
|
|
|
Provide a single API endpoint that routes LLM requests to multiple providers (OpenAI, Anthropic, etc.) with centralized auth and observability. Stateless to start; extend to MCP and A2A as needed.
|
|
|
|
## Deployment targets
|
|
|
|
| Target | Status |
|
|
|--------|--------|
|
|
| Docker Compose | **current** — single stateless container |
|
|
| Kubernetes | planned |
|
|
|
|
## Stack
|
|
|
|
- **agentgateway** — the sole service; stateless, single binary in a container
|
|
|
|
No database or cache needed for the current scope.
|
|
|
|
## Repository layout
|
|
|
|
```
|
|
.
|
|
├── CLAUDE.md
|
|
├── compose/
|
|
│ ├── docker-compose.yml
|
|
│ └── .env.example
|
|
└── config/
|
|
└── config.yaml # model list and routing rules
|
|
```
|
|
|
|
## Key configuration
|
|
|
|
All provider API keys are supplied via environment variables (never committed).
|
|
Copy `compose/.env.example` → `compose/.env` and fill in values.
|
|
|
|
Required env vars:
|
|
- `OPENAI_API_KEY`
|
|
- `ANTHROPIC_API_KEY`
|
|
|
|
agentgateway reads `config/config.yaml` on startup. Restart the container to pick up changes.
|
|
|
|
## Common commands
|
|
|
|
```bash
|
|
# Start
|
|
docker compose -f compose/docker-compose.yml --env-file compose/.env up -d
|
|
|
|
# Tail logs
|
|
docker compose -f compose/docker-compose.yml logs -f
|
|
|
|
# Restart after config change
|
|
docker compose -f compose/docker-compose.yml restart
|
|
|
|
# Stop
|
|
docker compose -f compose/docker-compose.yml down
|
|
```
|
|
|
|
## Ports
|
|
|
|
| Endpoint | Port |
|
|
|----------|------|
|
|
| OpenAI-compatible API | 4000 |
|
|
| agentgateway UI | 15000 |
|
|
|
|
## Test
|
|
|
|
```bash
|
|
curl -s http://localhost:4000/v1/chat/completions \
|
|
-H "Content-Type: application/json" \
|
|
-d '{
|
|
"model": "gpt-4o-mini",
|
|
"messages": [{"role": "user", "content": "Hello"}]
|
|
}' | jq .
|
|
```
|
|
|
|
## Coding conventions
|
|
|
|
- Secrets in `compose/.env` only — gitignored. `compose/.env.example` is committed.
|
|
- Pin the agentgateway image tag; avoid `latest` in production.
|
|
- Keep `config/config.yaml` as the single source of truth for model/routing config.
|
|
|
|
## Future extensions
|
|
|
|
- **MCP gateway**: add `mcp.servers` block to `config.yaml` to expose tools
|
|
- **A2A gateway**: add `a2a` block for agent-to-agent routing
|
|
- **Persistence**: add Postgres if spend tracking or virtual keys are needed
|
|
- **Kubernetes**: translate compose to k8s manifests under `k8s/`
|