Stéphane Tailland
11edfa8ed1
The audit trail recorded metadata only for this gateway, so the prompts sent to the LLM and the completions coming back — the one thing worth auditing on an AI gateway — were never written down. The five business domains have captured bodies since they were built; this gateway declares its own copy of the observability plugins and never got them. Adds the same pre-function/post-function pair the domains use, bound to the proxy service, and the custom_fields_by_lua that reads them back in the log phase. If the capture plugins are ever removed the fields go nil and are silently omitted, so the log degrades to metadata rather than breaking. The response accumulator earns its keep here more than elsewhere: LLM answers stream as SSE, so body_filter runs once per chunk and the complete answer only exists after the last one. Still dev-only, for the same reason as the rest of this file: it puts prompt and completion content in the log stream. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
platform-llm
Platform-owned llm bounded context — Gateway infrastructure and the
shared LLM proxy in one repo. Never split per functional domain, and never
split between infra/routing here either: both are owned by the same
(platform) team, unlike catalog's dataplane-catalog/catalog-api split
(feature team vs. platform team, a real access boundary that doesn't exist
for this domain).
What it does
Fronts OpenRouter (https://openrouter.ai) via one proxy route
(/llm/v1/chat/completions) — the client's own model field selects among
any OpenRouter-hosted model, not just a fixed set Kong picks.
Active policies:
ai-rate-limiting-advanced, token-based, both global (across all callers) and per-client (per authenticated consumer).ai-prompt-guard(regex) andai-semantic-prompt-guard(embedding-based, backed by Redis Stack) for prompt-injection/jailbreak safeguarding.- Two-tier auth:
key-auth(static, shared key) for chatbot UIs that can't do OIDC on their outbound LLM connection (LibreChat and similar tools — structural limitation of that whole category, not fixable client-side),openid-connectfor agents built in-house. Either succeeds; a request passing neither is rejected (request-terminationon the fallback anonymous consumer).
Not yet active: PII redaction (ai-sanitizer) — needs a private Kong
Cloudsmith registry image and an AI Gateway Enterprise entitlement, blocked
pending that access. Everything else in this repo is unaffected.
Structure
dev/ai-gateway-kustomization.yaml+dev/ai-gateway-externalsecret.yaml— Gateway/DataPlane/Control Plane infra (same sharedk8s-platformtemplate every domain uses).dev/proxy-kustomization.yaml+dev/proxy-externalsecret.yaml— the LLM proxy itself (./kong).dev/gitrepository.yaml— this repo, self-referential.