kong-platform-llm/README.md

platform-llm

Platform-owned llm bounded context — Gateway infrastructure and the shared LLM proxy in one repo. Never split per functional domain, and never split between infra/routing here either: both are owned by the same (platform) team, unlike catalog's dataplane-catalog/catalog-api split (feature team vs. platform team, a real access boundary that doesn't exist for this domain).

What it does

Fronts OpenRouter (https://openrouter.ai) via one proxy route (/llm/v1/chat/completions) — the client's own model field selects among any OpenRouter-hosted model, not just a fixed set Kong picks.

Active policies:

• ai-rate-limiting-advanced, token-based, both global (across all callers) and per-client (per authenticated consumer).
• ai-prompt-guard (regex) and ai-semantic-prompt-guard (embedding-based, backed by Redis Stack) for prompt-injection/jailbreak safeguarding.
• Two-tier auth: key-auth (static, shared key) for chatbot UIs that can't do OIDC on their outbound LLM connection (LibreChat and similar tools — structural limitation of that whole category, not fixable client-side), openid-connect for agents built in-house. Either succeeds; a request passing neither is rejected (request-termination on the fallback anonymous consumer).

Not yet active: PII redaction (ai-sanitizer) — needs a private Kong Cloudsmith registry image and an AI Gateway Enterprise entitlement, blocked pending that access. Everything else in this repo is unaffected.

Structure

• dev/ai-gateway-kustomization.yaml + dev/ai-gateway-externalsecret.yaml — Gateway/DataPlane/Control Plane infra (same shared k8s-platform template every domain uses).
• dev/proxy-kustomization.yaml + dev/proxy-externalsecret.yaml — the LLM proxy itself (./kong).
• dev/gitrepository.yaml — this repo, self-referential.