Stéphane Tailland
2d4debd12a
A tool-calling agent re-sends the whole tool result on every iteration of its loop, so a single question against a tool returning a full collection burns the old budget in three turns and answers 429 in the middle of reasoning. Pagination on those tools is the real fix; this buys room until then and stays well inside the 500k global ceiling. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
platform-llm
Platform-owned llm bounded context — Gateway infrastructure and the
shared LLM proxy in one repo. Never split per functional domain, and never
split between infra/routing here either: both are owned by the same
(platform) team, unlike catalog's dataplane-catalog/catalog-api split
(feature team vs. platform team, a real access boundary that doesn't exist
for this domain).
What it does
Fronts OpenRouter (https://openrouter.ai) via one proxy route
(/llm/v1/chat/completions) — the client's own model field selects among
any OpenRouter-hosted model, not just a fixed set Kong picks.
Active policies:
ai-rate-limiting-advanced, token-based, both global (across all callers) and per-client (per authenticated consumer).ai-prompt-guard(regex) andai-semantic-prompt-guard(embedding-based, backed by Redis Stack) for prompt-injection/jailbreak safeguarding.- Two-tier auth:
key-auth(static, shared key) for chatbot UIs that can't do OIDC on their outbound LLM connection (LibreChat and similar tools — structural limitation of that whole category, not fixable client-side),openid-connectfor agents built in-house. Either succeeds; a request passing neither is rejected (request-terminationon the fallback anonymous consumer).
Not yet active: PII redaction (ai-sanitizer) — needs a private Kong
Cloudsmith registry image and an AI Gateway Enterprise entitlement, blocked
pending that access. Everything else in this repo is unaffected.
Structure
dev/ai-gateway-kustomization.yaml+dev/ai-gateway-externalsecret.yaml— Gateway/DataPlane/Control Plane infra (same sharedk8s-platformtemplate every domain uses).dev/proxy-kustomization.yaml+dev/proxy-externalsecret.yaml— the LLM proxy itself (./kong).dev/gitrepository.yaml— this repo, self-referential.