docs: stop naming the middleware

An agent talks to URLs that honour a contract: an MCP endpoint, an OIDC token
endpoint, an OpenAI-shaped LLM endpoint. Whatever serves those URLs is
replaceable, and naming it in an agent's code or docs implies a coupling that
does not exist.

Every product name is gone from prose, comments and identifiers, replaced by
what the agent actually depends on. Where a behaviour of the current middleware
leaks into the contract — the second audience scope, the tool ACL — the text
now says what the caller must do and why, not who imposes it.

One string survives and cannot be fixed here: the broker client an agent must
name in its audience scopes is itself named after the middleware. Closing that
means renaming it in Keycloak and following the rename through the gateway
configuration and LibreChat. Recorded as a blocker.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Stéphane Tailland
2026-08-29 07:35:15 +02:00
parent 31020e25dc
commit 601bd6343e
8 changed files with 47 additions and 41 deletions
+7 -2
View File
@@ -46,13 +46,18 @@ blockers. This file only covers what is specific to this agent.
## Non-negotiables ## Non-negotiables
- **Inference goes through the Kong AI gateway**, `https://llm-dev.sttlab.local/llm/v1/chat/completions`. - **Inference goes through the estate's LLM endpoint**, `https://llm-dev.sttlab.local/llm/v1/chat/completions`.
Calling a provider directly bypasses the rate limits and prompt guards. Calling a model provider directly bypasses the rate limits and prompt guards.
- **One MCP client per conversation turn**, built with freshly exchanged tokens. - **One MCP client per conversation turn**, built with freshly exchanged tokens.
Do not hold a long-lived MCP connection with a fixed token. Do not hold a long-lived MCP connection with a fixed token.
- **A2A calls return answers, not data.** Expose narrow decision-shaped skills, - **A2A calls return answers, not data.** Expose narrow decision-shaped skills,
never a generic `query`. never a generic `query`.
- Deliverables in English. - Deliverables in English.
- **Never name the middleware.** The agent talks to URLs that honour a contract:
an MCP endpoint, an OIDC token endpoint, an OpenAI-shaped LLM endpoint. What
serves them is replaceable and the agent must not know or care. Write "the
MCP server", "the LLM endpoint", "the gateway" — never a product name, in
code, comments, commits or docs.
## Running it against the lab from macOS ## Running it against the lab from macOS
+9 -9
View File
@@ -3,8 +3,8 @@
Product management agent for the Classic Models catalog domain. Serves a product Product management agent for the Classic Models catalog domain. Serves a product
manager: what exists, how it is priced, whether it sells, what to discontinue. manager: what exists, how it is priced, whether it sells, what to discontinue.
Reached over **A2A**; reads data over **MCP** through the Kong gateways; thinks Reached over **A2A**; reads data over **MCP** from the domain servers; thinks
through the **Kong AI gateway**. It is one MCP client among others, exactly as through the estate's **LLM endpoint**. It is one MCP client among others, as
LibreChat is — nothing under it changes to accommodate it. LibreChat is — nothing under it changes to accommodate it.
## Running it ## Running it
@@ -40,7 +40,7 @@ is fine.
``` ```
A2A SendMessage ──► exchange the user's token, once per audience A2A SendMessage ──► exchange the user's token, once per audience
──► build one MCP toolset per server, with that turn's tokens ──► build one MCP toolset per server, with that turn's tokens
──► PydanticAI runs the model through the Kong AI gateway ──► PydanticAI runs the model through the LLM endpoint
──► answer, toolsets dropped ──► answer, toolsets dropped
``` ```
@@ -50,8 +50,8 @@ holding one long-lived connection would mean one long-lived token.
Several narrow tokens rather than one broad one: a token minted for Several narrow tokens rather than one broad one: a token minted for
`mcp-catalog` is refused by the sales gateway, so a prompt injection that talks `mcp-catalog` is refused by the sales gateway, so a prompt injection that talks
the agent into reaching the wrong domain fails at Kong rather than succeeding the agent into reaching the wrong domain is refused at the endpoint rather than
quietly. succeeding quietly.
## What it can and cannot do ## What it can and cannot do
@@ -62,10 +62,10 @@ quietly.
| Calls | nothing yet. `Approvisionnement.check_availability` comes with discontinuation decisions | | Calls | nothing yet. `Approvisionnement.check_availability` comes with discontinuation decisions |
| Exposes | `describe_catalog` | | Exposes | `describe_catalog` |
Who may write is decided by Kong, not by this agent. `ai-mcp-proxy` carries a Who may write is decided by the MCP server, not by this agent. It authorises
per-tool ACL — `create-product` allows the `admin` and `product_manager` each tool against the roles in the end user's own token — `create-product`
consumer groups — and the group comes from `realm_access.roles` in the end needs `admin` or `product_manager` — and it does so at **discovery**, not only
user's own token. The filtering happens at **discovery**, not just at call time: at call time:
``` ```
pm-user catalog ['create-product', 'get-product', 'list-products'] pm-user catalog ['create-product', 'get-product', 'list-products']
+1 -1
View File
@@ -1,7 +1,7 @@
# Watches ./dev, so the GitRepository and the Kustomization below it reconcile # Watches ./dev, so the GitRepository and the Kustomization below it reconcile
# from git like any other manifest. It lives in ./dev itself, so it also manages # from git like any other manifest. It lives in ./dev itself, so it also manages
# itself — the arrangement `flux bootstrap` sets up, which flux-operator does # itself — the arrangement `flux bootstrap` sets up, which flux-operator does
# not create, and the same one every kong-* repo now uses. # not create, and the same one the platform repos use.
# #
# It sits in the agents namespace, not flux-system: a sourceRef resolves in the # It sits in the agents namespace, not flux-system: a sourceRef resolves in the
# Kustomization's own namespace, and this is the agent team's to own. # Kustomization's own namespace, and this is the agent team's to own.
+3 -4
View File
@@ -1,12 +1,11 @@
# HTTPS with no credentials, unlike the kong-* repos which use SSH and a deploy # HTTPS with no credentials, unlike the platform repos which use SSH and a
# key from Vault. Deliberate: this repo is already anonymously readable over # deploy key from Vault. Deliberate: this repo is already anonymously readable over
# HTTPS on gitea, so a deploy key would guard nothing while adding a Vault entry # HTTPS on gitea, so a deploy key would guard nothing while adding a Vault entry
# and a gitea key to create by hand before anything deploys. TLS still gives # and a gitea key to create by hand before anything deploys. TLS still gives
# integrity and authenticity of the source. # integrity and authenticity of the source.
# #
# If the gitea repos are ever made private, this becomes an SSH sourceRef with # If the gitea repos are ever made private, this becomes an SSH sourceRef with
# an ExternalSecret holding the deploy key — the pattern in # an ExternalSecret holding the deploy key — the pattern the platform repos use.
# kong-catalog-api/dev/.
apiVersion: source.toolkit.fluxcd.io/v1 apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository kind: GitRepository
metadata: metadata:
+1 -1
View File
@@ -13,7 +13,7 @@ dependencies = [
# implementation is the safer base for the others to interoperate with. # implementation is the safer base for the others to interoperate with.
"a2a-sdk[http-server]>=1.1.2,<2", "a2a-sdk[http-server]>=1.1.2,<2",
# Reasoning and tool calling. -slim keeps the dependency surface to the # Reasoning and tool calling. -slim keeps the dependency surface to the
# providers actually used; [openai] because the Kong AI gateway speaks the # providers actually used; [openai] because the LLM endpoint speaks the
# OpenAI protocol, [mcp] for the MCP client. # OpenAI protocol, [mcp] for the MCP client.
"pydantic-ai-slim[openai,mcp]>=2.35.3,<3", "pydantic-ai-slim[openai,mcp]>=2.35.3,<3",
"httpx2>=2.12.0,<3", "httpx2>=2.12.0,<3",
+8 -9
View File
@@ -44,18 +44,17 @@ Writing to the catalog:
def build_model( def build_model(
settings: Settings, http_client: httpx2.AsyncClient, access_token: str settings: Settings, http_client: httpx2.AsyncClient, access_token: str
) -> OpenAIChatModel: ) -> OpenAIChatModel:
"""Inference through the Kong AI gateway, never a provider directly. """Inference through the estate's LLM endpoint, never a provider directly.
The gateway speaks the OpenAI protocol, hence this provider. Going straight It speaks the OpenAI protocol, hence this provider. Going straight to a
to OpenRouter would bypass the token rate limits, the prompt guards and the model provider would bypass the token rate limits, the prompt guards and the
audit trail that records what was sent to the model — all of which are the audit trail that records what was sent to the model — which are the whole
point of routing inference through Kong. reason inference is routed rather than direct.
`api_key` carries an exchanged token, not a static secret: the provider `api_key` carries an exchanged token, not a static secret: the provider
sends it as `Authorization: Bearer`, which is exactly what the gateway's sends it as `Authorization: Bearer`, which is what the endpoint validates.
openid-connect plugin reads. So the model call is attributable to the same So the model call is attributable to the same (agent, end user) pair as the
(agent, end user) pair as the tool calls, and is rate-limited per client tool calls, and is accounted per client rather than per application.
rather than per application.
""" """
return OpenAIChatModel( return OpenAIChatModel(
settings.llm_model, settings.llm_model,
+16 -13
View File
@@ -24,27 +24,30 @@ class Settings(BaseSettings):
# as a SCOPE, not through the `audience` parameter — Keycloak answers # as a SCOPE, not through the `audience` parameter — Keycloak answers
# "Requested audience not available" to the latter. See identity.py. # "Requested audience not available" to the latter. See identity.py.
# #
# Two audiences per server, not one. The gateway's own openid-connect plugin # Two audiences per server, not one. The MCP endpoint exchanges the token
# exchanges the token again before reaching the backend, so the token must # again on its own account before reaching the domain API, so the token has
# also name `kong-token-exchange-broker` — without it Kong's onward exchange # to name the broker that performs that second exchange as well as the
# is refused and the caller sees a bare 401, with the real cause only in the # server itself. Without it the call fails with a bare 401 and no usable
# gateway log ("invalid status code received from the token endpoint (403)"). # detail. This is part of the endpoint's contract, not of this agent's
# design — if the middleware serving these URLs is replaced by something
# that does not re-exchange, the second scope simply becomes unnecessary.
mcp_catalog_url: str = "https://catalog-dev.sttlab.local/catalog-mcp" mcp_catalog_url: str = "https://catalog-dev.sttlab.local/catalog-mcp"
mcp_catalog_scope: str = "mcp-catalog-audience kong-token-exchange-broker-audience" mcp_catalog_scope: str = "mcp-catalog-audience kong-token-exchange-broker-audience"
mcp_sales_url: str = "https://sales-dev.sttlab.local/sales-mcp" mcp_sales_url: str = "https://sales-dev.sttlab.local/sales-mcp"
mcp_sales_scope: str = "mcp-sales-audience kong-token-exchange-broker-audience" mcp_sales_scope: str = "mcp-sales-audience kong-token-exchange-broker-audience"
# --- inference ---------------------------------------------------------- # --- inference ----------------------------------------------------------
# Always the Kong AI gateway, never a provider directly: it carries the rate # Always the estate's LLM endpoint, never a provider directly: it carries
# limits, the prompt guards and the audit trail. It speaks the OpenAI # the rate limits, the prompt guards and the audit trail. It speaks the
# protocol, which is why the OpenAI provider is used against it. # OpenAI protocol, which is why the OpenAI provider is used against it.
llm_base_url: str = "https://llm-dev.sttlab.local/llm/v1" llm_base_url: str = "https://llm-dev.sttlab.local/llm/v1"
llm_model: str = "google/gemini-3.7-flash" llm_model: str = "google/gemini-3.7-flash"
# The gateway's openid-connect plugin requires this audience. The bearer it # The LLM endpoint requires this audience. The bearer it validates is an
# validates is an exchanged token, not a static key, so inference carries # exchanged token, not a static API key, so inference carries the same
# the same `azp`/`sub` pair as every tool call and lands in the same audit # `azp`/`sub` pair as every tool call and lands in the same audit trail.
# trail. The key-auth path alongside it exists only for callers that cannot # Static-key authentication is also accepted there, for callers that cannot
# do OIDC outbound (LibreChat) and proves nothing about the end user. # do OIDC outbound, but it proves nothing about the end user — this agent
# can do better and does.
llm_scope: str = "llm-proxy-audience" llm_scope: str = "llm-proxy-audience"
# --- transport ---------------------------------------------------------- # --- transport ----------------------------------------------------------
+2 -2
View File
@@ -54,8 +54,8 @@ async def build_toolsets(
Several narrow tokens rather than one broad one, deliberately: a token Several narrow tokens rather than one broad one, deliberately: a token
minted for `mcp-catalog` is refused by the sales gateway, so a prompt minted for `mcp-catalog` is refused by the sales gateway, so a prompt
injection that talks the agent into reaching the wrong domain fails at injection that talks the agent into reaching the wrong domain is refused at
Kong rather than succeeding quietly. the endpoint rather than succeeding quietly.
""" """
def client_factory(**kwargs: Any) -> httpx.AsyncClient: def client_factory(**kwargs: Any) -> httpx.AsyncClient:
# fastmcp builds this client itself and would otherwise take httpx's # fastmcp builds this client itself and would otherwise take httpx's