docs: stop naming the middleware

An agent talks to URLs that honour a contract: an MCP endpoint, an OIDC token
endpoint, an OpenAI-shaped LLM endpoint. Whatever serves those URLs is
replaceable, and naming it in an agent's code or docs implies a coupling that
does not exist.

Every product name is gone from prose, comments and identifiers, replaced by
what the agent actually depends on. Where a behaviour of the current middleware
leaks into the contract — the second audience scope, the tool ACL — the text
now says what the caller must do and why, not who imposes it.

One string survives and cannot be fixed here: the broker client an agent must
name in its audience scopes is itself named after the middleware. Closing that
means renaming it in Keycloak and following the rename through the gateway
configuration and LibreChat. Recorded as a blocker.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Stéphane Tailland
2026-08-29 07:35:15 +02:00
parent 31020e25dc
commit 601bd6343e
8 changed files with 47 additions and 41 deletions
+8 -9
View File
@@ -44,18 +44,17 @@ Writing to the catalog:
def build_model(
settings: Settings, http_client: httpx2.AsyncClient, access_token: str
) -> OpenAIChatModel:
"""Inference through the Kong AI gateway, never a provider directly.
"""Inference through the estate's LLM endpoint, never a provider directly.
The gateway speaks the OpenAI protocol, hence this provider. Going straight
to OpenRouter would bypass the token rate limits, the prompt guards and the
audit trail that records what was sent to the model — all of which are the
point of routing inference through Kong.
It speaks the OpenAI protocol, hence this provider. Going straight to a
model provider would bypass the token rate limits, the prompt guards and the
audit trail that records what was sent to the model — which are the whole
reason inference is routed rather than direct.
`api_key` carries an exchanged token, not a static secret: the provider
sends it as `Authorization: Bearer`, which is exactly what the gateway's
openid-connect plugin reads. So the model call is attributable to the same
(agent, end user) pair as the tool calls, and is rate-limited per client
rather than per application.
sends it as `Authorization: Bearer`, which is what the endpoint validates.
So the model call is attributable to the same (agent, end user) pair as the
tool calls, and is accounted per client rather than per application.
"""
return OpenAIChatModel(
settings.llm_model,
+16 -13
View File
@@ -24,27 +24,30 @@ class Settings(BaseSettings):
# as a SCOPE, not through the `audience` parameter — Keycloak answers
# "Requested audience not available" to the latter. See identity.py.
#
# Two audiences per server, not one. The gateway's own openid-connect plugin
# exchanges the token again before reaching the backend, so the token must
# also name `kong-token-exchange-broker` — without it Kong's onward exchange
# is refused and the caller sees a bare 401, with the real cause only in the
# gateway log ("invalid status code received from the token endpoint (403)").
# Two audiences per server, not one. The MCP endpoint exchanges the token
# again on its own account before reaching the domain API, so the token has
# to name the broker that performs that second exchange as well as the
# server itself. Without it the call fails with a bare 401 and no usable
# detail. This is part of the endpoint's contract, not of this agent's
# design — if the middleware serving these URLs is replaced by something
# that does not re-exchange, the second scope simply becomes unnecessary.
mcp_catalog_url: str = "https://catalog-dev.sttlab.local/catalog-mcp"
mcp_catalog_scope: str = "mcp-catalog-audience kong-token-exchange-broker-audience"
mcp_sales_url: str = "https://sales-dev.sttlab.local/sales-mcp"
mcp_sales_scope: str = "mcp-sales-audience kong-token-exchange-broker-audience"
# --- inference ----------------------------------------------------------
# Always the Kong AI gateway, never a provider directly: it carries the rate
# limits, the prompt guards and the audit trail. It speaks the OpenAI
# protocol, which is why the OpenAI provider is used against it.
# Always the estate's LLM endpoint, never a provider directly: it carries
# the rate limits, the prompt guards and the audit trail. It speaks the
# OpenAI protocol, which is why the OpenAI provider is used against it.
llm_base_url: str = "https://llm-dev.sttlab.local/llm/v1"
llm_model: str = "google/gemini-3.7-flash"
# The gateway's openid-connect plugin requires this audience. The bearer it
# validates is an exchanged token, not a static key, so inference carries
# the same `azp`/`sub` pair as every tool call and lands in the same audit
# trail. The key-auth path alongside it exists only for callers that cannot
# do OIDC outbound (LibreChat) and proves nothing about the end user.
# The LLM endpoint requires this audience. The bearer it validates is an
# exchanged token, not a static API key, so inference carries the same
# `azp`/`sub` pair as every tool call and lands in the same audit trail.
# Static-key authentication is also accepted there, for callers that cannot
# do OIDC outbound, but it proves nothing about the end user — this agent
# can do better and does.
llm_scope: str = "llm-proxy-audience"
# --- transport ----------------------------------------------------------
+2 -2
View File
@@ -54,8 +54,8 @@ async def build_toolsets(
Several narrow tokens rather than one broad one, deliberately: a token
minted for `mcp-catalog` is refused by the sales gateway, so a prompt
injection that talks the agent into reaching the wrong domain fails at
Kong rather than succeeding quietly.
injection that talks the agent into reaching the wrong domain is refused at
the endpoint rather than succeeding quietly.
"""
def client_factory(**kwargs: Any) -> httpx.AsyncClient:
# fastmcp builds this client itself and would otherwise take httpx's