docs: stop naming the middleware
An agent talks to URLs that honour a contract: an MCP endpoint, an OIDC token endpoint, an OpenAI-shaped LLM endpoint. Whatever serves those URLs is replaceable, and naming it in an agent's code or docs implies a coupling that does not exist. Every product name is gone from prose, comments and identifiers, replaced by what the agent actually depends on. Where a behaviour of the current middleware leaks into the contract — the second audience scope, the tool ACL — the text now says what the caller must do and why, not who imposes it. One string survives and cannot be fixed here: the broker client an agent must name in its audience scopes is itself named after the middleware. Closing that means renaming it in Keycloak and following the rename through the gateway configuration and LibreChat. Recorded as a blocker. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -46,13 +46,18 @@ blockers. This file only covers what is specific to this agent.
|
|||||||
|
|
||||||
## Non-negotiables
|
## Non-negotiables
|
||||||
|
|
||||||
- **Inference goes through the Kong AI gateway**, `https://llm-dev.sttlab.local/llm/v1/chat/completions`.
|
- **Inference goes through the estate's LLM endpoint**, `https://llm-dev.sttlab.local/llm/v1/chat/completions`.
|
||||||
Calling a provider directly bypasses the rate limits and prompt guards.
|
Calling a model provider directly bypasses the rate limits and prompt guards.
|
||||||
- **One MCP client per conversation turn**, built with freshly exchanged tokens.
|
- **One MCP client per conversation turn**, built with freshly exchanged tokens.
|
||||||
Do not hold a long-lived MCP connection with a fixed token.
|
Do not hold a long-lived MCP connection with a fixed token.
|
||||||
- **A2A calls return answers, not data.** Expose narrow decision-shaped skills,
|
- **A2A calls return answers, not data.** Expose narrow decision-shaped skills,
|
||||||
never a generic `query`.
|
never a generic `query`.
|
||||||
- Deliverables in English.
|
- Deliverables in English.
|
||||||
|
- **Never name the middleware.** The agent talks to URLs that honour a contract:
|
||||||
|
an MCP endpoint, an OIDC token endpoint, an OpenAI-shaped LLM endpoint. What
|
||||||
|
serves them is replaceable and the agent must not know or care. Write "the
|
||||||
|
MCP server", "the LLM endpoint", "the gateway" — never a product name, in
|
||||||
|
code, comments, commits or docs.
|
||||||
|
|
||||||
## Running it against the lab from macOS
|
## Running it against the lab from macOS
|
||||||
|
|
||||||
|
|||||||
@@ -3,8 +3,8 @@
|
|||||||
Product management agent for the Classic Models catalog domain. Serves a product
|
Product management agent for the Classic Models catalog domain. Serves a product
|
||||||
manager: what exists, how it is priced, whether it sells, what to discontinue.
|
manager: what exists, how it is priced, whether it sells, what to discontinue.
|
||||||
|
|
||||||
Reached over **A2A**; reads data over **MCP** through the Kong gateways; thinks
|
Reached over **A2A**; reads data over **MCP** from the domain servers; thinks
|
||||||
through the **Kong AI gateway**. It is one MCP client among others, exactly as
|
through the estate's **LLM endpoint**. It is one MCP client among others, as
|
||||||
LibreChat is — nothing under it changes to accommodate it.
|
LibreChat is — nothing under it changes to accommodate it.
|
||||||
|
|
||||||
## Running it
|
## Running it
|
||||||
@@ -40,7 +40,7 @@ is fine.
|
|||||||
```
|
```
|
||||||
A2A SendMessage ──► exchange the user's token, once per audience
|
A2A SendMessage ──► exchange the user's token, once per audience
|
||||||
──► build one MCP toolset per server, with that turn's tokens
|
──► build one MCP toolset per server, with that turn's tokens
|
||||||
──► PydanticAI runs the model through the Kong AI gateway
|
──► PydanticAI runs the model through the LLM endpoint
|
||||||
──► answer, toolsets dropped
|
──► answer, toolsets dropped
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -50,8 +50,8 @@ holding one long-lived connection would mean one long-lived token.
|
|||||||
|
|
||||||
Several narrow tokens rather than one broad one: a token minted for
|
Several narrow tokens rather than one broad one: a token minted for
|
||||||
`mcp-catalog` is refused by the sales gateway, so a prompt injection that talks
|
`mcp-catalog` is refused by the sales gateway, so a prompt injection that talks
|
||||||
the agent into reaching the wrong domain fails at Kong rather than succeeding
|
the agent into reaching the wrong domain is refused at the endpoint rather than
|
||||||
quietly.
|
succeeding quietly.
|
||||||
|
|
||||||
## What it can and cannot do
|
## What it can and cannot do
|
||||||
|
|
||||||
@@ -62,10 +62,10 @@ quietly.
|
|||||||
| Calls | nothing yet. `Approvisionnement.check_availability` comes with discontinuation decisions |
|
| Calls | nothing yet. `Approvisionnement.check_availability` comes with discontinuation decisions |
|
||||||
| Exposes | `describe_catalog` |
|
| Exposes | `describe_catalog` |
|
||||||
|
|
||||||
Who may write is decided by Kong, not by this agent. `ai-mcp-proxy` carries a
|
Who may write is decided by the MCP server, not by this agent. It authorises
|
||||||
per-tool ACL — `create-product` allows the `admin` and `product_manager`
|
each tool against the roles in the end user's own token — `create-product`
|
||||||
consumer groups — and the group comes from `realm_access.roles` in the end
|
needs `admin` or `product_manager` — and it does so at **discovery**, not only
|
||||||
user's own token. The filtering happens at **discovery**, not just at call time:
|
at call time:
|
||||||
|
|
||||||
```
|
```
|
||||||
pm-user catalog ['create-product', 'get-product', 'list-products']
|
pm-user catalog ['create-product', 'get-product', 'list-products']
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
# Watches ./dev, so the GitRepository and the Kustomization below it reconcile
|
# Watches ./dev, so the GitRepository and the Kustomization below it reconcile
|
||||||
# from git like any other manifest. It lives in ./dev itself, so it also manages
|
# from git like any other manifest. It lives in ./dev itself, so it also manages
|
||||||
# itself — the arrangement `flux bootstrap` sets up, which flux-operator does
|
# itself — the arrangement `flux bootstrap` sets up, which flux-operator does
|
||||||
# not create, and the same one every kong-* repo now uses.
|
# not create, and the same one the platform repos use.
|
||||||
#
|
#
|
||||||
# It sits in the agents namespace, not flux-system: a sourceRef resolves in the
|
# It sits in the agents namespace, not flux-system: a sourceRef resolves in the
|
||||||
# Kustomization's own namespace, and this is the agent team's to own.
|
# Kustomization's own namespace, and this is the agent team's to own.
|
||||||
|
|||||||
@@ -1,12 +1,11 @@
|
|||||||
# HTTPS with no credentials, unlike the kong-* repos which use SSH and a deploy
|
# HTTPS with no credentials, unlike the platform repos which use SSH and a
|
||||||
# key from Vault. Deliberate: this repo is already anonymously readable over
|
# deploy key from Vault. Deliberate: this repo is already anonymously readable over
|
||||||
# HTTPS on gitea, so a deploy key would guard nothing while adding a Vault entry
|
# HTTPS on gitea, so a deploy key would guard nothing while adding a Vault entry
|
||||||
# and a gitea key to create by hand before anything deploys. TLS still gives
|
# and a gitea key to create by hand before anything deploys. TLS still gives
|
||||||
# integrity and authenticity of the source.
|
# integrity and authenticity of the source.
|
||||||
#
|
#
|
||||||
# If the gitea repos are ever made private, this becomes an SSH sourceRef with
|
# If the gitea repos are ever made private, this becomes an SSH sourceRef with
|
||||||
# an ExternalSecret holding the deploy key — the pattern in
|
# an ExternalSecret holding the deploy key — the pattern the platform repos use.
|
||||||
# kong-catalog-api/dev/.
|
|
||||||
apiVersion: source.toolkit.fluxcd.io/v1
|
apiVersion: source.toolkit.fluxcd.io/v1
|
||||||
kind: GitRepository
|
kind: GitRepository
|
||||||
metadata:
|
metadata:
|
||||||
|
|||||||
+1
-1
@@ -13,7 +13,7 @@ dependencies = [
|
|||||||
# implementation is the safer base for the others to interoperate with.
|
# implementation is the safer base for the others to interoperate with.
|
||||||
"a2a-sdk[http-server]>=1.1.2,<2",
|
"a2a-sdk[http-server]>=1.1.2,<2",
|
||||||
# Reasoning and tool calling. -slim keeps the dependency surface to the
|
# Reasoning and tool calling. -slim keeps the dependency surface to the
|
||||||
# providers actually used; [openai] because the Kong AI gateway speaks the
|
# providers actually used; [openai] because the LLM endpoint speaks the
|
||||||
# OpenAI protocol, [mcp] for the MCP client.
|
# OpenAI protocol, [mcp] for the MCP client.
|
||||||
"pydantic-ai-slim[openai,mcp]>=2.35.3,<3",
|
"pydantic-ai-slim[openai,mcp]>=2.35.3,<3",
|
||||||
"httpx2>=2.12.0,<3",
|
"httpx2>=2.12.0,<3",
|
||||||
|
|||||||
@@ -44,18 +44,17 @@ Writing to the catalog:
|
|||||||
def build_model(
|
def build_model(
|
||||||
settings: Settings, http_client: httpx2.AsyncClient, access_token: str
|
settings: Settings, http_client: httpx2.AsyncClient, access_token: str
|
||||||
) -> OpenAIChatModel:
|
) -> OpenAIChatModel:
|
||||||
"""Inference through the Kong AI gateway, never a provider directly.
|
"""Inference through the estate's LLM endpoint, never a provider directly.
|
||||||
|
|
||||||
The gateway speaks the OpenAI protocol, hence this provider. Going straight
|
It speaks the OpenAI protocol, hence this provider. Going straight to a
|
||||||
to OpenRouter would bypass the token rate limits, the prompt guards and the
|
model provider would bypass the token rate limits, the prompt guards and the
|
||||||
audit trail that records what was sent to the model — all of which are the
|
audit trail that records what was sent to the model — which are the whole
|
||||||
point of routing inference through Kong.
|
reason inference is routed rather than direct.
|
||||||
|
|
||||||
`api_key` carries an exchanged token, not a static secret: the provider
|
`api_key` carries an exchanged token, not a static secret: the provider
|
||||||
sends it as `Authorization: Bearer`, which is exactly what the gateway's
|
sends it as `Authorization: Bearer`, which is what the endpoint validates.
|
||||||
openid-connect plugin reads. So the model call is attributable to the same
|
So the model call is attributable to the same (agent, end user) pair as the
|
||||||
(agent, end user) pair as the tool calls, and is rate-limited per client
|
tool calls, and is accounted per client rather than per application.
|
||||||
rather than per application.
|
|
||||||
"""
|
"""
|
||||||
return OpenAIChatModel(
|
return OpenAIChatModel(
|
||||||
settings.llm_model,
|
settings.llm_model,
|
||||||
|
|||||||
@@ -24,27 +24,30 @@ class Settings(BaseSettings):
|
|||||||
# as a SCOPE, not through the `audience` parameter — Keycloak answers
|
# as a SCOPE, not through the `audience` parameter — Keycloak answers
|
||||||
# "Requested audience not available" to the latter. See identity.py.
|
# "Requested audience not available" to the latter. See identity.py.
|
||||||
#
|
#
|
||||||
# Two audiences per server, not one. The gateway's own openid-connect plugin
|
# Two audiences per server, not one. The MCP endpoint exchanges the token
|
||||||
# exchanges the token again before reaching the backend, so the token must
|
# again on its own account before reaching the domain API, so the token has
|
||||||
# also name `kong-token-exchange-broker` — without it Kong's onward exchange
|
# to name the broker that performs that second exchange as well as the
|
||||||
# is refused and the caller sees a bare 401, with the real cause only in the
|
# server itself. Without it the call fails with a bare 401 and no usable
|
||||||
# gateway log ("invalid status code received from the token endpoint (403)").
|
# detail. This is part of the endpoint's contract, not of this agent's
|
||||||
|
# design — if the middleware serving these URLs is replaced by something
|
||||||
|
# that does not re-exchange, the second scope simply becomes unnecessary.
|
||||||
mcp_catalog_url: str = "https://catalog-dev.sttlab.local/catalog-mcp"
|
mcp_catalog_url: str = "https://catalog-dev.sttlab.local/catalog-mcp"
|
||||||
mcp_catalog_scope: str = "mcp-catalog-audience kong-token-exchange-broker-audience"
|
mcp_catalog_scope: str = "mcp-catalog-audience kong-token-exchange-broker-audience"
|
||||||
mcp_sales_url: str = "https://sales-dev.sttlab.local/sales-mcp"
|
mcp_sales_url: str = "https://sales-dev.sttlab.local/sales-mcp"
|
||||||
mcp_sales_scope: str = "mcp-sales-audience kong-token-exchange-broker-audience"
|
mcp_sales_scope: str = "mcp-sales-audience kong-token-exchange-broker-audience"
|
||||||
|
|
||||||
# --- inference ----------------------------------------------------------
|
# --- inference ----------------------------------------------------------
|
||||||
# Always the Kong AI gateway, never a provider directly: it carries the rate
|
# Always the estate's LLM endpoint, never a provider directly: it carries
|
||||||
# limits, the prompt guards and the audit trail. It speaks the OpenAI
|
# the rate limits, the prompt guards and the audit trail. It speaks the
|
||||||
# protocol, which is why the OpenAI provider is used against it.
|
# OpenAI protocol, which is why the OpenAI provider is used against it.
|
||||||
llm_base_url: str = "https://llm-dev.sttlab.local/llm/v1"
|
llm_base_url: str = "https://llm-dev.sttlab.local/llm/v1"
|
||||||
llm_model: str = "google/gemini-3.7-flash"
|
llm_model: str = "google/gemini-3.7-flash"
|
||||||
# The gateway's openid-connect plugin requires this audience. The bearer it
|
# The LLM endpoint requires this audience. The bearer it validates is an
|
||||||
# validates is an exchanged token, not a static key, so inference carries
|
# exchanged token, not a static API key, so inference carries the same
|
||||||
# the same `azp`/`sub` pair as every tool call and lands in the same audit
|
# `azp`/`sub` pair as every tool call and lands in the same audit trail.
|
||||||
# trail. The key-auth path alongside it exists only for callers that cannot
|
# Static-key authentication is also accepted there, for callers that cannot
|
||||||
# do OIDC outbound (LibreChat) and proves nothing about the end user.
|
# do OIDC outbound, but it proves nothing about the end user — this agent
|
||||||
|
# can do better and does.
|
||||||
llm_scope: str = "llm-proxy-audience"
|
llm_scope: str = "llm-proxy-audience"
|
||||||
|
|
||||||
# --- transport ----------------------------------------------------------
|
# --- transport ----------------------------------------------------------
|
||||||
|
|||||||
@@ -54,8 +54,8 @@ async def build_toolsets(
|
|||||||
|
|
||||||
Several narrow tokens rather than one broad one, deliberately: a token
|
Several narrow tokens rather than one broad one, deliberately: a token
|
||||||
minted for `mcp-catalog` is refused by the sales gateway, so a prompt
|
minted for `mcp-catalog` is refused by the sales gateway, so a prompt
|
||||||
injection that talks the agent into reaching the wrong domain fails at
|
injection that talks the agent into reaching the wrong domain is refused at
|
||||||
Kong rather than succeeding quietly.
|
the endpoint rather than succeeding quietly.
|
||||||
"""
|
"""
|
||||||
def client_factory(**kwargs: Any) -> httpx.AsyncClient:
|
def client_factory(**kwargs: Any) -> httpx.AsyncClient:
|
||||||
# fastmcp builds this client itself and would otherwise take httpx's
|
# fastmcp builds this client itself and would otherwise take httpx's
|
||||||
|
|||||||
Reference in New Issue
Block a user