assistant-agent.sttlab.local and its two siblings were long, and two of them
were French where nothing else in the estate is.
The Kubernetes object names are unchanged — they name a Deployment and a
Service, and nobody types them.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Nothing outside the cluster could talk to it: every conversation went through
kubectl run curl, which makes an agent that works and nobody can use.
The same authentication subrequest that guards the n8n endpoints guards this
one. The agent forwards and exchanges the caller's token, so a forged one
fails at the realm — but failing downstream is not refusing, and an
unauthenticated caller could still make it do inference.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two bugs from review, the second in both agents because identity.py is a copy.
The exchanged-token cache grew for the life of the process. It is keyed on
(subject token, scope) and the subject token is minted fresh every turn, so
every turn added keys nothing would ever read again — and expiry was checked on
read but never collected. A slow leak that only shows in a pod up for days,
which is the worst way to find one.
It now drops expired entries first and evicts oldest beyond a bound. Expired
first on purpose: an expired entry is dead weight wherever it sits in the
ordering, so evicting by age alone would keep it while discarding a live one.
The docstring was wrong before the code was: it claimed the cache lasted one
turn. It never did. Corrected rather than quietly made true.
message_id reused the context id, which is the conversation — and a conversation
makes several calls to the same peer, relaying an approval being exactly that.
A peer treating the second call as a replay of the first would surface as a
stale answer rather than an error.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two bugs in the peer call, both found in review.
message_id was context_id + peer name, so every call in a conversation carried
the same id. A2A wants one id per message; a peer treating the second call as a
replay of the first would have looked like a stale answer rather than a bug.
The peer's context_id was derived from hash(subject_token), which was wrong
twice over. hash() on a str is salted per process, so a restart changed the key
— the reported bug. And a fresh token is minted every turn, so the key changed
every turn anyway, which is worse: the peer started a new thread on each call
and lost anything it had parked, the opposite of what the key exists for. It is
now the conversation id, which is what "one conversation per peer" meant.
Not changed: output_type as a list, also raised in review. OutputSpec is
declared as an item OR a Sequence of them, so a list is the API's own form and
not a union written badly.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Closes the last item on this agent's backlog. A reference about to be reordered
is not a reference to discontinue, and the catalog's own quantity field does not
say whether it is — that is the supply agent's decision, and it answers rather
than handing back its stock table.
One narrow tool, not a generic call_agent(name, text): a generic one lets the
model reach any peer with any prompt, which is how an agent becomes a proxy onto
its neighbours. The peer's token is minted per call for that peer alone, so an
injection that talks this agent into consulting the wrong one fails at the
realm.
A peer ending in any state but completed is reported as such, never smoothed
over: a specialist that escalated has not answered, and presenting its
escalation as an answer is how a refusal turns into a fact.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The agent asked for confirmation because the system prompt told it to, and read
the answer itself. Both halves are now enforced.
Each MCP toolset is wrapped in ApprovalRequiredToolset whose predicate reads the
readOnlyHint the server publishes — the gateway decides what counts as a write,
and the agent keeps no list of tool names in step with it.
output_type=[str, DeferredToolRequests] lets a turn end holding a proposed call
instead of making it; the executor parks the A2A task in input-required and
renders the arguments the model actually sent, not what it said it would do.
The reply is matched against a closed set of words, so "oui mais change le prix"
denies the call and the correction comes back for approval on its own merits —
verified end to end, along with a draft the user approved that was still refused
because msrp sat below buyprice.
Also wired: a ProductDraft validated after approval and before the call leaves;
UsageLimits on every run; FilteredToolset denying sales' write tools inside the
agent as well as at the gateway; PrefixedToolset namespacing the two servers;
dynamic instructions naming the caller; Agent.instrument_all() for a span per
model call and per tool call; and pydantic_ai.retries on inference only, so a
429 from the gateway is waited out rather than failing mid-reasoning.
Inference gets its own httpx client: validate_response raises on any 4xx, and
applying that to the shared client would turn a token-exchange refusal into an
exception carrying no body — which is where Keycloak says what was wrong. Its
transport is built explicitly, because a transport passed to the client makes
verify= inert and the lab CA would otherwise go undeclared.
Tests grew from 7 to 31, covering what approval decides and what a draft must
satisfy. evals/catalogue_evals.py adds a behavioural suite, kept out of pytest
because every case spends real tokens; its first case checks that a count comes
from the tool rather than from what the model remembers of a public dataset.
CLAUDE.md corrected: instrument=True is not an Agent parameter in 2.35.3,
pydantic_ai.retries needs tenacity, TemplateStr needs pydantic-handlebars, and
approval does not remove the in-memory history — resuming still needs the
messages, so replicas: 1 stands until they live in the A2A task store.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The broker client was renamed in Keycloak so that the contract an agent has to
honour no longer names the middleware serving it. This is the last consumer to
move; the gateways and LibreChat went first.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
An agent talks to URLs that honour a contract: an MCP endpoint, an OIDC token
endpoint, an OpenAI-shaped LLM endpoint. Whatever serves those URLs is
replaceable, and naming it in an agent's code or docs implies a coupling that
does not exist.
Every product name is gone from prose, comments and identifiers, replaced by
what the agent actually depends on. Where a behaviour of the current middleware
leaks into the contract — the second audience scope, the tool ACL — the text
now says what the caller must do and why, not who imposes it.
One string survives and cannot be fixed here: the broker client an agent must
name in its audience scopes is itself named after the middleware. Closing that
means renaming it in Keycloak and following the rename through the gateway
configuration and LibreChat. Recorded as a blocker.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The image now sits beside the source it was built from, in gitea's own
registry, and inherits the repository's visibility. The repo is public, so the
cluster pulls with no credentials — which removes the imagePullSecret entirely,
along with the hand-copied GitHub token it depended on and the cross-namespace
copy that kept it alive.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Same arrangement classic-models already uses: a hand-created ghcr-pull-secret
rather than one committed or held in Vault, since the credential is a GitHub
token that belongs to no estate in particular. Without it the pod would sit in
ImagePullBackOff with no hint as to why.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two-stage Dockerfile: the runtime image carries neither uv nor a build
toolchain, runs as uid 10001, drops every capability and needs no writable
filesystem (verified with --read-only and no tmpfs).
k8s/ is what runs, dev/ is the Flux plumbing that applies it, self-managing
after three manual applies — the same shape every kong-* repo settled on.
Three choices worth naming:
- One namespace for the estate. What isolates an agent is its Keycloak client
and the audiences it may request, not a namespace boundary.
- The GitRepository uses anonymous HTTPS rather than SSH with a Vault-held
deploy key. The repo is already anonymously readable on gitea, so a key would
guard nothing while adding two objects to create by hand before anything
deploys. If the repos ever go private this becomes the kong-* SSH pattern.
- The lab CA certificate is committed. It is a certificate, never a key, and
checking it in means the deployment has no hidden prerequisite.
replicas: 1 is a constraint, not a default — conversation history lives in the
process, so a second replica would route a user's "yes" to a pod that never saw
the question.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The gateway already authorised writes for a product_manager — ai-mcp-proxy
filters tool discovery by consumer group, so create-product is simply absent
from a read-only user's turn. Only the system prompt was holding the agent
back, and it now describes how to write instead: restate the record, ask for
confirmation, never invent a missing field, report the identifier, and stop on
a refusal rather than working around it.
That flow cannot terminate without history. Each turn built a fresh Agent with
no message_history, so the user's "yes" arrived at an agent that had no idea
what it was agreeing to. History is now kept per A2A contextId in a bounded
in-memory map — enough for confirm-then-act, deliberately not durable, and
documented as needing sticky routing beyond one replica.
Verified end to end: pm-user creates S99_TEST1 across two turns and the row
appears in MySQL; ro-user asking for the same thing is told no write tool
exists, because for that token none does. Both test rows removed afterwards.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two things, both discovered by instrumenting the failure rather than trusting
the message it printed.
The MCP session never failed on protocol. macOS resolves any `.local` name
through mDNS first and takes a flat ~5.0s doing it, which sits exactly on
httpx's 5s default connect timeout; fastmcp reported the resulting
ConnectTimeout only as "Failed to initialize server session". Earlier tests
used `curl --resolve`, which bypasses DNS entirely and made it look like DNS
had been eliminated. Settings.http_timeout (60s) now covers the shared client,
the client fastmcp builds, and MCPToolset's init_timeout. list-products,
get-product and create-product now enumerate over the real gateway.
Inference used a static "unused" api_key against a gateway whose
openid-connect plugin requires audience ["llm-proxy"]. The model call now
carries a token exchanged for llm-proxy-audience, so it is attributable to the
same (agent, end user) pair as every tool call.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A token audienced only for mcp-catalog is refused by the gateway with a
bare 401. The reason is not visible to the caller: Kong's own
openid-connect plugin exchanges the token again before reaching the
backend, and Keycloak refuses that second exchange unless the token also
names kong-token-exchange-broker. The gateway log says "invalid status
code received from the token endpoint (403)"; the client sees only 401.
LibreChat already requests both audiences for the same reason. Proven end
to end with a pm-user token: aud=[mcp-catalog, kong-token-exchange-broker]
returns 200 from catalog-mcp, and the same request with aud=[mcp-catalog]
alone returns 401.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
First of the six agents, and the one that gives the A2A facade for the
others to interoperate with. A turn exchanges the user's token once per
audience, builds one MCP toolset per server with those tokens, runs the
model through the Kong AI gateway, and drops the toolsets.
Per-turn MCP clients are the point, not an implementation detail: a token
that outlives its turn can be replayed after the session ended. Several
narrow tokens rather than one broad one, so a prompt injection that talks
the agent into the wrong domain is refused at Kong rather than succeeding
quietly.
a2a-sdk rather than FastA2A, revising the May decision. That decision
rested on Agent.to_a2a() being removed in PydanticAI v2 — which has now
happened, so the argument no longer separates them. With four other
runtimes due to interoperate, the protocol author's implementation is the
safer base. httpx2 rather than httpx, which PydanticAI deprecates for
OpenAI-compatible providers and drops in v3.
Four things about a2a-sdk 1.1 that no documentation gave and that were
established against the running library, all recorded where they bite:
a2a.types are protobuf, not pydantic; DefaultRequestHandler speaks 0.3 and
V2 is needed for 1.0; a client must send A2A-Version: 1.0 or the server
assumes 0.3 and answers VERSION_NOT_SUPPORTED on a perfectly valid
request; and an executor must enqueue a Task before any status update.
Verified: the app builds with no deprecation warnings, serves its Agent
Card, and fails a turn cleanly with "no bearer token" rather than
answering from the model's own knowledge — which would hide a broken
identity chain behind a plausible reply. Seven tests cover the exchange
shape (scope not audience, per-audience caching, refusal) and the card.
The Keycloak client does not exist yet, so no end-to-end run has happened.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Design agreed in discussion, nothing built yet. Records the agent's role, its
scopes and MCP servers, its A2A surface, and the framework choice with its
reason — so the next session starts from the decisions rather than re-deriving
them.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Stéphane Tailland