9b713a894c
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
171 lines
9.0 KiB
Markdown
171 lines
9.0 KiB
Markdown
# Catalogue — product management agent
|
|
|
|
**Framework:** PydanticAI + `a2a-sdk` · **Mode:** conversational · **Client id:** `classic-models-agent-catalogue`
|
|
|
|
Serves the product manager: create and update products and product lines, set
|
|
prices, decide what to discontinue.
|
|
|
|
## Scopes and MCP servers
|
|
|
|
| | |
|
|
|---|---|
|
|
| MCP | catalog (`mcp-catalog-audience`), sales (`mcp-sales-audience`) |
|
|
| Scopes | `catalog:read`, `catalog:write`, `sales:read` |
|
|
|
|
`sales:read` is there to answer "does this product actually sell?" before a
|
|
pricing or discontinuation decision. Read only — this agent never touches orders.
|
|
|
|
## A2A
|
|
|
|
**Calls** — `the supply agent.check_availability`, to know remaining stock and
|
|
open orders before discontinuing a reference.
|
|
|
|
**Exposes** — one read-only skill, `describe_catalog`: what exists in the
|
|
catalog, how it is priced and positioned, and whether it sells. Skills are
|
|
decision-shaped, never a generic `query` — add one when another agent has a
|
|
specific question, not in advance.
|
|
|
|
## Why PydanticAI here
|
|
|
|
Not arbitrary. This is the agent that **writes structured data under
|
|
constraints** — prices, quantities, references, product line membership. Typed
|
|
validation of the model's output earns its place here in a way it would not on a
|
|
purely conversational agent.
|
|
|
|
**That argument is not yet cashed in.** `agent.py` builds `Agent(model,
|
|
instructions=..., toolsets=...)` with no `output_type`, no output validator and
|
|
no local tool, so the output is a free `str`. On the code as it stands,
|
|
PydanticAI is a bare MCP client and nothing more. Either wire the validation
|
|
(see below) or stop claiming it as the reason for the framework.
|
|
|
|
## What is wired, and what it cost
|
|
|
|
All of the below is in the code as of 2026-08-29. Verified against the installed
|
|
`pydantic-ai-slim` 2.35.3 — **three of these do add a dependency**, contrary to
|
|
what an earlier revision of this file claimed:
|
|
|
|
- `pydantic_ai.retries` ships in the package but imports `tenacity`, which the
|
|
slim extras do not pull. Added.
|
|
- `pydantic-evals` is a separate distribution. Added as a dev dependency.
|
|
- `TemplateStr` needs `pydantic-handlebars` (`pydantic-ai-slim[spec]`). **Not
|
|
added** — a template engine earns nothing against a single interpolation, so
|
|
the dynamic instructions are a plain function, which is the idiomatic API and
|
|
needs nothing.
|
|
|
|
1. **Approval as a mechanism, not a prompt rule.** Done, and it is the change
|
|
that matters. Each MCP toolset is wrapped in `ApprovalRequiredToolset`, whose
|
|
predicate reads the **`readOnlyHint` the server publishes** — so the gateway
|
|
decides what counts as a write and the agent keeps no list of tool names in
|
|
step with it. `output_type=[str, DeferredToolRequests]` lets a turn end
|
|
holding a proposed call instead of making it; the executor parks the task
|
|
with `TaskUpdater.requires_input()` and renders the arguments *the model
|
|
actually proposed*, not what it said it would do. The reply is read
|
|
deterministically against a closed set of words — a model must not be what
|
|
decides whether the user said yes, and "oui mais change le prix" is not
|
|
consent.
|
|
|
|
One claim in the earlier revision does not hold: this does **not** let the
|
|
in-memory history go. Resuming needs `message_history`, so the pending state
|
|
still lives in the process and `replicas: 1` still stands. Moving it into the
|
|
A2A task store is what would lift that, and it is not done.
|
|
2. **A validated draft on writes.** `approval.py` holds `ProductDraft` —
|
|
price above zero, stock within range, msrp above buyprice, no empty strings,
|
|
no extra fields. It runs *after* the user approves and *before* the call
|
|
leaves, so a record the user waved through is still refused if the model
|
|
built it wrong. Note this is argument validation, not `output_type`:
|
|
`output_type` describes a run's result, and the thing worth constraining
|
|
here is what goes out, not what comes back.
|
|
3. **Instrumentation.** `Agent.instrument_all()` at start-up, in
|
|
`__main__.py` — **not** `instrument=True` on `Agent(...)`, which does not
|
|
exist in 2.35.3; the parameters are `capabilities, defer_model_check,
|
|
deps_type, description, end_strategy, instructions, max_concurrency,
|
|
metadata, model, model_settings, name, output_type, retries, system_prompt,
|
|
tool_timeout, tools, toolsets, validation_context`. OpenTelemetry and
|
|
`logfire_api` are already present transitively. One span per model call and
|
|
per tool call: the MCP server records *which* tool ran, this records why.
|
|
4. **`UsageLimits(tool_calls_limit=...)`** on every run, from
|
|
`Settings.max_tool_calls` (20).
|
|
5. **Toolset shaping.** `FilteredToolset` denies sales' write tools inside the
|
|
agent as well as at the gateway — an admin reaching this agent would
|
|
otherwise be offered `create-order`. `PrefixedToolset` namespaces each
|
|
server, so a future collision between two domains is impossible rather than
|
|
silent.
|
|
6. **Dynamic instructions.** A `caller_instructions(ctx)` function appended
|
|
per run, naming the identity the agent is acting under. Deps-typed
|
|
(`CallerDeps`), no template engine — see above.
|
|
7. **`pydantic_ai.retries`** on inference only. `wait_retry_after` honours
|
|
the gateway's own `429 AI token rate limit exceeded`, so a turn survives the
|
|
limit instead of failing mid-reasoning. It gets a **separate httpx client**:
|
|
`validate_response` raises on any 4xx, and applying that to the shared client
|
|
would turn a token-exchange refusal into an exception carrying no body —
|
|
which is where Keycloak says what was wrong.
|
|
8. **`durable_exec`** — not done, on the reasoning in the earlier revision:
|
|
heavy, and the right fix for single-replica history is the A2A task store,
|
|
not a workflow engine.
|
|
|
|
**`pydantic-evals`**, now a dev dependency, with a suite in
|
|
`evals/catalogue_evals.py`. It is not part of `pytest`: every case spends real
|
|
tokens through the gateway and needs a live realm.
|
|
|
|
Its cases are the failures actually seen, not coverage. The first one is the
|
|
important one: an agent asked for a product count once answered the true figure
|
|
while its tool had returned a page holding a different one. Right answer, green
|
|
trace, ungrounded number — and only a check against the data tells the two
|
|
apart.
|
|
|
|
The unit tests grew to 31 alongside, covering what approval decides and what a
|
|
draft must satisfy. Every rule they check used to live in the system prompt,
|
|
which is to say it used to be a request.
|
|
|
|
Keep the split straight while doing any of this: none of it is authorisation.
|
|
Who may write is decided by the MCP server against the end user's own roles, at
|
|
discovery time. Everything above makes the agent predictable, not privileged.
|
|
|
|
## a2a-sdk, not FastA2A
|
|
|
|
An earlier revision of this file said FastA2A, on the grounds that
|
|
`Agent.to_a2a()` was going away in PydanticAI v2 — which has since happened, so
|
|
that argument no longer separates the two. The code uses the official
|
|
`a2a-sdk` (1.1.x): this estate exists to prove A2A interoperability across five
|
|
runtimes, and the reference implementation is the safer thing for four other
|
|
stacks to interoperate with.
|
|
|
|
It is framework-agnostic too, so it stays the natural basis for the A2A adapter
|
|
in front of the n8n agents — see `classic-models-agent-flows`.
|
|
|
|
## Before writing code
|
|
|
|
Read `classic-models-agent-platform/CLAUDE.md` first. It holds the identity
|
|
model, the MCP and LLM endpoints, the token-exchange recipe and the known
|
|
blockers. This file only covers what is specific to this agent.
|
|
|
|
## Non-negotiables
|
|
|
|
- **Inference goes through the estate's LLM endpoint**, `https://llm-dev.sttlab.local/llm/v1/chat/completions`.
|
|
Calling a model provider directly bypasses the rate limits and prompt guards.
|
|
- **One MCP client per conversation turn**, built with freshly exchanged tokens.
|
|
Do not hold a long-lived MCP connection with a fixed token.
|
|
- **A2A calls return answers, not data.** Expose narrow decision-shaped skills,
|
|
never a generic `query`.
|
|
- Deliverables in English.
|
|
- **Never name the middleware.** The agent talks to URLs that honour a contract:
|
|
an MCP endpoint, an OIDC token endpoint, an OpenAI-shaped LLM endpoint. What
|
|
serves them is replaceable and the agent must not know or care. Write "the
|
|
MCP server", "the LLM endpoint", "the gateway" — never a product name, in
|
|
code, comments, commits or docs.
|
|
|
|
## Running it against the lab from macOS
|
|
|
|
macOS resolves every `.local` name through mDNS before falling back to
|
|
`/etc/hosts`, and that costs a **flat ~5.0s per hostname** — `dscacheutil -q host
|
|
-a name catalog-dev.sttlab.local` shows it, and so does `curl -w
|
|
%{time_namelookup}`. httpx's default connect timeout is 5s, so the very first
|
|
request to each host lands on the boundary and fails intermittently. fastmcp
|
|
reports that as `Failed to initialize server session` and swallows the cause,
|
|
which is why it read as an MCP protocol problem for a long time. `curl --resolve`
|
|
hides it completely — do not use `--resolve` to "prove" DNS is fine.
|
|
|
|
`Settings.http_timeout` (60s) is the fix, applied to the shared httpx2 client,
|
|
to the client fastmcp builds, and to `MCPToolset(init_timeout=...)`. In the
|
|
cluster these names resolve instantly and the timeout never binds.
|