The tool argument name Kong actually forwards comes from the operation's
declared parameters — a header cannot be sent as an arbitrary argument name,
it has to be in the spec like any other parameter.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Adds the restocker KongConsumerGroup and patch-product to the tools policy,
allow: [admin, product_manager, restocker]. classic-models-api backs this the
same way — restocker may PATCH a product and nothing else.
Regenerated with generate-mcp-tools.py.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The proxy cache held every GET on the whole service for 60 seconds, and these
APIs accept writes. Kong's proxy cache has no invalidation on write: a POST
that changes a product passes through the same gateway, and the gateway goes on
serving the value it captured before it, for up to a minute, with nothing in the
response saying so.
It produced a real wrong answer. An agent woken by a stock event read the figure
from before that event and reported, in one sentence, "68 in stock, above the
threshold" and "it crossed the threshold, from 68".
The manifest is kept, unreferenced, carrying the reason — so reinstating it is a
decision rather than a rediscovery.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The ACL answers 'may this identity call create-product'. Nothing answered 'may
it call it with these arguments' — write authorises writing, not this
particular record. And the read cap counts calls, so one call asking for a
hundred thousand rows cost the same as one asking for ten.
The OPA plugin sends the decoded JSON-RPC body, so the decision point sees the
tool name and its arguments. The policy lives once in k8s-platform/opa and every
domain asks the same server.
Enforced at the gateway rather than in an agent, because an agent that has been
talked into something is the case this exists for.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A scope authorises a read and says nothing about volume. An agent reads a
thousand records in seconds where a person clicks ten, and until now nothing
noticed: the token is valid, the tool is permitted, the ACL passes. That gap
stopped being theoretical the day an autonomous run started reading the whole
catalog on a schedule.
identifier: credential keys the counter on the sub claim — the end user,
whoever is asking on their behalf — so an agent looping under one person's
identity spends that person's budget and cannot spend everyone's. Verified on
catalog: pm-user and ro-user hold separate counters.
Per-route namespace, so a busy domain cannot exhaust another's.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A scope authorises a read and says nothing about volume. An agent reads a
thousand records in seconds where a person clicks ten, and until now nothing
noticed: the token is valid, the tool is permitted, the ACL passes.
identifier: credential keys the counter on the sub claim — the end user, whoever
is asking on their behalf — so an agent looping under one person's identity
spends that person's budget and cannot spend everyone's. Per-route namespace, so
a busy catalog cannot exhaust another domain's.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The response schemas added yesterday made every paginated list tool fail. The
plugin's fields are documented as OpenAPI format and validated with a JSON
Schema validator, and `nullable: true` is not a JSON Schema keyword — so it is
ignored, `type: string` is enforced, and `previous: null` on the first page is
rejected as *null is not of type "string"*.
The failure lands after the backend has already done the work, surfaces to the
caller as a bare tool retry exhaustion, and appears nowhere in the gateway's own
logs. Found only because an orchestrating agent asked a question that paged.
The generator now rewrites `nullable: true` into `type: [T, "null"]` wherever it
inlines a schema, request bodies included. A 3.1 spec needs none of this.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The tools: array was hand-maintained and had drifted from openapi.yaml in two
ways that only surfaced when an agent hit them: create-product omitted
productcode and could never succeed, and list-products declared no parameters
so it could not paginate — which let a model answer a product count that looked
right and had not come from the tool's output.
It is now produced by k8s-platform/tools/generate-mcp-tools.py: deck file
openapi2mcp does the conversion, and the wrapper re-injects the five things
Kong's generator cannot emit — per-tool ACLs, the MCP behavioural annotations,
the response schemas the plugin uses for structuredOutput, the real upstream
host, and JSON-only request bodies. Each is a documented gap; see
k8s-platform/MCP-TOOL-GENERATION.md.
tools-policy.yaml carries the ACLs and doubles as the allow-list: an operation
becomes a tool only if the policy names it. Kong would otherwise expose every
operation in the spec, DELETE included, and can express no authorization at all
from a bearer-token spec — silently, exit 0, no warning.
Same three tools as before, same ACLs. What changed: pagination is now
callable, create-product's required list matches the API (quantityinstock
included, which the hand-written repair had missed), each tool declares its
response schema, and the request body is no longer tripled across three
identical content types.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every agent has to name this client in the audience scopes it requests, so the
one thing an agent cannot avoid naming was the very thing it should know
nothing about. The gateway is replaceable; the contract should not say
otherwise.
token-exchange-broker, same credential, same behaviour. The old client stays in
the realm until every consumer has moved.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The tool's request schema omitted productcode, which catalog-api requires
(ProductRequest in openapi.yaml lists it among the eight required fields).
Since the declared schema is the only thing an MCP caller sees, there was no
input that worked: omit it and the API answers
{"productcode":["This field is required."]}; supply it and ai-mcp-proxy rejects
the call first with "Additional properties are not allowed ('productcode' was
unexpected)".
The schema now mirrors ProductRequest — productcode added, the same eight
fields marked required, and the length and format constraints carried over so
a model reads them rather than guessing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
dependsOn still named flux-system, where the gateway Kustomization used to
live. It moved into the domain namespace so the feature team can reach it,
and a dependency naming the old location resolves to nothing — Flux
reported "dependency not found" and held this Kustomization not ready
rather than failing loudly.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
dev/ declares this repo's GitRepository, the Kustomization that deploys
./kong, and its ExternalSecrets — but nothing watched dev/, so those files
were applied once by hand and never re-read. Editing them in git changed
nothing until someone remembered to kubectl apply, which is what silently
swallowed the CONTROL_PLANE_REF_NAME rollout: correct in git for hours
while the cluster still held the old value.
Add the bootstrap Kustomization that watches ./dev and, living in ./dev,
manages itself. This is what `flux bootstrap` would have created;
flux-operator installs the controllers and no root.
It sits in the domain namespace rather than flux-system, because sourceRef
resolves in the Kustomization's own namespace and because this is the
team's to own.
Needs one kubectl apply to exist. That is the last one this repo needs.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This spec predated the other four and documented the wrong surface: the
paths were the backend's (/classic-models/api/v1/...) and the server was
http://localhost:8000. Neither is reachable from the developer portal,
so every operation it published failed as written — the gateway serves
/catalog/v1/... and rewrites onto the backend path upstream.
Regenerated the same way as the other four domains: from the backend's
drf-spectacular schema, filtered to the operations this domain's gateway
exposes, derived from the Kong routes and their request-transformer
rewrites in Konnect. All 14 REST routes resolved to a backend operation.
Same six paths as before, now expressed as the gateway's, with
https://catalog-dev.sttlab.local as the server and path parameters
renamed to the gateway's capture names.
The five published specs now follow one convention.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
catalog was the only domain that split its MCP route in two: the main
route on POST/DELETE, and a separate -stream route matching GET, whose
only purpose was to carry a request-termination plugin answering 405
"SSE stream is not offered on this endpoint". The other four domains
keep one route with GET, POST and DELETE.
That opt-out is not a fix the other four missed — it gives up a feature
the plugin implements. ai-mcp-proxy handles non-POST itself: its handler
carries an explicit non-POST branch, MCP session handling, an SSE
subscription waiting on notifications/tools/list_changed, and Accept
header validation for application/json and text/event-stream. GET is the
server-to-client channel of MCP Streamable HTTP, and on the other four
domains it reaches the plugin and works.
Nothing in git or in any comment records why catalog gave it up, and the
plugin and binding behind it were applied by hand three days after the
route landed — they were never in git at all. So on a cluster rebuilt
from git the split had no guard behind it: GET fell through to a route
with no plugin whose service points back at the gateway itself.
Drop the stream route and put GET back on the MCP route, so the five
domains are identical again.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
CONTROL_PLANE_REF_NAME held the random suffix the Gateway operator gives
its auto-created mirror CR, looked up by hand. Replacing the Konnect org
recreated those CRs under new suffixes, so the ref named an object that
no longer existed and the domain's Kong entities sat Programmed=False.
Name the ${NAME}-cp mirror declared in k8s-platform's dataplane-template
instead — derived from NAME, so it stays valid across Gateway
recreations and Konnect org changes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Post http-log to /catalog instead of /logs. Fluent Bit's http input derives the
record tag from the request URI, so the path is what identifies this domain
in the collector and becomes the Loki label domain=catalog.
The collector keeps writing every domain to the on-disk audit trail as
well, so this is reversible by restoring the previous path.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
metadata.name must be an RFC 1123 subdomain (no underscores); spec.name
(the value actually matched against the JWT role claim) stays read_only/
product_manager to match the realm role names exactly.
KongConsumerGroups were still named sttlab-user/sttlab-admin, a leftover
from before the classic-models realm's role model existed. Real users'
JWTs carry realm roles (admin, read_only, product_manager, customer,
support) via consumer_groups_claim, so no group ever matched and every
tool call was rejected with 403 "consumer groups were not found".
Groups now match the realm roles directly; tool ACLs gate on each role's
catalog:read/catalog:write grant from security-model.yaml.
Body capture lives in dedicated audit-log-body-capture(-bindings).yaml
files, separate from audit-log.yaml itself, so they can be dropped
without touching the base logging plugin (degrades gracefully to
metadata-only via nil custom_fields). Response body captured in
body_filter (chunk-accumulated) rather than the log phase, which
cannot call kong.response.get_raw_body() directly.
kong.request.get_raw_body()/kong.response.get_raw_body() cannot be
called from the log phase (confirmed via dataplane error logs:
"function cannot be called in log phase") — this silently killed
log_by_lua for every plugin on the request, not just body capture.
Back to metadata-only logging while body capture gets redone properly
(pre-function/post-function storing into kong.ctx.shared, read back
in custom_fields_by_lua).
custom_fields_by_lua adds request_body/response_body to the http-log
payload. Bodies and the raw Authorization header are now both logged
in clear text — dev only, flagged in the plugin file itself.
Turned out unnecessary: the real blocker was the missing
konghq.com/secret=true label on the Vault-sourced Secret, not a gap
in the Gateway API TLS translation itself. Once that label was added
(k8s-platform), the operator's own auto-generated KongCertificate/SNI
started working and this manual one just collided with it.
Gateway API's listeners[].tls.certificateRefs never gets synced to
Konnect by the gateway operator in this setup — same working pattern
as every other resource here (KongService, KongRoute, ...): a
Kong-native CRD, not the Gateway API's own translation.
Stéphane Tailland