diff --git a/MCP-TOOL-GENERATION.md b/MCP-TOOL-GENERATION.md new file mode 100644 index 0000000..b086b2d --- /dev/null +++ b/MCP-TOOL-GENERATION.md @@ -0,0 +1,226 @@ +# MCP tool generation — what Kong does, what it does not, and what we do about it + +The five domain gateways expose their REST APIs as MCP servers through the +`ai-mcp-proxy` plugin. This file records how those tool definitions are produced, +the limits of Kong's generator, and the workarounds each limit forces on us. + +It exists because those workarounds are indistinguishable from design choices +once they are in the repo. Written down, each one is a measurement of a gap in +the product, and each can be deleted the day Kong closes it. + +**Measured 2026-08-29** against decK v1.64.0, Kong Gateway 3.15, and +`catalog-api/openapi.yaml` (the catalog domain's spec, extracted from the live +backend). Every claim below was reproduced, not read in documentation. + +## How the tool definitions are produced today + +They are not produced. `-api/kong/mcp/ai-mcp-proxy.yaml` carries a +hand-maintained `tools:` array, committed to git and edited by hand. + +The array was almost certainly generated once — every tool's +`annotations.title` is its OpenAPI `summary` verbatim, and tool names are +`operationId` with underscores turned to hyphens, which is exactly what +`deck file openapi2mcp` produces. It then stopped being regenerated while the +specs kept moving. + +That drift is not theoretical. Two defects found on 2026-08-28 and 2026-08-29, +both invisible until an agent hit them: + +- **`create-product` could never succeed.** Its schema omitted `productcode`, + which the API requires. Omit it and the API answers + `{"productcode":["This field is required."]}`; supply it and `ai-mcp-proxy` + rejects the call first with *Additional properties are not allowed + ('productcode' was unexpected)*. No input worked. +- **`list-products` cannot paginate.** The API accepts `page` and `page_size`; + the tool declares no parameters at all. The caller gets the first page, 50 of + 110 products, with nothing to say so. An agent asked how many products are in + a product line answered *38* — the true figure — while the tool had returned + a page containing 22. The number did not come from the data. The dataset is + public and the model knows it. + + This is the worst failure mode for an observable estate: the answer is right, + the trace shows a successful tool call, and nothing marks the number as + ungrounded. + +Kong's generator emits `page` and `page_size` correctly. The bug is drift, not +Kong. What follows are the things Kong genuinely does not do. + +## The generator + +`deck file openapi2mcp` is the only generally available OAS → MCP tool in the +Kong ecosystem, confirmed three ways: + +- the `ai-mcp-proxy` plugin schema (read from the compiled bytecode in the + gateway image) has no spec-ingestion field — `parameters`, `request_body` and + `responses` are per-tool OpenAPI *fragments* you supply; +- the Konnect API returns 404 for every MCP-related path + (`v3/mcp-servers`, `v2/mcp-servers`, `v3/mcp`, `v3/ai`, `v3/agents`, + `v3/tools`, `v3/composer`, `v3/ai-composer`, `v3/mcp-composer`) with a token + that returns 200 on `v2/control-planes`, `v3/apis` and `v3/portals`; +- **MCP Composer and MCP Runner never shipped.** Announced for early 2026, the + product page that carried the AI Composer now describes *Context Mesh, in + tech preview*, with a registration form and no GA date. MCP Runner is no + longer named anywhere. What did land from that roadmap is the MCP Registry — + tech preview since February 2026, and a catalogue of existing MCP servers, + not a generator. + +Caveat on the Konnect probe: the token used belongs to org +`b4531006-…`, not `2254dc3b-…` where the six domains live. Entitlements can +differ per org, so the 404s prove absence from that org, not absence from the +product. + +What it gets right, on our spec: `operationId` → kebab-case tool name matching +our convention, `summary` → `annotations.title`, `description` → description, +path *and* query parameters (including the pagination we are missing today), +and `requestBody` with `$ref`s resolved and inlined. Chained through +`deck file kong2kic` it produces a `KongPlugin` — the estate's own format. + +## The limits + +### 1. Per-tool ACLs cannot express our authorization model, and fail silently + +Kong generates per-tool ACLs only from an `oauth2` security scheme carrying the +`x-kong-mcp-acl` extension, keyed on **OAuth scopes**. + +Our authorization is realm roles → consumer groups +(`consumer_groups_claim: [realm_access, roles]`), and our specs declare +`ApiKeyAuth` and `JWTAuth` (http bearer). Neither is `oauth2`, so no ACL is +produced — and no `include_consumer_groups` either. + +**Nothing warns.** Verified at `--verbose 3`: no error, no warning, exit 0. A +regeneration would silently publish an MCP server with every operation open to +every caller, `create-*` and `delete-*` included. + +This is the most dangerous of the five, because the failure is silent and the +blast radius is the whole domain. + +Note that the *plugin* supports our model perfectly well — its schema carries +`include_consumer_groups`, `acl_attribute_type`, `access_token_claim_field`, +`acl_list_field` and `default_acl`. The gap is in the generator only, and it +matters: the ACL is what makes the estate work. Measured today, `pm-user` sees +`create-product` in `tools/list` and `ro-user` does not. Kong filters at +**discovery**, so a read-only user's turn contains no write tool for a prompt +injection to reach. + +### 2. MCP behavioural annotations are not generated + +Kong emits `annotations.title` and nothing else. The MCP specification defines +four more — `readOnlyHint`, `idempotentHint`, `destructiveHint`, +`openWorldHint` — and all four are mechanically derivable from the HTTP method. +The committed configuration has them; a regeneration would silently drop them. + +They are not decoration. An MCP client reads `destructiveHint` to decide +whether to ask its user before calling a tool. + +### 3. Response schemas are not generated, though the plugin consumes them + +The plugin schema documents a per-tool `responses` field: *"This specification +will be used to validate the upstream response and map it back to the +structuredOutput."* + +So `ai-mcp-proxy` can validate the backend's response and return structured MCP +output. Our specs contain the `responses` needed. `openapi2mcp` does not emit +them, and neither does our hand-written configuration. Every tool therefore +returns raw JSON for the model to interpret, and a backend that changes shape +is caught by nobody. + +### 4. `servers` is taken as the upstream target + +The generated Kong Service points at `catalog-dev.sttlab.local:443` — the +gateway's own public hostname, because that is what the spec documents for the +developer portal. Applied as-is, the MCP proxy would loop back into the gateway +instead of reaching the REST service (`${REST_HOSTNAME}`). + +The generator has no notion of *documented URL* versus *upstream target*. Any +spec written for a portal — which is to say any spec that is also API +documentation — hits this. + +### 5. Every content type is copied, and every operation is exposed + +DRF declares `application/json`, `application/x-www-form-urlencoded` and +`multipart/form-data` for each write operation. Kong inlines the identical +schema three times. `create-product`'s definition triples in size, for a +payload the plugin will send as JSON. There is no flag to restrict it. + +And generation is all-or-nothing: 56 tools across the five domains against the +25 we expose today, `PUT`, `PATCH` and `DELETE` included. `x-kong-mcp-exclude` +exists but lives *in the spec*, and our specs are extracted from the live +backend — the exclusions would be overwritten at the next extraction. + +## The comparison point + +FastMCP 3.4.7, run on the same `catalog-api/openapi.yaml`: + +``` +outils exposés: 4 -> ['create_product', 'get_product', 'list_product_lines', 'list_products'] +create_product readOnly=False idempotent=False destructive=False outputSchema=oui +get_product readOnly=True idempotent=True destructive=False outputSchema=oui +list_products readOnly=True idempotent=True destructive=False outputSchema=oui +``` + +Allow-listing through `route_map_fn`, the four behavioural annotations through a +six-line `mcp_component_fn` deriving them from the HTTP method, and **output +schemas generated from the OpenAPI `responses`** with no work at all. Limits 2, +3 and 5 do not exist there. + +This is a comparison point, not a proposal. FastMCP produces a standalone MCP +server, so adopting it would not swap one generator for another — it would take +Kong out of the MCP server role and put five new services in its place, moving +per-tool authorization from the realm into application code in each of them. +That is the one thing the current architecture gets right, and the reason the +recommendation below keeps Kong. + +What no tool does, Kong and FastMCP alike, is produce **decision-shaped** tools. +Every generator maps one operation to one tool. An agent asking "can I promise +delivery on 15 March?" wants one tool, not eleven CRUD endpoints to orchestrate. +FastMCP's `Tool.from_tool` allows composing higher-level tools by hand; +generating them is nobody's feature yet. + +## Workarounds + +### In place today + +**The `tools:` array is hand-maintained.** That is the workaround, and it is +also the cause of both defects above. `create-product`'s schema was repaired by +hand on 2026-08-29 — and the repair itself was wrong: it omitted +`quantityinstock` from `required`, because it was written from the `Product` +response schema instead of `ProductRequest`. Kong's generator gets that right. +The hand-maintained file is not a stable state; it is a debt accruing silently. + +### Proposed, not yet built + +Keep `openapi2mcp` as the generator and add one post-processing step that +re-injects what it cannot produce: + +| Step | Closes | Source of truth | +|---|---|---| +| Inject per-tool `acl.allow` and `include_consumer_groups` | limit 1 | a per-domain policy file, `kong/mcp/tools-policy.yaml` | +| Derive the four annotations from the HTTP method | limit 2 | the spec's method | +| Copy `responses` into each tool | limit 3 | the spec | +| Rewrite the Service host to `${REST_HOSTNAME}` | limit 4 | the domain's own variables | +| Keep `application/json` only; drop operations absent from the policy | limit 5 | the policy file | + +The policy file doubles as the allow-list: an operation becomes a tool only if +the policy grants it an ACL. Adding a tool is then a deliberate act, and the +current 25-tool surface is preserved exactly rather than silently tripled. + +Each row is a line item against Kong. If a future release generates +annotations, that row is deleted and nothing else changes. + +## Reproducing the measurements + +```bash +cd -api +deck file openapi2mcp -s openapi.yaml --no-id -o /tmp/gen.yaml # generate +deck file kong2kic -s /tmp/gen.yaml # to KongPlugin +deck file openapi2mcp -s openapi.yaml --no-id --verbose 3 -o /dev/null | grep -i warn +``` + +The plugin's own schema, if the field list is ever in doubt: + +```bash +POD=$(kubectl -n catalog get pods -o name | grep dataplane | head -1) +kubectl -n catalog exec ${POD#pod/} -- \ + cat /usr/local/share/lua/5.1/kong/plugins/ai-mcp-proxy/schema.ljbc > schema.ljbc +LC_ALL=C strings schema.ljbc | LC_ALL=C grep -E '^[a-z][a-z_]{2,30}$' | sort -u +``` diff --git a/README.md b/README.md index c001489..8ac5516 100644 --- a/README.md +++ b/README.md @@ -7,6 +7,13 @@ Per-domain gateway infrastructure (`Gateway`, `GatewayConfiguration`, `KonnectAPIAuthConfiguration`) does NOT live here — see the dedicated `dataplane-` repos, to avoid write contention across domains. +## MCP-TOOL-GENERATION.md + +How the domains' MCP tool definitions are produced, the five limits of Kong's +`deck file openapi2mcp` generator, and the workaround each one forces. Read it +before editing any `-api/kong/mcp/ai-mcp-proxy.yaml` by hand — that file +is hand-maintained today, and that is precisely the problem it documents. + ## flux/ The Flux Operator (`flux-system` namespace) that everything else in this