Two ceilings had to clear 30 KB prompts, and neither alone was enough:
nginx's client_body_buffer_size, which decides whether the body stays in
memory at all, and ai-proxy's max_request_body_size, which decides whether
the plugin will introspect it. With the first at 8k the body went to a
temp file and kong.request.get_raw_body() returned nil, so log_payloads
and log_statistics were enabled and empty, and no kong_ai_llm_* metric
ever appeared.
256k here against the 8k the domain gateways keep. Affordable because this
gateway is low-traffic; the buffer is per connection carrying a body, so
copying the value to the domain gateways needs that arithmetic redone for
their volume.
Storage cost measured rather than assumed: 18 LLM requests a day at
roughly 60 KB of prompt and completion is 1.1 MiB/day, against 425 MiB/day
of Kong audit records already flowing. My earlier warning that the audit
trail would grow fast was wrong at this volume.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
ai-gateway-kustomization sat in flux-system, so the team's own bootstrap
was writing into a namespace the team has no access to — the thing the
move was meant to end. It follows the others into llm, with an explicit
sourceRef namespace since k8s-platform stays in flux-system.
The control-plane ExternalSecret goes with it, for the same reason it did
in the dataplane repos: it only lived in flux-system because
substituteFrom resolves in the Kustomization's own namespace.
And this retires a workaround. proxy-kustomization inlined
OTEL_TRACES_ENDPOINT because it must stay in llm — it substitutes from a
Secret holding OPENROUTER_BEARER and LIBRECHAT_APIKEY, which must not be
copied into flux-system — and could therefore not read
dataplane-template-defaults, which existed only there. k8s-platform now
places that ConfigMap in every domain namespace, so the reference works
and the duplicated value goes. Changing the cluster trace backend is one
edit again.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two hardcoded values tied this repo to one specific Konnect org, and
replacing that org broke both.
CONTROL_PLANE_ID was pasted into gateway-kustomization.yaml. It now
comes from Vault through External Secrets, into a Secret in flux-system
that postBuild.substituteFrom reads. flux-system is acceptable for the
id — it is visible in the Konnect UI URL — but not for the token, which
keeps going straight to the domain namespace as before.
CONTROL_PLANE_REF_NAME held the random suffix the Gateway operator gives
its auto-created mirror CR, looked up by hand. The new org recreated
those CRs under new suffixes, so the ref named an object that no longer
existed and the domain's Kong entities sat Programmed=False. It now
names the ${NAME}-cp mirror declared in k8s-platform's
dataplane-template — derived from NAME, stable across Gateway
recreations and org changes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Stéphane Tailland