Twenty-four Flux objects across eleven repos, none of them managed by
Flux. Each repo declares its own GitRepository and Kustomization, which
is the right ownership model, but nothing ever watched the directory
those files live in — so editing one in git changed nothing until
someone remembered to kubectl apply it. That cost real time twice this
week: the CONTROL_PLANE_REF_NAME rollout looked applied and was not, and
LLM_INPUT_COST failed the build with "variable not set" while sitting
correctly in git.
The gap is the bootstrap Kustomization that `flux bootstrap` would have
created and flux-operator does not. Add one per repo, watching that
repo's own ./dev, living in ./dev so it manages itself. Nothing moves
between repos; each domain keeps its entrypoints.
This is the first of eleven, starting with platform-llm because its cost
settings are blocked behind exactly this.
Needs one kubectl apply to exist. That is the last one this repo needs.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Four settings, all of them off, which is why the AI gateway produced
HTTP-level telemetry and nothing about the model.
logging.log_payloads the prompt and the completion, in the log
records and — since OTel instrumentation is on
here — in the spans too
logging.log_statistics model usage and token counts
prometheus.ai_metrics kong_ai_llm_* : requests, tokens by type, cost,
provider latency, per model and per consumer
input_cost / output_cost cost per 1M tokens, which is what makes
kong_ai_llm_cost_total mean anything
Also removes this morning's pre-function/post-function body capture from
this gateway. It was the wrong tool for the LLM route: Kong's own
implementation reassembles a streamed SSE answer, while accumulating raw
chunks in kong.ctx.shared does not. The five business domains keep theirs
— their MCP traffic is not streamed the same way.
The costs are a snapshot, checked against OpenRouter's price list for
google/gemini-3.7-flash on 2026-08-27: $0.375 in, $1.875 out per 1M
tokens. They are substitution variables rather than literals precisely
because a stale figure produces confidently wrong money rather than an
error.
Note what this does NOT fix: usage is missing on streamed responses
because the OpenAI-compatible API only reports it when the caller asks
for stream_options.include_usage, and that is LibreChat's request to
make, not Kong's. Expect token counts to stay partial until then.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The audit trail recorded metadata only for this gateway, so the prompts
sent to the LLM and the completions coming back — the one thing worth
auditing on an AI gateway — were never written down. The five business
domains have captured bodies since they were built; this gateway declares
its own copy of the observability plugins and never got them.
Adds the same pre-function/post-function pair the domains use, bound to
the proxy service, and the custom_fields_by_lua that reads them back in
the log phase. If the capture plugins are ever removed the fields go nil
and are silently omitted, so the log degrades to metadata rather than
breaking.
The response accumulator earns its keep here more than elsewhere: LLM
answers stream as SSE, so body_filter runs once per chunk and the
complete answer only exists after the last one.
Still dev-only, for the same reason as the rest of this file: it puts
prompt and completion content in the log stream.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This gateway declared its own copy of the shared observability plugins,
claiming parity with k8s-platform's dataplane-globalpolicy-template. The
propagation block was later added to that template and never mirrored
here, so while the five business domains continued the caller's trace,
this one minted a fresh trace id on every request and injected nothing
upstream.
That breaks end-to-end tracing at the worst possible hop: LibreChat
enters here, so a transaction that fans out to MCP, the LLM and the
backend was recorded as a set of unrelated traces. Verified live before
the fix — the domain gateways reported
extract: [w3c, b3, jaeger] / inject: [w3c], this one reported
extract: null / inject: null.
Copy the block across and flag in the header comment that the parity is
manual and has already drifted once.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This Kustomization pulled dataplane-template-defaults via substituteFrom
with optional: false, but that ConfigMap is deliberately single-instance
in flux-system and this object lives in the llm namespace, so the build
failed outright: configmaps "dataplane-template-defaults" not found.
That is why the stack was never in the cluster and the llm KongService,
KongRoute and plugin bindings had been applied by hand instead.
Moving the Kustomization to flux-system would fix the lookup and break
something worse: its substitution Secret holds OPENROUTER_BEARER and
LIBRECHAT_APIKEY, and copying real credentials into flux-system is
exactly what the Konnect PAT comment in gateway-kustomization.yaml
refuses to do.
OTEL_TRACES_ENDPOINT was the only key this stack took from that
ConfigMap, so inline it and drop the reference. The trade-off is
recorded next to the value: changing the cluster trace backend now takes
an edit here too.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two hardcoded values tied this repo to one specific Konnect org, and
replacing that org broke both.
CONTROL_PLANE_ID was pasted into gateway-kustomization.yaml. It now
comes from Vault through External Secrets, into a Secret in flux-system
that postBuild.substituteFrom reads. flux-system is acceptable for the
id — it is visible in the Konnect UI URL — but not for the token, which
keeps going straight to the domain namespace as before.
CONTROL_PLANE_REF_NAME held the random suffix the Gateway operator gives
its auto-created mirror CR, looked up by hand. The new org recreated
those CRs under new suffixes, so the ref named an object that no longer
existed and the domain's Kong entities sat Programmed=False. It now
names the ${NAME}-cp mirror declared in k8s-platform's
dataplane-template — derived from NAME, stable across Gateway
recreations and org changes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This gateway had none of the three: no prometheus, opentelemetry or http-log
plugin, so it produced only Kong's own node and shared-dict metrics and was
invisible in Tempo and Loki.
The five business domains inherit these from k8s-platform's
dataplane-globalpolicy-template, but that template also carries
openid-connect, rate-limiting and cors, and this gateway already defines its
own AI-specific versions of all three. Applying it here would duplicate them,
so the observability plugins are declared locally with the same configuration
as the shared template.
prometheus and opentelemetry bind control-plane-wide since they describe the
gateway itself; the audit log binds to the proxy service, matching how the
domains bind theirs.
The trace endpoint now comes from the shared dataplane-template-defaults
ConfigMap rather than being repeated here, so a change of trace backend does
not need a second edit. Its only key overlapping the inline block is
OIDC_ISSUER, and both carry the same value.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Stéphane Tailland