From 9893221da0f4c48fb2be09057d6280bb578c9d8c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?St=C3=A9phane=20Tailland?= Date: Thu, 27 Aug 2026 06:48:31 +0200 Subject: [PATCH] Propagate trace context on the AI gateway MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This gateway declared its own copy of the shared observability plugins, claiming parity with k8s-platform's dataplane-globalpolicy-template. The propagation block was later added to that template and never mirrored here, so while the five business domains continued the caller's trace, this one minted a fresh trace id on every request and injected nothing upstream. That breaks end-to-end tracing at the worst possible hop: LibreChat enters here, so a transaction that fans out to MCP, the LLM and the backend was recorded as a set of unrelated traces. Verified live before the fix — the domain gateways reported extract: [w3c, b3, jaeger] / inject: [w3c], this one reported extract: null / inject: null. Copy the block across and flag in the header comment that the parity is manual and has already drifted once. Co-Authored-By: Claude Opus 5 --- kong/observability.yaml | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/kong/observability.yaml b/kong/observability.yaml index b213c10..46a3ac2 100644 --- a/kong/observability.yaml +++ b/kong/observability.yaml @@ -6,6 +6,12 @@ # defines its own AI-specific versions of. Applying it here would duplicate # them, so the three observability plugins are declared locally instead, # with the same configuration as the shared template. +# +# That parity is manual and has already broken once: the propagation block +# below was added to the shared template and not here, so this gateway kept +# starting a fresh trace per request while the five domains propagated +# correctly. Any change to the shared template's observability plugins has to +# be mirrored here by hand. apiVersion: configuration.konghq.com/v1 kind: KongPlugin metadata: @@ -30,6 +36,23 @@ config: service.name: ${NAME} sampling_rate: 1 http_response_header_for_traceid: X-Trace-Id + # Without this block every hop starts its own trace: a single end-to-end + # transaction shows up in Tempo as a dozen unrelated traces, one per service. + # `extract` is what makes the gateway continue the caller's trace instead of + # opening a new one, and `inject` is what carries it to the upstream so the + # backend can continue it in turn. + # + # This gateway is where LibreChat enters, so without it the chain breaks at + # the very first hop and nothing downstream can be correlated. + propagation: + extract: + - w3c + - b3 + - jaeger + inject: + - w3c + default_format: w3c + clear: [] --- apiVersion: configuration.konghq.com/v1 kind: KongPlugin