Stéphane Tailland
9893221da0
This gateway declared its own copy of the shared observability plugins, claiming parity with k8s-platform's dataplane-globalpolicy-template. The propagation block was later added to that template and never mirrored here, so while the five business domains continued the caller's trace, this one minted a fresh trace id on every request and injected nothing upstream. That breaks end-to-end tracing at the worst possible hop: LibreChat enters here, so a transaction that fans out to MCP, the LLM and the backend was recorded as a set of unrelated traces. Verified live before the fix — the domain gateways reported extract: [w3c, b3, jaeger] / inject: [w3c], this one reported extract: null / inject: null. Copy the block across and flag in the header comment that the parity is manual and has already drifted once. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
119 lines
3.7 KiB
YAML
119 lines
3.7 KiB
YAML
# Observability plugins for the AI gateway.
|
|
#
|
|
# The five business domains get these from k8s-platform's
|
|
# dataplane-globalpolicy-template, but that template also carries
|
|
# openid-connect, rate-limiting and cors — all of which this gateway already
|
|
# defines its own AI-specific versions of. Applying it here would duplicate
|
|
# them, so the three observability plugins are declared locally instead,
|
|
# with the same configuration as the shared template.
|
|
#
|
|
# That parity is manual and has already broken once: the propagation block
|
|
# below was added to the shared template and not here, so this gateway kept
|
|
# starting a fresh trace per request while the five domains propagated
|
|
# correctly. Any change to the shared template's observability plugins has to
|
|
# be mirrored here by hand.
|
|
apiVersion: configuration.konghq.com/v1
|
|
kind: KongPlugin
|
|
metadata:
|
|
name: ${NAME}-llm-prometheus
|
|
namespace: ${NAMESPACE}
|
|
plugin: prometheus
|
|
config:
|
|
status_code_metrics: true
|
|
latency_metrics: true
|
|
bandwidth_metrics: true
|
|
upstream_health_metrics: true
|
|
---
|
|
apiVersion: configuration.konghq.com/v1
|
|
kind: KongPlugin
|
|
metadata:
|
|
name: ${NAME}-llm-opentelemetry
|
|
namespace: ${NAMESPACE}
|
|
plugin: opentelemetry
|
|
config:
|
|
traces_endpoint: ${OTEL_TRACES_ENDPOINT}
|
|
resource_attributes:
|
|
service.name: ${NAME}
|
|
sampling_rate: 1
|
|
http_response_header_for_traceid: X-Trace-Id
|
|
# Without this block every hop starts its own trace: a single end-to-end
|
|
# transaction shows up in Tempo as a dozen unrelated traces, one per service.
|
|
# `extract` is what makes the gateway continue the caller's trace instead of
|
|
# opening a new one, and `inject` is what carries it to the upstream so the
|
|
# backend can continue it in turn.
|
|
#
|
|
# This gateway is where LibreChat enters, so without it the chain breaks at
|
|
# the very first hop and nothing downstream can be correlated.
|
|
propagation:
|
|
extract:
|
|
- w3c
|
|
- b3
|
|
- jaeger
|
|
inject:
|
|
- w3c
|
|
default_format: w3c
|
|
clear: []
|
|
---
|
|
apiVersion: configuration.konghq.com/v1
|
|
kind: KongPlugin
|
|
metadata:
|
|
name: ${NAME}-llm-audit-log
|
|
namespace: ${NAMESPACE}
|
|
# dev only: the raw Authorization header is captured in clear text, which is
|
|
# not acceptable as-is outside dev.
|
|
plugin: http-log
|
|
config:
|
|
http_endpoint: ${AUDIT_LOG_ENDPOINT}
|
|
method: POST
|
|
content_type: application/json
|
|
---
|
|
# prometheus and opentelemetry are bound control-plane-wide, matching the
|
|
# shared template: they describe the gateway itself, not one service.
|
|
apiVersion: configuration.konghq.com/v1alpha1
|
|
kind: KongPluginBinding
|
|
metadata:
|
|
name: ${NAME}-llm-prometheus-binding
|
|
namespace: ${NAMESPACE}
|
|
spec:
|
|
controlPlaneRef:
|
|
type: konnectNamespacedRef
|
|
konnectNamespacedRef:
|
|
name: ${CONTROL_PLANE_REF_NAME}
|
|
pluginRef:
|
|
name: ${NAME}-llm-prometheus
|
|
scope: GlobalInControlPlane
|
|
---
|
|
apiVersion: configuration.konghq.com/v1alpha1
|
|
kind: KongPluginBinding
|
|
metadata:
|
|
name: ${NAME}-llm-opentelemetry-binding
|
|
namespace: ${NAMESPACE}
|
|
spec:
|
|
controlPlaneRef:
|
|
type: konnectNamespacedRef
|
|
konnectNamespacedRef:
|
|
name: ${CONTROL_PLANE_REF_NAME}
|
|
pluginRef:
|
|
name: ${NAME}-llm-opentelemetry
|
|
scope: GlobalInControlPlane
|
|
---
|
|
# The audit log is bound to the proxy service, like the business domains bind
|
|
# theirs, so only real gateway traffic is captured.
|
|
apiVersion: configuration.konghq.com/v1alpha1
|
|
kind: KongPluginBinding
|
|
metadata:
|
|
name: ${NAME}-llm-audit-log-binding
|
|
namespace: ${NAMESPACE}
|
|
spec:
|
|
controlPlaneRef:
|
|
type: konnectNamespacedRef
|
|
konnectNamespacedRef:
|
|
name: ${CONTROL_PLANE_REF_NAME}
|
|
pluginRef:
|
|
name: ${NAME}-llm-audit-log
|
|
targets:
|
|
serviceRef:
|
|
group: configuration.konghq.com
|
|
kind: KongService
|
|
name: ${NAME}-llm-proxy
|