Trace search lived only in Explore. `obs-traces` brings it into a dashboard:
TraceQL assembled from a service selector, a minimum duration, a status and a
free-text field, plus the aggregate RED view from the span metrics Tempo derives
from the traces themselves.
The status and free-text variables hold TraceQL fragments rather than values, so
an empty one contributes nothing and the query stays valid — the same reason the
log dashboards inject their search term the way they do.
Both span correlations were declared but non-functional.
tracesToLogsV2 mapped service.name onto Loki's `app` label, which Alloy derives
from app.kubernetes.io/name. That label exists for Keycloak and the Classic
Models API but not for the six Kong gateways or LibreChat — and the gateways are
the only place the trace id actually appears in the logs, so the mapping pointed
squarely at the services where it could not resolve. Replaced with a custom
query on ${__trace.traceId}, which matches any source writing the id anywhere in
the line.
tracesToMetrics had no queries at all and rendered a bare datasource link. It
now carries rate, error-rate and p95 against traces_spanmetrics_*, scoped to the
span's own service.
The service graph stays a table: serviceMap is a query type Grafana's frontend
assembles for Explore and returns no frames through the dashboard query API.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Stéphane Tailland
367ebd1da5