Stéphane Tailland
367ebd1da5
Trace search lived only in Explore. `obs-traces` brings it into a dashboard:
TraceQL assembled from a service selector, a minimum duration, a status and a
free-text field, plus the aggregate RED view from the span metrics Tempo derives
from the traces themselves.
The status and free-text variables hold TraceQL fragments rather than values, so
an empty one contributes nothing and the query stays valid — the same reason the
log dashboards inject their search term the way they do.
Both span correlations were declared but non-functional.
tracesToLogsV2 mapped service.name onto Loki's `app` label, which Alloy derives
from app.kubernetes.io/name. That label exists for Keycloak and the Classic
Models API but not for the six Kong gateways or LibreChat — and the gateways are
the only place the trace id actually appears in the logs, so the mapping pointed
squarely at the services where it could not resolve. Replaced with a custom
query on ${__trace.traceId}, which matches any source writing the id anywhere in
the line.
tracesToMetrics had no queries at all and rendered a bare datasource link. It
now carries rate, error-rate and p95 against traces_spanmetrics_*, scoped to the
span's own service.
The service graph stays a table: serviceMap is a query type Grafana's frontend
assembles for Explore and returns no frames through the dashboard query API.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
37 lines
1.2 KiB
YAML
37 lines
1.2 KiB
YAML
# Grafana dashboards -- declarative provisioning.
|
|
#
|
|
# Rules:
|
|
# * one .json file per dashboard, named after its UID;
|
|
# * every JSON carries a STABLE "uid" field (never regenerated): that is what
|
|
# keeps URLs, cross-links and alerts alive across redeployments;
|
|
# * `disableNameSuffixHash: true` -> the ConfigMap always keeps the same name,
|
|
# so `kubectl apply` updates it in place and Grafana reloads WITHOUT
|
|
# restarting the pod, in about 70s (see dashboard-providers.yaml).
|
|
#
|
|
# Adding a dashboard = drop the .json here and add it to `files:` below.
|
|
apiVersion: kustomize.config.k8s.io/v1beta1
|
|
kind: Kustomization
|
|
namespace: observability
|
|
|
|
configMapGenerator:
|
|
- name: grafana-dashboards
|
|
files:
|
|
# >>> dashboards (generated by `make dashboards-sync`) >>>
|
|
- obs-access-logs.json
|
|
- obs-application-logs.json
|
|
- obs-audit-logs.json
|
|
- obs-classic-models-api.json
|
|
- obs-keycloak.json
|
|
- obs-kong-gateways.json
|
|
- obs-librechat.json
|
|
- obs-stack-health.json
|
|
- obs-traces.json
|
|
# <<< dashboards <<<
|
|
|
|
generatorOptions:
|
|
disableNameSuffixHash: true
|
|
labels:
|
|
app.kubernetes.io/name: grafana
|
|
app.kubernetes.io/component: dashboards
|
|
app.kubernetes.io/part-of: observability
|