Add a Kong gateways dashboard

One dashboard for all six gateways, with a selector accepting one, several or
all of them.

The selector has to bridge a mismatch: metrics carry the full gateway name
(catalog-dev) while audit logs carry the short domain (catalog), because the
former comes from the scrape job's label and the latter from the URI path each
gateway posts its http-log to. The variable holds the short form, PromQL
re-adds the -dev suffix and LogQL uses it as-is, so a single control drives
both signals. The capture group in the variable's regex also excludes the
unrelated `kong` gateway, which has no -dev suffix.

Latency is split three ways rather than shown as one number: total request
latency, time spent inside Kong, and upstream latency. The gap between the
first and the last is what the gateway itself costs, which is the question
worth asking of a proxy.

Status classes come from label_replace on the `code` label for metrics and
from LogQL's json parser for the audit records, since those carry no severity
of their own.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

README.md

@@ -191,6 +191,14 @@ changes in `container_start_time_seconds`.
 | `librechat` | OTel SDK → `tempo:4318` | built-in `/metrics`, Bearer token | stdout | `obs-librechat` |
 | `classic-models-api` | `opentelemetry-instrument` → `tempo:4318` | django-prometheus `/metrics` | JSON + `otelTraceID` | `obs-classic-models-api` |
 
+The six Kong gateways share a single dashboard, `obs-kong-gateways`, with a
+`gateway` selector at the top (one, several, or all). Its variable resolves a
+subtlety of this setup: the metrics label is the full gateway name
+(`catalog-dev`) while the audit-log label is the short domain (`catalog`). The
+variable holds the short form, PromQL re-adds the `-dev` suffix, and LogQL uses
+it as-is — so one control drives both signals. The capture group in its regex
+also drops the unrelated `kong` gateway, which carries no `-dev` suffix.
+
 Their manifests live in the `infra-keycloak`, `infra-librechat` and
 `classic-models-api` repositories; this repository only holds the platform.
 

k8s/grafana/dashboards/kustomization.yaml

@@ -19,6 +19,7 @@ configMapGenerator:
       # >>> dashboards (generated by `make dashboards-sync`) >>>
       - obs-classic-models-api.json
       - obs-keycloak.json
+      - obs-kong-gateways.json
       - obs-librechat.json
       - obs-stack-health.json
       # <<< dashboards <<<