Stéphane Tailland
9bf9ecc358
Exposes Prometheus, Loki and Tempo as tools an LLM can call, so a question
like "what failed in the last transaction" is answered by querying rather
than by building another panel — which is where the trace dashboard kept
running out of road.
Works against OSS Grafana; the Cloud-only tools (Incidents, OnCall, Sift,
Agent Observability, Assistant) are simply absent. Verified end to end:
the MCP handshake succeeds and list_datasources returns Loki, Prometheus
and Tempo.
Authenticates with a Grafana service account token, Viewer role, created
once through /api/serviceaccounts and stored in the gitignored
k8s/secret.yaml. secret.example.yaml documents how to mint it, since
Grafana returns the value only at creation.
Two things the deployment needs that are not obvious:
* runAsUser must be numeric. The image declares USER mcp-grafana by
name, Kubernetes cannot resolve that to check it is not root, and
refuses to start the container under runAsNonRoot with "image has
non-numeric user". 1000/999 is what the image actually uses.
* --allowed-hosts must name the Service. The server validates the Host
header against a loopback-only default as DNS-rebinding protection, so
the first in-cluster call returned "forbidden: host not allowed".
Named rather than disabled with "*": the check is worth keeping.
Deliberately in-cluster only, with no OIDC and no audit trail — anything
that can reach this Service reads all the telemetry. The intended end
state is the same treatment as the five domain MCP servers, behind a Kong
gateway with openid-connect, consumer groups and http-log.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
18 lines
737 B
YAML
18 lines
737 B
YAML
# Template: copy to secret.yaml (gitignored) and fill in the values.
|
|
# make secret
|
|
apiVersion: v1
|
|
kind: Secret
|
|
metadata:
|
|
name: observability-secrets
|
|
namespace: observability
|
|
type: Opaque
|
|
stringData:
|
|
GF_SECURITY_ADMIN_USER: admin
|
|
GF_SECURITY_ADMIN_PASSWORD: <CHANGE_ME>
|
|
# Token of a Grafana service account, created once with the admin credentials:
|
|
# POST /api/serviceaccounts {"name":"mcp-grafana","role":"Viewer"}
|
|
# POST /api/serviceaccounts/<id>/tokens {"name":"mcp-grafana-token"}
|
|
# Grafana returns the value once, at creation. Viewer is enough to query the
|
|
# datasources and list dashboards; raise it only if a tool needs writes.
|
|
GRAFANA_SERVICE_ACCOUNT_TOKEN: glsa_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|