LibreChat served the lab CA's own certificate as its server certificate — a
CA:TRUE certificate valid for the whole *.sttlab.local wildcard, from a
hand-made tls.yaml that also carried the CA's private key. Gitignored, so never
leaked, but the only service in the estate still copying the CA around instead
of being issued a leaf.
Now a cert-manager Certificate for librechat.sttlab.local: CA:FALSE, one
dnsNames, key generated in the cluster and rotated there.
The two uses of that Secret had to be separated first, which is the part worth
knowing: sttlab-tls was both the ingress certificate and the CA bundle mounted
as NODE_EXTRA_CA_CERTS for outbound calls. Replacing it with a leaf alone would
have left LibreChat unable to verify Keycloak. The CA certificate now comes from
ca-configmap.yaml — the certificate, never the key, committed like the agents'.
Verified: the ingress serves CA:FALSE with SAN librechat.sttlab.local, the pod's
outbound call to Keycloak returns 200 with TLS verified, and the old secret is
deleted.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Point the OTLP exporter at Tempo instead of Jaeger.
Enable the built-in /metrics endpoint by setting METRICS_SECRET: without
it LibreChat answers 401 and exposes nothing. Prometheus scrapes it with a
Bearer token from a dedicated job, since annotation-based discovery cannot
carry credentials.
A Kubernetes Secret is namespaced, so the same value must also exist as
`librechat-metrics` in the observability namespace — one side verifies the
token, the other presents it. secret.example.yaml documents the pairing;
the real secret.yaml stays gitignored.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Stéphane Tailland