fix: serve a leaf certificate, not the CA

LibreChat served the lab CA's own certificate as its server certificate — a CA:TRUE certificate valid for the whole *.sttlab.local wildcard, from a hand-made tls.yaml that also carried the CA's private key. Gitignored, so never leaked, but the only service in the estate still copying the CA around instead of being issued a leaf. Now a cert-manager Certificate for librechat.sttlab.local: CA:FALSE, one dnsNames, key generated in the cluster and rotated there. The two uses of that Secret had to be separated first, which is the part worth knowing: sttlab-tls was both the ingress certificate and the CA bundle mounted as NODE_EXTRA_CA_CERTS for outbound calls. Replacing it with a leaf alone would have left LibreChat unable to verify Keycloak. The CA certificate now comes from ca-configmap.yaml — the certificate, never the key, committed like the agents'. Verified: the ingress serves CA:FALSE with SAN librechat.sttlab.local, the pod's outbound call to Keycloak returns 200 with TLS verified, and the old secret is deleted. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-29 18:01:00 +02:00
parent bfc748a925
commit 26aa89643c
7 changed files with 86 additions and 23 deletions
@@ -1,6 +1,8 @@
 # Secrets — ne jamais committer de valeurs réelles
 secret.yaml
-tls.yaml
+# tls.yaml : plus de fichier TLS local. Le certificat de service est émis par
+# cert-manager (ingress/certificate.yaml) et sa clé reste dans le cluster ; le
+# certificat du CA, lui, est public et versionné dans ca-configmap.yaml.

 # Certificats TLS
 *.crt
@@ -44,7 +44,8 @@ k8s/
 │   └── service.yaml
 └── ingress/
    ├── ingress.yaml
-    └── tls.yaml               TLS secret (gitignored — never commit)
+    ├── certificate.yaml       cert-manager Certificate → librechat-tls
+    └── (no TLS file: the key is generated in the cluster)
 ```

 ## Prerequisites
@@ -72,19 +73,22 @@ openssl rand -hex 8       # CREDS_IV
 mongodb://librechat:<password>@mongodb:27017/LibreChat?authSource=admin
 ```

-### 2. Create the TLS secret
+### 2. TLS

-```bash
-kubectl apply -f k8s/namespace.yaml
+Nothing to create. `ingress/certificate.yaml` asks cert-manager for a leaf
+certificate for `librechat.sttlab.local`, issued by the `sttlab-local-ca`
+ClusterIssuer into the `librechat-tls` Secret. The private key is generated in
+the cluster, rotated automatically, and never exists as a file.

-kubectl create secret tls sttlab-tls \
-  --cert=$HOME/tls/sttlab.pc.crt \
-  --key=$HOME/tls/sttlab.pc.key \
-  --namespace=librechat \
-  --dry-run=client -o yaml > k8s/ingress/tls.yaml
+This replaced a hand-made `tls.yaml` holding the lab CA's own certificate *and
+private key*, which LibreChat then served as its server certificate — a
+`CA:TRUE` certificate valid for the whole `*.sttlab.local` wildcard where a leaf
+for one name is what a server needs.

-kubectl apply -f k8s/ingress/tls.yaml
-```
+The CA **certificate** is still needed, for the outbound calls LibreChat makes
+to Keycloak and the domain gateways. It lives in `ca-configmap.yaml`, is mounted
+as `NODE_EXTRA_CA_CERTS`, and is committed — a certificate is public by
+construction; the key is what must never be in a file.

 ### 3. Deploy

@@ -0,0 +1,38 @@
+# The lab CA, so LibreChat can verify keycloak.sttlab.local and the domain
+# gateways rather than disabling verification.
+#
+# The certificate, never the key — which is the whole difference from the
+# `ingress/tls.yaml` this replaces. A CA certificate is public by construction, so it is
+# committed and the deployment has no hidden prerequisite.
+#
+# Issued by the sttlab-local-ca ClusterIssuer, valid until 2026-11-24.
+# Regenerate on rotation:
+#   kubectl -n cert-manager get secret sttlab-local-ca \
+#     -o jsonpath='{.data.tls\.crt}' | base64 -d
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: sttlab-ca
+  namespace: librechat
+data:
+  ca.crt: |
+    -----BEGIN CERTIFICATE-----
+    MIIDPDCCAiSgAwIBAgIUd/utDf+f0yQR3v3qTjZqx+wvTZMwDQYJKoZIhvcNAQEL
+    BQAwGTEXMBUGA1UEAwwOKi5zdHRsYWIubG9jYWwwHhcNMjUxMTI0MTUzODQ1WhcN
+    MjYxMTI0MTUzODQ1WjAZMRcwFQYDVQQDDA4qLnN0dGxhYi5sb2NhbDCCASIwDQYJ
+    KoZIhvcNAQEBBQADggEPADCCAQoCggEBALj0sryq8qXtkShT/B+/EjVJNN/TLHIn
+    QJYc6xuUH/f7q3dYdLm4iH0XnFgrSQLUo4iqd/Mczef5iWCYwT5FFa7P17/TcLVf
+    gso0j41qNLBjGxDQGQ6Z1VCxi6oan8ow98rumjVj0E0Qgs5CX/rN+gcEmZ/TZFWU
+    qudVoztpeiVqJ0dxCIq+KMutzd1BU7BXx4VRO9Y+t1idUc7oY7BPmS2qPAN7TtXP
+    j1KhH2BgLsmF5GRhYHNS578eCltcmS2sbQ3c7GwOV96lpgF9TixEvFXeN4xk43E6
+    1COUD7CRiWAZ0XI8FMlpZGeNZeV4bbV/MqVig1ybWNOz9MDKYj3eoXUCAwEAAaN8
+    MHowHQYDVR0OBBYEFDbzYWOPr4vxa+VONs7xHbcNx70YMB8GA1UdIwQYMBaAFDbz
+    YWOPr4vxa+VONs7xHbcNx70YMA8GA1UdEwEB/wQFMAMBAf8wJwYDVR0RBCAwHoIO
+    Ki5zdHRsYWIubG9jYWyCDHN0dGxhYi5sb2NhbDANBgkqhkiG9w0BAQsFAAOCAQEA
+    tU4p5/ONXm7x4YDzSeL8NdThVLguiFur0jcTxvvPUS4NFNsnQJk4OXxBgjy1YwpL
+    3i56/4mCEd9OCQF/Hl0eFwGZBzNKlMoTA08AC/lBDvPrnVjuUu0qe/b/2KOkhESa
+    x8rQNp0o8pcj0f29nu0JfRqYGkzy0Lh2qgXhPu2iC+t2PfdleRAky1tt68wogdFL
+    TvHDsJbVo/ihsPpONlSyHfxG6bb/0Mkz64PH46TTt9GvJflEKMqDwivNFUA1P5S/
+    U4Ic8Qapnxc7h2+WbRjmYrnuygC2wMBE6rL9OF3gwCr1fkgGGRr5lVRUq2X2HZV5
+    5jqViv6eurq8O61N3vWGCQ==
+    -----END CERTIFICATE-----
@@ -0,0 +1,21 @@
+# cert-manager issues this into the `librechat-tls` Secret.
+#
+# It replaces a hand-made `tls.yaml` that carried the lab CA's own certificate
+# and private key, which LibreChat then served as its server certificate — a
+# CA:TRUE certificate valid for the whole `*.sttlab.local` wildcard, where a
+# leaf for one name is what a server needs.
+#
+# The key is generated in the cluster, rotated by cert-manager, and never
+# reaches a file anyone has to remember not to commit.
+apiVersion: cert-manager.io/v1
+kind: Certificate
+metadata:
+  name: librechat-tls
+  namespace: librechat
+spec:
+  secretName: librechat-tls
+  dnsNames:
+    - librechat.sttlab.local
+  issuerRef:
+    name: sttlab-local-ca
+    kind: ClusterIssuer
@@ -25,7 +25,7 @@ spec:
  tls:
    - hosts:
        - librechat.sttlab.local
-      secretName: sttlab-tls
+      secretName: librechat-tls
  rules:
    - host: librechat.sttlab.local
      http:
@@ -5,6 +5,7 @@ resources:
  - namespace.yaml
  - secret.yaml
  - configmap.yaml
+  - ca-configmap.yaml
  - mongodb/pvc.yaml
  - mongodb/statefulset.yaml
  - mongodb/service.yaml
@@ -14,5 +15,5 @@ resources:
  - librechat/configmap-app.yaml
  - librechat/deployment.yaml
  - librechat/service.yaml
-  - ingress/tls.yaml
+  - ingress/certificate.yaml
  - ingress/ingress.yaml
@@ -136,7 +136,7 @@ spec:
            # NODE_TLS_REJECT_UNAUTHORIZED=0 which would disable TLS
            # verification globally.
            - name: NODE_EXTRA_CA_CERTS
-              value: /etc/sttlab-tls/tls.crt
+              value: /etc/sttlab-ca/ca.crt
            # 512Mi was too tight for LibreChat under real use (MCP clients,
            # OTel instrumentation) and caused repeated V8 heap-limit crashes.
            # Cap old-space explicitly, leaving headroom under the new
@@ -166,16 +166,13 @@ spec:
            - name: app-config
              mountPath: /app/librechat.yaml
              subPath: librechat.yaml
-            - name: sttlab-tls-cert
-              mountPath: /etc/sttlab-tls
+            - name: sttlab-ca
+              mountPath: /etc/sttlab-ca
              readOnly: true
      volumes:
        - name: app-config
          configMap:
            name: librechat-app-config
-        - name: sttlab-tls-cert
-          secret:
-            secretName: sttlab-tls
-            items:
-              - key: tls.crt
-                path: tls.crt
+        - name: sttlab-ca
+          configMap:
+            name: sttlab-ca