Stéphane Tailland
dc13738e86
The runtime for the estate's autonomous agents. Postgres as a StatefulSet with its own claim, n8n behind a cert-manager certificate and the nginx ingress, and both secrets in Vault behind an AppRole scoped to n8n/* alone. The certificate is issued into the cluster rather than copied into the repo, so the private key never reaches git. The CA certificate is committed and mounted as NODE_EXTRA_CA_CERTS, so a workflow can reach keycloak and the domain gateways without anyone disabling verification to make it work. One replica and strategy: Recreate, both deliberate — n8n needs Redis and queue mode before a second scheduler is safe, and a ReadWriteOnce volume deadlocks a rolling update. N8N_ENCRYPTION_KEY is generated once and kept in Vault. Regenerating it does not re-encrypt the stored credentials, it orphans them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
80 lines
3.3 KiB
Markdown
80 lines
3.3 KiB
Markdown
# infra-n8n
|
|
|
|
n8n on the lab cluster: the runtime for the estate's autonomous agents,
|
|
`Approvisionnement` and `Recouvrement`, whose workflows live in
|
|
`classic-models-agent-flows`.
|
|
|
|
Nothing business-specific is here. This repo deploys the engine; the workflows
|
|
are imported into it.
|
|
|
|
## Layout
|
|
|
|
| | |
|
|
|---|---|
|
|
| `k8s/postgres/` | the database, a StatefulSet with its own volume claim |
|
|
| `k8s/n8n/` | the workload, its config and its data volume |
|
|
| `k8s/ingress/` | a cert-manager `Certificate` and the nginx `Ingress` |
|
|
| `k8s/secrets/` | the `ClusterSecretStore` for this estate and the `ExternalSecret` |
|
|
| `dev/` | the Flux plumbing that applies `k8s/`, and manages itself |
|
|
|
|
Reachable at **https://n8n.sttlab.local** once `192.168.139.101` (ingress-nginx)
|
|
resolves for that name.
|
|
|
|
## Two secrets, both of which must outlive any pod
|
|
|
|
They live in Vault at `secrets/n8n/instance`, read through the `vault-n8n`
|
|
store — an AppRole whose policy can read `n8n/*` and nothing else, so a
|
|
compromise here is not a way into the keycloak estate's secrets.
|
|
|
|
- `DB_PASSWORD`
|
|
- `N8N_ENCRYPTION_KEY` — encrypts the credentials n8n stores in its database.
|
|
**Never regenerate it.** A new key does not re-encrypt what is stored; it
|
|
orphans it. The workflows survive, every credential in them does not.
|
|
|
|
## Bootstrap
|
|
|
|
Three manual applies, once. The Vault side is set up first — the commands are in
|
|
`k8s/secrets/clustersecretstore.yaml`.
|
|
|
|
```
|
|
kubectl apply -f k8s/namespace.yaml
|
|
kubectl apply -f dev/gitrepository.yaml
|
|
kubectl apply -f dev/bootstrap-kustomization.yaml
|
|
```
|
|
|
|
After that everything reconciles from git, `dev/` included. Never `kubectl
|
|
apply` the objects under `k8s/` afterwards — Flux reverts them within its
|
|
interval and the cluster silently stops matching what you edited.
|
|
|
|
## Choices worth knowing
|
|
|
|
**One replica, and `strategy: Recreate`.** n8n splits editor from workers only
|
|
in queue mode, which needs Redis; without it a second replica is two schedulers
|
|
racing over the same cron triggers. And the data volume is ReadWriteOnce, so a
|
|
rolling update would deadlock — the new pod waiting for a volume the old one has
|
|
not released.
|
|
|
|
**The certificate is issued, not committed.** cert-manager writes `n8n-tls` from
|
|
the `sttlab-local-ca` ClusterIssuer. The private key stays in the cluster.
|
|
|
|
**The CA certificate *is* committed**, in `k8s/ca-configmap.yaml`, and mounted
|
|
as `NODE_EXTRA_CA_CERTS`. It is a certificate, never a key, and checking it in
|
|
means a workflow can call `keycloak.sttlab.local` and the domain gateways
|
|
without anyone disabling verification to make it work. Regenerate on rotation —
|
|
it expires 2026-11-24.
|
|
|
|
**`N8N_HOST` and `WEBHOOK_URL` are the ingress host, not the Service.** They go
|
|
into the webhook URLs n8n hands out; advertised as the in-cluster name, they
|
|
would be URLs nothing outside can call.
|
|
|
|
**Executions are pruned after two weeks.** The default keeps them for ever,
|
|
which turns the database into a log.
|
|
|
|
## What is deliberately not here
|
|
|
|
Authentication. n8n's own user management is left at its defaults, and the
|
|
instance is reachable to anyone who can resolve the host on the lab network.
|
|
Before it holds anything that matters, it needs either n8n's SSO against the
|
|
`classic-models` realm or an `openid-connect` filter at the ingress — the same
|
|
Keycloak the rest of the estate already uses.
|