refactor: make this a reusable deployment, not ours
Every environment value is a ${VAR} for the caller to substitute: hostname,
images, storage, ingress class, TLS issuer, Vault server and path, timezone,
retention. Nothing in k8s/ names a host, a realm, an agent or an estate.
Removed: the Flux plumbing under dev/, which described one deployment, and the
ExternalSecret pulling a specific agent's Keycloak client secret. Both belong to
whoever deploys this. The lab CA ConfigMap goes with them — the mount stays,
parameterised and marked optional, because 'trust a private CA' is a property of
an environment and not of n8n.
The README is now the template's: how to consume it, what every variable means,
and the three things it deliberately does not do.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -1,79 +1,95 @@
|
|||||||
# infra-n8n
|
# infra-n8n
|
||||||
|
|
||||||
n8n on the lab cluster: the runtime for the estate's autonomous agents,
|
A reusable n8n deployment: Postgres, the workload, TLS and an ingress, with
|
||||||
`Approvisionnement` and `Recouvrement`, whose workflows live in
|
every environment value left as a `${VAR}` for the caller to substitute.
|
||||||
`classic-models-agent-flows`.
|
|
||||||
|
|
||||||
Nothing business-specific is here. This repo deploys the engine; the workflows
|
**Nothing here is specific to a deployment.** No hostname, no issuer, no Vault
|
||||||
are imported into it.
|
path, no image tag. A consumer supplies those through its own Flux
|
||||||
|
`Kustomization` and its `postBuild.substitute` block — the same shape used for
|
||||||
|
any templated estate. This repo is the template; it is not itself deployed.
|
||||||
|
|
||||||
## Layout
|
## Using it
|
||||||
|
|
||||||
| | |
|
Create, in the repo that owns the deployment:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
apiVersion: source.toolkit.fluxcd.io/v1
|
||||||
|
kind: GitRepository
|
||||||
|
metadata: { name: infra-n8n, namespace: <ns> }
|
||||||
|
spec:
|
||||||
|
interval: 5m
|
||||||
|
url: <this repo>
|
||||||
|
ref: { branch: main }
|
||||||
|
---
|
||||||
|
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
||||||
|
kind: Kustomization
|
||||||
|
metadata: { name: n8n, namespace: <ns> }
|
||||||
|
spec:
|
||||||
|
interval: 10m
|
||||||
|
sourceRef: { kind: GitRepository, name: infra-n8n }
|
||||||
|
path: ./k8s
|
||||||
|
prune: true
|
||||||
|
wait: true
|
||||||
|
postBuild:
|
||||||
|
substitute:
|
||||||
|
NAMESPACE: <ns>
|
||||||
|
# … every variable in the table below
|
||||||
|
```
|
||||||
|
|
||||||
|
The namespace must exist first, and so must the AppRole Secret the store reads.
|
||||||
|
|
||||||
|
## Variables
|
||||||
|
|
||||||
|
| Variable | What it is |
|
||||||
|---|---|
|
|---|---|
|
||||||
| `k8s/postgres/` | the database, a StatefulSet with its own volume claim |
|
| `NAMESPACE` | where everything lands |
|
||||||
| `k8s/n8n/` | the workload, its config and its data volume |
|
| `N8N_HOSTNAME` | the public host. It goes into `WEBHOOK_URL` and the editor's links, so it must be what callers outside the cluster use — an in-cluster name here produces webhooks nothing can call |
|
||||||
| `k8s/ingress/` | a cert-manager `Certificate` and the nginx `Ingress` |
|
| `N8N_IMAGE`, `POSTGRES_IMAGE` | pinned images, tag included |
|
||||||
| `k8s/secrets/` | the `ClusterSecretStore` for this estate and the `ExternalSecret` |
|
| `N8N_STORAGE`, `POSTGRES_STORAGE` | claim sizes, e.g. `2Gi`, `5Gi` |
|
||||||
| `dev/` | the Flux plumbing that applies `k8s/`, and manages itself |
|
| `TIMEZONE` | e.g. `Europe/Paris`; drives cron triggers |
|
||||||
|
| `INGRESS_CLASS`, `INGRESS_BODY_SIZE` | e.g. `nginx`, `"50m"` |
|
||||||
Reachable at **https://n8n.sttlab.local** once `192.168.139.101` (ingress-nginx)
|
| `TLS_ISSUER_NAME`, `TLS_ISSUER_KIND` | a cert-manager issuer, e.g. `ClusterIssuer` |
|
||||||
resolves for that name.
|
| `VAULT_SERVER`, `VAULT_MOUNT`, `VAULT_SECRET_PATH` | where the two secrets live |
|
||||||
|
| `VAULT_STORE_NAME`, `VAULT_APPROLE_SECRET` | the `ClusterSecretStore` this creates, and the AppRole Secret in `external-secrets` it authenticates with |
|
||||||
|
| `EXTRA_ENV_SECRET` | a Secret whose keys become environment variables for workflows. Optional in intent — supply an empty Secret if there is nothing |
|
||||||
|
| `EXTRA_CA_CONFIGMAP` | a ConfigMap with `ca.crt`, mounted and set as `NODE_EXTRA_CA_CERTS`, so workflows can call internal TLS endpoints without verification being disabled. Marked `optional`, so an absent one is not an error |
|
||||||
|
| `BLOCK_ENV_ACCESS` | `"true"` blocks `$env` in workflow expressions. n8n's default, and the right one wherever more than one team writes workflows: any workflow can otherwise read any variable in the pod. `"false"` is how a secret reaches a workflow without being committed in its JSON |
|
||||||
|
| `EXECUTIONS_MAX_AGE_HOURS` | how long executions are kept. n8n keeps them for ever by default, which turns the database into a log |
|
||||||
|
|
||||||
## Two secrets, both of which must outlive any pod
|
## Two secrets, both of which must outlive any pod
|
||||||
|
|
||||||
They live in Vault at `secrets/n8n/instance`, read through the `vault-n8n`
|
Read from `VAULT_SECRET_PATH` as a whole, so the entry must hold exactly:
|
||||||
store — an AppRole whose policy can read `n8n/*` and nothing else, so a
|
|
||||||
compromise here is not a way into the keycloak estate's secrets.
|
|
||||||
|
|
||||||
- `DB_PASSWORD`
|
- `DB_PASSWORD`
|
||||||
- `N8N_ENCRYPTION_KEY` — encrypts the credentials n8n stores in its database.
|
- `N8N_ENCRYPTION_KEY` — encrypts the credentials n8n stores in its database.
|
||||||
**Never regenerate it.** A new key does not re-encrypt what is stored; it
|
**Never regenerate it.** A new key does not re-encrypt what is stored; it
|
||||||
orphans it. The workflows survive, every credential in them does not.
|
orphans it. The workflows survive, every credential in them does not.
|
||||||
|
|
||||||
## Bootstrap
|
The AppRole's policy should reach `VAULT_SECRET_PATH` and nothing else, so a
|
||||||
|
workload here cannot read another deployment's secrets. The commands are in
|
||||||
Three manual applies, once. The Vault side is set up first — the commands are in
|
|
||||||
`k8s/secrets/clustersecretstore.yaml`.
|
`k8s/secrets/clustersecretstore.yaml`.
|
||||||
|
|
||||||
```
|
|
||||||
kubectl apply -f k8s/namespace.yaml
|
|
||||||
kubectl apply -f dev/gitrepository.yaml
|
|
||||||
kubectl apply -f dev/bootstrap-kustomization.yaml
|
|
||||||
```
|
|
||||||
|
|
||||||
After that everything reconciles from git, `dev/` included. Never `kubectl
|
|
||||||
apply` the objects under `k8s/` afterwards — Flux reverts them within its
|
|
||||||
interval and the cluster silently stops matching what you edited.
|
|
||||||
|
|
||||||
## Choices worth knowing
|
## Choices worth knowing
|
||||||
|
|
||||||
**One replica, and `strategy: Recreate`.** n8n splits editor from workers only
|
**One replica, and `strategy: Recreate`.** n8n separates editor from workers
|
||||||
in queue mode, which needs Redis; without it a second replica is two schedulers
|
only in queue mode, which needs Redis; without it a second replica is two
|
||||||
racing over the same cron triggers. And the data volume is ReadWriteOnce, so a
|
schedulers racing over the same cron triggers. And the data volume is
|
||||||
rolling update would deadlock — the new pod waiting for a volume the old one has
|
ReadWriteOnce, so a rolling update deadlocks — the new pod waiting for a volume
|
||||||
not released.
|
the old one has not released.
|
||||||
|
|
||||||
**The certificate is issued, not committed.** cert-manager writes `n8n-tls` from
|
**Postgres is a StatefulSet with a `volumeClaimTemplate`.** The claim is not
|
||||||
the `sttlab-local-ca` ClusterIssuer. The private key stays in the cluster.
|
deleted with the workload, so losing the database to a `kubectl delete` takes
|
||||||
|
more than one mistake.
|
||||||
|
|
||||||
**The CA certificate *is* committed**, in `k8s/ca-configmap.yaml`, and mounted
|
**The certificate is issued, never committed.** The private key is generated in
|
||||||
as `NODE_EXTRA_CA_CERTS`. It is a certificate, never a key, and checking it in
|
the cluster and rotated by cert-manager.
|
||||||
means a workflow can call `keycloak.sttlab.local` and the domain gateways
|
|
||||||
without anyone disabling verification to make it work. Regenerate on rotation —
|
|
||||||
it expires 2026-11-24.
|
|
||||||
|
|
||||||
**`N8N_HOST` and `WEBHOOK_URL` are the ingress host, not the Service.** They go
|
## What this repo deliberately does not do
|
||||||
into the webhook URLs n8n hands out; advertised as the in-cluster name, they
|
|
||||||
would be URLs nothing outside can call.
|
|
||||||
|
|
||||||
**Executions are pruned after two weeks.** The default keeps them for ever,
|
**Authentication.** n8n's own user management is left at its defaults. A fresh
|
||||||
which turns the database into a log.
|
instance is *unclaimed*: anyone who can reach it can create the owner account.
|
||||||
|
Claim it immediately, and put it behind an identity-aware proxy or n8n's SSO
|
||||||
|
before it holds anything that matters.
|
||||||
|
|
||||||
## What is deliberately not here
|
**Backups.** The Postgres claim survives a delete of the workload; it does not
|
||||||
|
survive the cluster.
|
||||||
Authentication. n8n's own user management is left at its defaults, and the
|
|
||||||
instance is reachable to anyone who can resolve the host on the lab network.
|
|
||||||
Before it holds anything that matters, it needs either n8n's SSO against the
|
|
||||||
`classic-models` realm or an `openid-connect` filter at the ingress — the same
|
|
||||||
Keycloak the rest of the estate already uses.
|
|
||||||
|
|||||||
@@ -1,30 +0,0 @@
|
|||||||
# Watches ./dev, so the GitRepository and the Kustomization below it reconcile
|
|
||||||
# from git like any other manifest. It lives in ./dev itself, so it also manages
|
|
||||||
# itself — the arrangement `flux bootstrap` sets up, which flux-operator does not
|
|
||||||
# create, and the same one the rest of the estate uses.
|
|
||||||
#
|
|
||||||
# Three manual applies, once, to close the loop — nothing can watch ./dev until
|
|
||||||
# the namespace and the source that serves it exist:
|
|
||||||
#
|
|
||||||
# kubectl apply -f k8s/namespace.yaml
|
|
||||||
# kubectl apply -f dev/gitrepository.yaml
|
|
||||||
# kubectl apply -f dev/bootstrap-kustomization.yaml
|
|
||||||
#
|
|
||||||
# After that every file in this repo, this one included, reconciles from git.
|
|
||||||
#
|
|
||||||
# Sharp edge: with prune enabled, deleting flux-kustomization.yaml from git
|
|
||||||
# deletes the Kustomization it defines, and that Kustomization's own prune then
|
|
||||||
# removes the deployment — including, on the PVCs, the workflows. The
|
|
||||||
# volumeClaimTemplate on Postgres survives that; n8n-data does not.
|
|
||||||
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
|
||||||
kind: Kustomization
|
|
||||||
metadata:
|
|
||||||
name: infra-n8n-bootstrap
|
|
||||||
namespace: n8n
|
|
||||||
spec:
|
|
||||||
interval: 10m
|
|
||||||
sourceRef:
|
|
||||||
kind: GitRepository
|
|
||||||
name: infra-n8n
|
|
||||||
path: ./dev
|
|
||||||
prune: true
|
|
||||||
@@ -1,14 +0,0 @@
|
|||||||
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
|
||||||
kind: Kustomization
|
|
||||||
metadata:
|
|
||||||
name: infra-n8n
|
|
||||||
namespace: n8n
|
|
||||||
spec:
|
|
||||||
interval: 10m
|
|
||||||
sourceRef:
|
|
||||||
kind: GitRepository
|
|
||||||
name: infra-n8n
|
|
||||||
path: ./k8s
|
|
||||||
prune: true
|
|
||||||
wait: true
|
|
||||||
timeout: 5m
|
|
||||||
@@ -1,14 +0,0 @@
|
|||||||
# HTTPS with no credentials: the repo is anonymously readable over HTTPS on
|
|
||||||
# gitea, so a deploy key would guard nothing while adding a Vault entry and a
|
|
||||||
# gitea key to create by hand before anything deploys. TLS still gives integrity
|
|
||||||
# and authenticity of the source.
|
|
||||||
apiVersion: source.toolkit.fluxcd.io/v1
|
|
||||||
kind: GitRepository
|
|
||||||
metadata:
|
|
||||||
name: infra-n8n
|
|
||||||
namespace: n8n
|
|
||||||
spec:
|
|
||||||
interval: 5m
|
|
||||||
url: https://gitea.sttlab.eu/stt/infra-n8n.git
|
|
||||||
ref:
|
|
||||||
branch: main
|
|
||||||
@@ -1,36 +0,0 @@
|
|||||||
# The lab CA, so n8n can verify keycloak.sttlab.local and the domain gateways
|
|
||||||
# rather than disabling verification. A CA certificate is public by
|
|
||||||
# construction — this is the certificate, never the key, and it is checked in so
|
|
||||||
# the deployment has no hidden prerequisite.
|
|
||||||
#
|
|
||||||
# Issued by the sttlab-local-ca ClusterIssuer, valid until 2026-11-24.
|
|
||||||
# Regenerate on rotation:
|
|
||||||
# kubectl -n cert-manager get secret sttlab-local-ca \
|
|
||||||
# -o jsonpath='{.data.tls\.crt}' | base64 -d
|
|
||||||
apiVersion: v1
|
|
||||||
kind: ConfigMap
|
|
||||||
metadata:
|
|
||||||
name: sttlab-ca
|
|
||||||
namespace: n8n
|
|
||||||
data:
|
|
||||||
ca.crt: |
|
|
||||||
-----BEGIN CERTIFICATE-----
|
|
||||||
MIIDPDCCAiSgAwIBAgIUd/utDf+f0yQR3v3qTjZqx+wvTZMwDQYJKoZIhvcNAQEL
|
|
||||||
BQAwGTEXMBUGA1UEAwwOKi5zdHRsYWIubG9jYWwwHhcNMjUxMTI0MTUzODQ1WhcN
|
|
||||||
MjYxMTI0MTUzODQ1WjAZMRcwFQYDVQQDDA4qLnN0dGxhYi5sb2NhbDCCASIwDQYJ
|
|
||||||
KoZIhvcNAQEBBQADggEPADCCAQoCggEBALj0sryq8qXtkShT/B+/EjVJNN/TLHIn
|
|
||||||
QJYc6xuUH/f7q3dYdLm4iH0XnFgrSQLUo4iqd/Mczef5iWCYwT5FFa7P17/TcLVf
|
|
||||||
gso0j41qNLBjGxDQGQ6Z1VCxi6oan8ow98rumjVj0E0Qgs5CX/rN+gcEmZ/TZFWU
|
|
||||||
qudVoztpeiVqJ0dxCIq+KMutzd1BU7BXx4VRO9Y+t1idUc7oY7BPmS2qPAN7TtXP
|
|
||||||
j1KhH2BgLsmF5GRhYHNS578eCltcmS2sbQ3c7GwOV96lpgF9TixEvFXeN4xk43E6
|
|
||||||
1COUD7CRiWAZ0XI8FMlpZGeNZeV4bbV/MqVig1ybWNOz9MDKYj3eoXUCAwEAAaN8
|
|
||||||
MHowHQYDVR0OBBYEFDbzYWOPr4vxa+VONs7xHbcNx70YMB8GA1UdIwQYMBaAFDbz
|
|
||||||
YWOPr4vxa+VONs7xHbcNx70YMA8GA1UdEwEB/wQFMAMBAf8wJwYDVR0RBCAwHoIO
|
|
||||||
Ki5zdHRsYWIubG9jYWyCDHN0dGxhYi5sb2NhbDANBgkqhkiG9w0BAQsFAAOCAQEA
|
|
||||||
tU4p5/ONXm7x4YDzSeL8NdThVLguiFur0jcTxvvPUS4NFNsnQJk4OXxBgjy1YwpL
|
|
||||||
3i56/4mCEd9OCQF/Hl0eFwGZBzNKlMoTA08AC/lBDvPrnVjuUu0qe/b/2KOkhESa
|
|
||||||
x8rQNp0o8pcj0f29nu0JfRqYGkzy0Lh2qgXhPu2iC+t2PfdleRAky1tt68wogdFL
|
|
||||||
TvHDsJbVo/ihsPpONlSyHfxG6bb/0Mkz64PH46TTt9GvJflEKMqDwivNFUA1P5S/
|
|
||||||
U4Ic8Qapnxc7h2+WbRjmYrnuygC2wMBE6rL9OF3gwCr1fkgGGRr5lVRUq2X2HZV5
|
|
||||||
5jqViv6eurq8O61N3vWGCQ==
|
|
||||||
-----END CERTIFICATE-----
|
|
||||||
@@ -1,17 +1,16 @@
|
|||||||
# cert-manager issues this into the `n8n-tls` Secret, from the same
|
# cert-manager issues this into the `n8n-tls` Secret.
|
||||||
# ClusterIssuer as the rest of the estate.
|
|
||||||
#
|
#
|
||||||
# Deliberately not a copy of a certificate committed to git: the private key
|
# Deliberately not a certificate copied into the repo: the private key is
|
||||||
# stays in the cluster, is rotated by cert-manager, and never reaches a repo.
|
# generated in the cluster, rotated by cert-manager, and never reaches git.
|
||||||
apiVersion: cert-manager.io/v1
|
apiVersion: cert-manager.io/v1
|
||||||
kind: Certificate
|
kind: Certificate
|
||||||
metadata:
|
metadata:
|
||||||
name: n8n-tls
|
name: n8n-tls
|
||||||
namespace: n8n
|
namespace: ${NAMESPACE}
|
||||||
spec:
|
spec:
|
||||||
secretName: n8n-tls
|
secretName: n8n-tls
|
||||||
dnsNames:
|
dnsNames:
|
||||||
- n8n.sttlab.local
|
- ${N8N_HOSTNAME}
|
||||||
issuerRef:
|
issuerRef:
|
||||||
name: sttlab-local-ca
|
name: ${TLS_ISSUER_NAME}
|
||||||
kind: ClusterIssuer
|
kind: ${TLS_ISSUER_KIND}
|
||||||
|
|||||||
@@ -2,24 +2,24 @@ apiVersion: networking.k8s.io/v1
|
|||||||
kind: Ingress
|
kind: Ingress
|
||||||
metadata:
|
metadata:
|
||||||
name: n8n
|
name: n8n
|
||||||
namespace: n8n
|
namespace: ${NAMESPACE}
|
||||||
annotations:
|
annotations:
|
||||||
# The editor pushes execution progress over a long-lived connection, and
|
# The editor pushes execution progress over a long-lived connection, and
|
||||||
# imports can be large. Buffering it defeats the first; the default body
|
# imports can be large. Buffering it defeats the first; the default body
|
||||||
# size defeats the second.
|
# size defeats the second.
|
||||||
nginx.ingress.kubernetes.io/proxy-body-size: "50m"
|
nginx.ingress.kubernetes.io/proxy-body-size: ${INGRESS_BODY_SIZE}
|
||||||
nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
|
nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
|
||||||
nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
|
nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
|
||||||
nginx.ingress.kubernetes.io/proxy-buffering: "off"
|
nginx.ingress.kubernetes.io/proxy-buffering: "off"
|
||||||
nginx.ingress.kubernetes.io/proxy-http-version: "1.1"
|
nginx.ingress.kubernetes.io/proxy-http-version: "1.1"
|
||||||
spec:
|
spec:
|
||||||
ingressClassName: nginx
|
ingressClassName: ${INGRESS_CLASS}
|
||||||
tls:
|
tls:
|
||||||
- hosts:
|
- hosts:
|
||||||
- n8n.sttlab.local
|
- ${N8N_HOSTNAME}
|
||||||
secretName: n8n-tls
|
secretName: n8n-tls
|
||||||
rules:
|
rules:
|
||||||
- host: n8n.sttlab.local
|
- host: ${N8N_HOSTNAME}
|
||||||
http:
|
http:
|
||||||
paths:
|
paths:
|
||||||
- path: /
|
- path: /
|
||||||
|
|||||||
@@ -2,10 +2,8 @@ apiVersion: kustomize.config.k8s.io/v1beta1
|
|||||||
kind: Kustomization
|
kind: Kustomization
|
||||||
resources:
|
resources:
|
||||||
- namespace.yaml
|
- namespace.yaml
|
||||||
- ca-configmap.yaml
|
|
||||||
- secrets/clustersecretstore.yaml
|
- secrets/clustersecretstore.yaml
|
||||||
- secrets/externalsecret.yaml
|
- secrets/externalsecret.yaml
|
||||||
- secrets/agent-externalsecret.yaml
|
|
||||||
- postgres/statefulset.yaml
|
- postgres/statefulset.yaml
|
||||||
- postgres/service.yaml
|
- postgres/service.yaml
|
||||||
- n8n/pvc.yaml
|
- n8n/pvc.yaml
|
||||||
|
|||||||
+12
-16
@@ -4,21 +4,21 @@ apiVersion: v1
|
|||||||
kind: ConfigMap
|
kind: ConfigMap
|
||||||
metadata:
|
metadata:
|
||||||
name: n8n-config
|
name: n8n-config
|
||||||
namespace: n8n
|
namespace: ${NAMESPACE}
|
||||||
data:
|
data:
|
||||||
GENERIC_TIMEZONE: Europe/Paris
|
GENERIC_TIMEZONE: ${TIMEZONE}
|
||||||
TZ: Europe/Paris
|
TZ: ${TIMEZONE}
|
||||||
NODE_ENV: production
|
NODE_ENV: production
|
||||||
|
|
||||||
# What n8n believes it is reachable at. It goes into the webhook URLs it
|
# What n8n believes it is reachable at. It goes into the webhook URLs it
|
||||||
# generates and into the links it puts in its own UI, so it must be the
|
# generates and into the links it puts in its own UI, so it must be the
|
||||||
# ingress host and not the Service — a webhook advertised as `n8n` inside the
|
# ingress host and not the Service — a webhook advertised as `n8n` inside the
|
||||||
# cluster is a webhook nothing outside can call.
|
# cluster is a webhook nothing outside can call.
|
||||||
N8N_HOST: n8n.sttlab.local
|
N8N_HOST: ${N8N_HOSTNAME}
|
||||||
N8N_PORT: "5678"
|
N8N_PORT: "5678"
|
||||||
N8N_PROTOCOL: https
|
N8N_PROTOCOL: https
|
||||||
WEBHOOK_URL: https://n8n.sttlab.local/
|
WEBHOOK_URL: https://${N8N_HOSTNAME}/
|
||||||
N8N_EDITOR_BASE_URL: https://n8n.sttlab.local/
|
N8N_EDITOR_BASE_URL: https://${N8N_HOSTNAME}/
|
||||||
|
|
||||||
# TLS terminates at the ingress, so n8n itself serves plain HTTP and must be
|
# TLS terminates at the ingress, so n8n itself serves plain HTTP and must be
|
||||||
# told it is behind a proxy — otherwise it reads the hop's address instead of
|
# told it is behind a proxy — otherwise it reads the hop's address instead of
|
||||||
@@ -28,18 +28,14 @@ data:
|
|||||||
# The default is to keep every execution for ever, which turns the database
|
# The default is to keep every execution for ever, which turns the database
|
||||||
# into a log. Two weeks is long enough to debug a failed run.
|
# into a log. Two weeks is long enough to debug a failed run.
|
||||||
EXECUTIONS_DATA_PRUNE: "true"
|
EXECUTIONS_DATA_PRUNE: "true"
|
||||||
EXECUTIONS_DATA_MAX_AGE: "336"
|
EXECUTIONS_DATA_MAX_AGE: ${EXECUTIONS_MAX_AGE_HOURS}
|
||||||
|
|
||||||
N8N_DIAGNOSTICS_ENABLED: "false"
|
N8N_DIAGNOSTICS_ENABLED: "false"
|
||||||
N8N_VERSION_NOTIFICATIONS_ENABLED: "false"
|
N8N_VERSION_NOTIFICATIONS_ENABLED: "false"
|
||||||
N8N_RUNNERS_ENABLED: "true"
|
N8N_RUNNERS_ENABLED: "true"
|
||||||
|
|
||||||
# Lets a workflow read `$env.<NAME>`. n8n blocks this by default, which is the
|
# "false" lets a workflow read `$env.<NAME>`, which is how a secret reaches a
|
||||||
# right default for an instance where anyone can write a workflow — here it is
|
# workflow without being stored in the workflow JSON. n8n blocks it by
|
||||||
# how an agent's client secret reaches the workflow that acts as that agent,
|
# default, and that default is right wherever more than one team can write a
|
||||||
# without the secret being stored in the workflow JSON and committed.
|
# workflow: any workflow can then read any variable in the pod.
|
||||||
#
|
N8N_BLOCK_ENV_ACCESS_IN_NODE: ${BLOCK_ENV_ACCESS}
|
||||||
# It does mean any workflow can read any variable in the pod. That is
|
|
||||||
# acceptable while the instance runs this estate's own agents and nothing
|
|
||||||
# else; it stops being acceptable the moment it is shared.
|
|
||||||
N8N_BLOCK_ENV_ACCESS_IN_NODE: "false"
|
|
||||||
|
|||||||
+19
-15
@@ -2,7 +2,7 @@ apiVersion: apps/v1
|
|||||||
kind: Deployment
|
kind: Deployment
|
||||||
metadata:
|
metadata:
|
||||||
name: n8n
|
name: n8n
|
||||||
namespace: n8n
|
namespace: ${NAMESPACE}
|
||||||
spec:
|
spec:
|
||||||
# One replica. n8n's queue mode splits the editor from workers and needs Redis
|
# One replica. n8n's queue mode splits the editor from workers and needs Redis
|
||||||
# to do it; without that, a second replica is two schedulers racing over the
|
# to do it; without that, a second replica is two schedulers racing over the
|
||||||
@@ -26,7 +26,7 @@ spec:
|
|||||||
fsGroup: 1000
|
fsGroup: 1000
|
||||||
initContainers:
|
initContainers:
|
||||||
- name: wait-for-postgres
|
- name: wait-for-postgres
|
||||||
image: postgres:16.10-alpine
|
image: ${POSTGRES_IMAGE}
|
||||||
command:
|
command:
|
||||||
- sh
|
- sh
|
||||||
- -c
|
- -c
|
||||||
@@ -36,7 +36,7 @@ spec:
|
|||||||
done
|
done
|
||||||
containers:
|
containers:
|
||||||
- name: n8n
|
- name: n8n
|
||||||
image: docker.n8n.io/n8nio/n8n:2.37.4
|
image: ${N8N_IMAGE}
|
||||||
ports:
|
ports:
|
||||||
- name: http
|
- name: http
|
||||||
containerPort: 5678
|
containerPort: 5678
|
||||||
@@ -64,24 +64,27 @@ spec:
|
|||||||
secretKeyRef:
|
secretKeyRef:
|
||||||
name: n8n-secrets
|
name: n8n-secrets
|
||||||
key: N8N_ENCRYPTION_KEY
|
key: N8N_ENCRYPTION_KEY
|
||||||
# The lab CA, so a workflow can call keycloak.sttlab.local and the
|
# A private CA to trust, so a workflow can call an internal
|
||||||
# domain gateways without disabling verification. n8n is Node, hence
|
# endpoint over TLS without anyone disabling verification to make it
|
||||||
# this variable rather than a trust store.
|
# work. n8n is Node, hence this variable rather than a trust store.
|
||||||
|
# Provided by the deployment; an empty ConfigMap disables it in
|
||||||
|
# practice, since the file will not exist and Node falls back to its
|
||||||
|
# own bundle.
|
||||||
- name: NODE_EXTRA_CA_CERTS
|
- name: NODE_EXTRA_CA_CERTS
|
||||||
value: /etc/sttlab-ca/ca.crt
|
value: /etc/extra-ca/ca.crt
|
||||||
envFrom:
|
envFrom:
|
||||||
- configMapRef:
|
- configMapRef:
|
||||||
name: n8n-config
|
name: n8n-config
|
||||||
# The agents' Keycloak client secrets, reached from a workflow as
|
# Whatever else this deployment's workflows need in their
|
||||||
# `$env.APPROVISIONNEMENT_CLIENT_SECRET`. A workflow acts as its
|
# environment. Provided by the deployment, not by this repo: an
|
||||||
# agent, so it is the workflow that must be able to exchange tokens.
|
# empty Secret of that name is a valid answer.
|
||||||
- secretRef:
|
- secretRef:
|
||||||
name: n8n-agent-secrets
|
name: ${EXTRA_ENV_SECRET}
|
||||||
volumeMounts:
|
volumeMounts:
|
||||||
- name: data
|
- name: data
|
||||||
mountPath: /home/node/.n8n
|
mountPath: /home/node/.n8n
|
||||||
- name: sttlab-ca
|
- name: extra-ca
|
||||||
mountPath: /etc/sttlab-ca
|
mountPath: /etc/extra-ca
|
||||||
readOnly: true
|
readOnly: true
|
||||||
readinessProbe:
|
readinessProbe:
|
||||||
httpGet:
|
httpGet:
|
||||||
@@ -110,6 +113,7 @@ spec:
|
|||||||
- name: data
|
- name: data
|
||||||
persistentVolumeClaim:
|
persistentVolumeClaim:
|
||||||
claimName: n8n-data
|
claimName: n8n-data
|
||||||
- name: sttlab-ca
|
- name: extra-ca
|
||||||
configMap:
|
configMap:
|
||||||
name: sttlab-ca
|
name: ${EXTRA_CA_CONFIGMAP}
|
||||||
|
optional: true
|
||||||
|
|||||||
+2
-2
@@ -5,9 +5,9 @@ apiVersion: v1
|
|||||||
kind: PersistentVolumeClaim
|
kind: PersistentVolumeClaim
|
||||||
metadata:
|
metadata:
|
||||||
name: n8n-data
|
name: n8n-data
|
||||||
namespace: n8n
|
namespace: ${NAMESPACE}
|
||||||
spec:
|
spec:
|
||||||
accessModes: ["ReadWriteOnce"]
|
accessModes: ["ReadWriteOnce"]
|
||||||
resources:
|
resources:
|
||||||
requests:
|
requests:
|
||||||
storage: 2Gi
|
storage: ${N8N_STORAGE}
|
||||||
|
|||||||
@@ -2,7 +2,7 @@ apiVersion: v1
|
|||||||
kind: Service
|
kind: Service
|
||||||
metadata:
|
metadata:
|
||||||
name: n8n
|
name: n8n
|
||||||
namespace: n8n
|
namespace: ${NAMESPACE}
|
||||||
spec:
|
spec:
|
||||||
selector:
|
selector:
|
||||||
app: n8n
|
app: n8n
|
||||||
|
|||||||
+1
-1
@@ -1,4 +1,4 @@
|
|||||||
apiVersion: v1
|
apiVersion: v1
|
||||||
kind: Namespace
|
kind: Namespace
|
||||||
metadata:
|
metadata:
|
||||||
name: n8n
|
name: ${NAMESPACE}
|
||||||
|
|||||||
@@ -2,7 +2,7 @@ apiVersion: v1
|
|||||||
kind: Service
|
kind: Service
|
||||||
metadata:
|
metadata:
|
||||||
name: n8n-postgres
|
name: n8n-postgres
|
||||||
namespace: n8n
|
namespace: ${NAMESPACE}
|
||||||
spec:
|
spec:
|
||||||
selector:
|
selector:
|
||||||
app: n8n-postgres
|
app: n8n-postgres
|
||||||
|
|||||||
@@ -6,7 +6,7 @@ apiVersion: apps/v1
|
|||||||
kind: StatefulSet
|
kind: StatefulSet
|
||||||
metadata:
|
metadata:
|
||||||
name: n8n-postgres
|
name: n8n-postgres
|
||||||
namespace: n8n
|
namespace: ${NAMESPACE}
|
||||||
spec:
|
spec:
|
||||||
serviceName: n8n-postgres
|
serviceName: n8n-postgres
|
||||||
replicas: 1
|
replicas: 1
|
||||||
@@ -22,7 +22,7 @@ spec:
|
|||||||
fsGroup: 999
|
fsGroup: 999
|
||||||
containers:
|
containers:
|
||||||
- name: postgres
|
- name: postgres
|
||||||
image: postgres:16.10-alpine
|
image: ${POSTGRES_IMAGE}
|
||||||
ports:
|
ports:
|
||||||
- name: postgres
|
- name: postgres
|
||||||
containerPort: 5432
|
containerPort: 5432
|
||||||
@@ -68,4 +68,4 @@ spec:
|
|||||||
accessModes: ["ReadWriteOnce"]
|
accessModes: ["ReadWriteOnce"]
|
||||||
resources:
|
resources:
|
||||||
requests:
|
requests:
|
||||||
storage: 5Gi
|
storage: ${POSTGRES_STORAGE}
|
||||||
|
|||||||
@@ -1,24 +0,0 @@
|
|||||||
# The agents' Keycloak client secrets, for the workflows that act as those
|
|
||||||
# agents. Read from the realm's own Vault entry through the keycloak store, not
|
|
||||||
# copied into `n8n/*`: there is one source for a client secret, and rotating it
|
|
||||||
# in Vault rotates it everywhere rather than leaving a stale twin behind.
|
|
||||||
#
|
|
||||||
# It is a narrow grant — this ExternalSecret names the properties it wants, so
|
|
||||||
# the n8n namespace receives those and nothing else of the realm.
|
|
||||||
apiVersion: external-secrets.io/v1
|
|
||||||
kind: ExternalSecret
|
|
||||||
metadata:
|
|
||||||
name: n8n-agent-secrets
|
|
||||||
namespace: n8n
|
|
||||||
spec:
|
|
||||||
refreshInterval: 1h
|
|
||||||
secretStoreRef:
|
|
||||||
kind: ClusterSecretStore
|
|
||||||
name: vault-keycloak
|
|
||||||
target:
|
|
||||||
name: n8n-agent-secrets
|
|
||||||
data:
|
|
||||||
- secretKey: APPROVISIONNEMENT_CLIENT_SECRET
|
|
||||||
remoteRef:
|
|
||||||
key: keycloak/classic-models-realm
|
|
||||||
property: KC_SECRET_CLASSIC_MODELS_AGENT_APPROVISIONNEMENT
|
|
||||||
@@ -1,37 +1,36 @@
|
|||||||
# One AppRole per estate, each with a policy that can read only its own path.
|
# Reads this deployment's secrets from Vault. Scope the AppRole's policy to the
|
||||||
# The n8n store cannot read `keycloak/*` and the keycloak store cannot read
|
# path this store reads and nothing more, so a workload here cannot reach
|
||||||
# `n8n/*` — so a compromised workload in one namespace does not become a way
|
# another estate's secrets.
|
||||||
# into the other's secrets.
|
|
||||||
#
|
#
|
||||||
# Its Vault side is created by hand, once:
|
# The Vault side is created once, outside this repo:
|
||||||
#
|
#
|
||||||
# vault policy write n8n-readonly - <<'HCL'
|
# vault policy write <policy> - <<'HCL'
|
||||||
# path "secrets/data/n8n/*" { capabilities = ["read"] }
|
# path "<mount>/data/<path>/*" { capabilities = ["read"] }
|
||||||
# path "secrets/metadata/n8n/*" { capabilities = ["list", "read"] }
|
# path "<mount>/metadata/<path>/*" { capabilities = ["list", "read"] }
|
||||||
# HCL
|
# HCL
|
||||||
# vault write auth/approle/role/n8n-eso token_policies=n8n-readonly \
|
# vault write auth/approle/role/<role> token_policies=<policy> \
|
||||||
# token_ttl=1h token_max_ttl=4h
|
# token_ttl=1h token_max_ttl=4h
|
||||||
# kubectl -n external-secrets create secret generic vault-approle-n8n \
|
# kubectl -n external-secrets create secret generic ${VAULT_APPROLE_SECRET} \
|
||||||
# --from-literal=roleId=$(vault read -field=role_id auth/approle/role/n8n-eso/role-id) \
|
# --from-literal=roleId=$(vault read -field=role_id auth/approle/role/<role>/role-id) \
|
||||||
# --from-literal=secretId=$(vault write -f -field=secret_id auth/approle/role/n8n-eso/secret-id)
|
# --from-literal=secretId=$(vault write -f -field=secret_id auth/approle/role/<role>/secret-id)
|
||||||
apiVersion: external-secrets.io/v1
|
apiVersion: external-secrets.io/v1
|
||||||
kind: ClusterSecretStore
|
kind: ClusterSecretStore
|
||||||
metadata:
|
metadata:
|
||||||
name: vault-n8n
|
name: ${VAULT_STORE_NAME}
|
||||||
spec:
|
spec:
|
||||||
provider:
|
provider:
|
||||||
vault:
|
vault:
|
||||||
server: https://vault.sttlab.eu
|
server: ${VAULT_SERVER}
|
||||||
path: secrets
|
path: ${VAULT_MOUNT}
|
||||||
version: v2
|
version: v2
|
||||||
auth:
|
auth:
|
||||||
appRole:
|
appRole:
|
||||||
path: approle
|
path: approle
|
||||||
roleRef:
|
roleRef:
|
||||||
name: vault-approle-n8n
|
name: ${VAULT_APPROLE_SECRET}
|
||||||
namespace: external-secrets
|
namespace: external-secrets
|
||||||
key: roleId
|
key: roleId
|
||||||
secretRef:
|
secretRef:
|
||||||
name: vault-approle-n8n
|
name: ${VAULT_APPROLE_SECRET}
|
||||||
namespace: external-secrets
|
namespace: external-secrets
|
||||||
key: secretId
|
key: secretId
|
||||||
|
|||||||
@@ -8,15 +8,15 @@ apiVersion: external-secrets.io/v1
|
|||||||
kind: ExternalSecret
|
kind: ExternalSecret
|
||||||
metadata:
|
metadata:
|
||||||
name: n8n-secrets
|
name: n8n-secrets
|
||||||
namespace: n8n
|
namespace: ${NAMESPACE}
|
||||||
spec:
|
spec:
|
||||||
refreshInterval: 1h
|
refreshInterval: 1h
|
||||||
secretStoreRef:
|
secretStoreRef:
|
||||||
kind: ClusterSecretStore
|
kind: ClusterSecretStore
|
||||||
name: vault-n8n
|
name: ${VAULT_STORE_NAME}
|
||||||
target:
|
target:
|
||||||
name: n8n-secrets
|
name: n8n-secrets
|
||||||
creationPolicy: Owner
|
creationPolicy: Owner
|
||||||
dataFrom:
|
dataFrom:
|
||||||
- extract:
|
- extract:
|
||||||
key: n8n/instance
|
key: ${VAULT_SECRET_PATH}
|
||||||
|
|||||||
Reference in New Issue
Block a user