# Tech Watch Automation Stack — Miniflux + n8n + digest workflow This covers the full stack described in `AGENTS.md` (namespace `watch`, shared PostgreSQL in namespace `postgres`, Miniflux with the initial feed bootstrap, n8n for workflow orchestration, and the daily LLM digest workflow) plus a small read-only file server for the generated digests. Everything below is deployed and active on this cluster. ## Layout ``` k8s/ namespace.yaml # namespace: watch (apps) postgres/ # shared PostgreSQL instance (namespace: postgres) namespace.yaml secret.example.yaml # superuser credentials statefulset.yaml service.yaml miniflux/ # namespace: watch (except create-db-job.yaml) secret.example.yaml # DATABASE_URL, admin credentials deployment.yaml service.yaml gateway.yaml # Gateway API (Envoy), HTTPS listener on 443 httproute.yaml # routes miniflux.sttlab.pc to the Service create-db-job.yaml # runs in ns postgres: creates the "miniflux" database bootstrap-feeds-job.yaml # runs miniflux-initializer to create categories/feeds n8n/ # namespace: watch (except create-db-job.yaml) secret.example.yaml # DB credentials, N8N_ENCRYPTION_KEY miniflux-token-secret.example.yaml # Miniflux API token, backup/reference for the n8n credential pvc.yaml # n8n-data (n8n's own /home/node/.n8n) + n8n-digest-workspace (/data, for Read/Write File nodes) deployment.yaml service.yaml gateway.yaml # Gateway API (Envoy), HTTPS listener on 443 httproute.yaml # routes n8n.sttlab.pc to the Service create-db-job.yaml # runs in ns postgres: creates the "n8n" database digest-files/ # namespace: watch — read-only nginx file server configmap.yaml # nginx.conf: autoindex on deployment.yaml # mounts n8n-digest-workspace PVC (digests/ subdir), read-only service.yaml gateway.yaml # routes digests.sttlab.pc httproute.yaml miniflux-initializer/ # container image used by bootstrap-feeds-job.yaml Dockerfile bootstrap_feeds.py feeds.yaml requirements.txt prompts/ digest.md # LLM system prompt for the digest workflow; mounted into n8n via # the n8n-digest-prompt ConfigMap (regenerate after editing, see below) workflows/ digest.json # "Tech Watch Digest" n8n workflow (daily 06:00 Europe/Paris cron) ``` ## Deploy 1. **Namespaces** ``` kubectl apply -f k8s/namespace.yaml kubectl apply -f k8s/postgres/namespace.yaml ``` 2. **PostgreSQL** — `k8s/postgres/{namespace,secret.example,statefulset,service}.yaml` deploy a standalone instance and are kept for reference/portability, but **on this cluster a shared PostgreSQL already runs** in the `postgres` namespace (Bitnami Helm chart, release `postgres`, service `postgres-postgresql`). Reuse it instead of applying the StatefulSet: ``` cp k8s/postgres/secret.example.yaml k8s/postgres/secret.yaml # edit k8s/postgres/secret.yaml: POSTGRES_PASSWORD must match the real # instance's password: # kubectl get secret postgres-postgresql -n postgres -o jsonpath='{.data.postgres-password}' | base64 -d kubectl apply -f k8s/postgres/secret.yaml kubectl apply -f k8s/miniflux/create-db-job.yaml kubectl wait --for=condition=complete job/create-db-miniflux -n postgres --timeout=60s ``` (If deploying to a cluster with no existing shared Postgres, apply `statefulset.yaml`/`service.yaml` first and update `create-db-job.yaml`'s host from `postgres-postgresql` to `postgres`.) 3. **TLS secret** for the Gateway listeners (using the existing wildcard cert for `*.sttlab.pc`) ``` kubectl create secret tls sttlab-pc-tls \ --cert=~/tls/sttlab.pc.crt --key=~/tls/sttlab.pc.key \ -n watch ``` 4. **Miniflux** ``` cp k8s/miniflux/secret.example.yaml k8s/miniflux/secret.yaml # edit k8s/miniflux/secret.yaml: # - DATABASE_URL password must match k8s/postgres/secret.yaml # - set ADMIN_USERNAME / ADMIN_PASSWORD for the Miniflux admin account kubectl apply -f k8s/miniflux/secret.yaml kubectl apply -f k8s/miniflux/deployment.yaml kubectl apply -f k8s/miniflux/service.yaml kubectl apply -f k8s/miniflux/gateway.yaml kubectl apply -f k8s/miniflux/httproute.yaml ``` Exposed via Gateway API (Envoy Gateway, `gatewayClassName: envoy`) on port 443 — this cluster's `ingress-nginx` only exposes ports 80/9443 externally, not 443, so a plain `Ingress` would not actually be reachable over HTTPS. Point DNS for `miniflux.sttlab.pc` at the Gateway address if not already done (`kubectl get gateway miniflux -n watch`). Feed `crawler` (fetch original article content instead of the raw RSS/Atom excerpt) is enabled on all feeds except OpenAI's — its blog is behind a Cloudflare bot challenge that always rejects Miniflux's fetcher, so it stays on the RSS excerpt. See `crawler:` per feed in `miniflux-initializer/feeds.yaml`. 5. **Bootstrap feeds** — runs as a one-off Job using a locally built image (no external registry needed on a single-node k3s cluster): ``` cd miniflux-initializer docker build --network=host -t miniflux-initializer:latest . docker save miniflux-initializer:latest | sudo k3s ctr images import - cd .. kubectl apply -f k8s/miniflux/bootstrap-feeds-job.yaml kubectl wait --for=condition=complete job/bootstrap-feeds -n watch --timeout=60s kubectl logs -n watch job/bootstrap-feeds ``` Safe to re-run: delete the Job (`kubectl delete job bootstrap-feeds -n watch`) and re-apply after editing `miniflux-initializer/feeds.yaml` (rebuild + re-import the image first if you changed it). Only adds new feeds — it does not update `crawler`/settings on feeds that already exist (use the Miniflux API or UI for that). > Note: `--network=host` is required for the `pip install` step during > the image build — the local router does not answer DNS queries from > Docker's bridge subnet, only the host's own resolver works. 6. **API token for n8n**: create it manually in the Miniflux UI under *Settings → API Keys* — there is no public REST endpoint to create one. Kept as the source of truth in `k8s/n8n/miniflux-token-secret.yaml` (see `.example` for the format); the n8n credential itself is created in step 8 below, from the same token. 7. **n8n** ``` cp k8s/n8n/secret.example.yaml k8s/n8n/secret.yaml # edit k8s/n8n/secret.yaml: # - DB_POSTGRESDB_PASSWORD must match k8s/postgres/secret.yaml # - N8N_ENCRYPTION_KEY: generate once with `openssl rand -hex 24` and # never rotate it afterwards (it decrypts every saved credential) kubectl apply -f k8s/n8n/secret.yaml kubectl apply -f k8s/n8n/create-db-job.yaml kubectl wait --for=condition=complete job/create-db-n8n -n postgres --timeout=60s kubectl apply -f k8s/n8n/pvc.yaml # ConfigMap holding the digest LLM prompt (prompts/digest.md is the source # of truth; regenerate and re-apply this any time the file changes): kubectl create configmap n8n-digest-prompt --from-file=digest.md=prompts/digest.md \ -n watch --dry-run=client -o yaml | kubectl apply -f - kubectl apply -f k8s/n8n/deployment.yaml kubectl apply -f k8s/n8n/service.yaml kubectl apply -f k8s/n8n/gateway.yaml kubectl apply -f k8s/n8n/httproute.yaml ``` Same Gateway API setup as Miniflux (TLS terminated by Envoy Gateway on 443, reusing the `sttlab-pc-tls` secret). Point DNS for `n8n.sttlab.pc` at the Gateway address once (`kubectl get gateway n8n -n watch`). On first login, n8n prompts to create the owner account (email/password) — there's no `CREATE_ADMIN`-style env var like Miniflux, so this step is manual in the UI. Two PVCs are mounted: `n8n-data` is n8n's own `/home/node/.n8n` (config, binary data cache); `n8n-digest-workspace` is mounted separately at `/data` for the digest workflow's Read/Write File nodes (prompt + generated digests). They're kept apart because n8n unconditionally blocks file-node access to its own `/home/node/.n8n` directory (`N8N_BLOCK_FILE_ACCESS_TO_N8N_FILES`, not overridable via `N8N_RESTRICT_FILE_ACCESS_TO`) — a dedicated volume avoids that restriction honestly instead of working around it. Note also that `N8N_RESTRICT_FILE_ACCESS_TO` only accepts a single path, not a list — hence both `prompts/` and `digests/` live under the same `/data` root. 8. **Digest workflow** — import `workflows/digest.json` (the "Tech Watch Digest" workflow: daily 06:00 Europe/Paris cron) and wire its credentials via the n8n Public API (Settings → n8n API → create a key first): ``` N8N_KEY= # Miniflux credential (Header Auth, X-Auth-Token) — use the same token # as k8s/n8n/miniflux-token-secret.yaml curl -s https://n8n.sttlab.pc/api/v1/credentials -X POST \ -H "X-N8N-API-KEY: $N8N_KEY" -H "Content-Type: application/json" \ -d '{"name":"Miniflux API","type":"httpHeaderAuth","data":{"name":"X-Auth-Token","value":"","allowedHttpRequestDomains":"all"}}' # LLM credential (Header Auth, Authorization: Bearer ) — OpenRouter or # any OpenAI-compatible provider curl -s https://n8n.sttlab.pc/api/v1/credentials -X POST \ -H "X-N8N-API-KEY: $N8N_KEY" -H "Content-Type: application/json" \ -d '{"name":"LLM API","type":"httpHeaderAuth","data":{"name":"Authorization","value":"Bearer ","allowedHttpRequestDomains":"all"}}' # Import the workflow, then edit the two credential IDs in its HTTP # Request nodes ("Get Unread Entries" / "Mark Entries As Read" -> Miniflux # API; "Summarize Category" -> LLM API) to match the IDs returned above, # and set the real llmBaseUrl / llmModel in the "Init" node's code. curl -s https://n8n.sttlab.pc/api/v1/workflows -X POST \ -H "X-N8N-API-KEY: $N8N_KEY" -H "Content-Type: application/json" \ --data-binary @<(python3 -c "import json; wf=json.load(open('workflows/digest.json')); print(json.dumps({k: wf[k] for k in ('name','nodes','connections','settings')}))") # Activate once credentials are wired and a manual run succeeds: curl -s https://n8n.sttlab.pc/api/v1/workflows//activate -X POST \ -H "X-N8N-API-KEY: $N8N_KEY" ``` The workflow uses Miniflux's **internal** cluster Service DNS (`http://miniflux.watch.svc.cluster.local/v1/entries`), not the external HTTPS hostname — `miniflux.sttlab.pc` isn't resolvable from inside the cluster (it only exists as a Gateway route, no DNS record). To test-run before activating (the running Deployment pod can't run `n8n execute` itself — its Task Broker port is already bound — so use a throwaway Job with the same image/env/volumes instead), or just use the "Test workflow" button in the n8n UI from the Schedule Trigger node. Pipeline: fetch unread Miniflux entries from the last 24h → group by category → for each category, call the LLM (system prompt from `prompts/digest.md`) to pick the 3-5 most relevant items + weak signals, strict JSON output → compose one Markdown file → write it to `/data/digests/digest-.md` → mark the processed entries as read in Miniflux. 9. **Digest file server** — serves `/data/digests/` (read-only) so digests can be browsed/downloaded from a browser instead of email: ``` kubectl apply -f k8s/digest-files/ ``` Same Gateway/TLS pattern, routes `digests.sttlab.pc`. Point DNS at the Gateway address (`kubectl get gateway digest-files -n watch`). ## How to add a source Add an entry under the relevant category (or a new category) in `miniflux-initializer/feeds.yaml`, verify the URL responds with `curl -I `, then rebuild the image and re-run the bootstrap Job (see step 5). ## Validate manifests ``` kubectl apply --dry-run=client -f k8s/namespace.yaml kubectl apply --dry-run=client -f k8s/postgres/ kubectl apply --dry-run=client -f k8s/miniflux/ kubectl apply --dry-run=client -f k8s/n8n/ kubectl apply --dry-run=client -f k8s/digest-files/ ``` (Run against a cluster with the `watch`/`postgres` namespaces and secrets already created, since some manifests reference them.) ## Smoke-test checklist - [ ] `kubectl get pods -n postgres` — postgres pod `Running`, `1/1` ready - [ ] `kubectl get pods -n watch` — miniflux pod `Running`, `1/1` ready - [ ] `kubectl logs -n watch deploy/miniflux` shows migrations applied, no errors - [ ] `https://miniflux.sttlab.pc/healthcheck` returns `OK` - [ ] Log in to the Miniflux UI with the admin credentials from the secret - [ ] `kubectl logs -n watch job/bootstrap-feeds` shows categories/feeds created - [ ] Categories and feeds appear in the Miniflux UI - [ ] At least one feed shows fetched entries (Miniflux polls periodically; use "Refresh" in the UI to force it) - [ ] `GET /v1/entries?status=unread` (with an API token) returns entries - [ ] `kubectl get pods -n watch` — n8n pod `Running`, `1/1` ready - [ ] `kubectl logs -n watch deploy/n8n` shows no DB connection errors - [ ] `https://n8n.sttlab.pc/healthz` returns `{"status":"ok"}` - [ ] Log in to the n8n UI and create the owner account - [ ] n8n can reach Miniflux: an HTTP Request node to `http://miniflux.watch.svc.cluster.local/v1/me` with the API token returns the admin user - [ ] "Tech Watch Digest" workflow imported, both credentials (Miniflux API, LLM API) attached to their HTTP Request nodes - [ ] A manual run ("Test workflow" in the UI, or a throwaway `n8n execute --id=` Job) produces a file under `/data/digests/` in the n8n pod, and the corresponding Miniflux entries are marked read - [ ] Workflow activated (`"active": true`) - [ ] `kubectl get pods -n watch` — digest-files pod `Running`, `1/1` ready - [ ] `https://digests.sttlab.pc/` lists the generated digest files ## Current status Deployed and active: Miniflux (9 feeds), n8n, "Tech Watch Digest" workflow (daily 06:00 Europe/Paris, OpenRouter `deepseek/deepseek-v4-flash`), and the digest file server at `https://digests.sttlab.pc/`. No email delivery (SMTP not configured) — digests are Markdown files served over HTTP instead.