2026-07-22 14:41:10 +00:00
2026-07-22 14:41:10 +00:00
2026-07-22 14:41:10 +00:00
2026-07-22 14:41:10 +00:00
2026-07-22 14:41:10 +00:00
2026-07-22 14:41:10 +00:00
2026-07-22 14:41:10 +00:00
2026-07-22 14:41:10 +00:00

Tech Watch Automation Stack — Miniflux + n8n + digest workflow

This covers the full stack described in AGENTS.md (namespace watch, shared PostgreSQL in namespace postgres, Miniflux with the initial feed bootstrap, n8n for workflow orchestration, and the daily LLM digest workflow) plus a small read-only file server for the generated digests. Everything below is deployed and active on this cluster.

Layout

k8s/
  namespace.yaml                # namespace: watch (apps)
  postgres/                     # shared PostgreSQL instance (namespace: postgres)
    namespace.yaml
    secret.example.yaml         # superuser credentials
    statefulset.yaml
    service.yaml
  miniflux/                     # namespace: watch (except create-db-job.yaml)
    secret.example.yaml         # DATABASE_URL, admin credentials
    deployment.yaml
    service.yaml
    gateway.yaml                 # Gateway API (Envoy), HTTPS listener on 443
    httproute.yaml                # routes miniflux.sttlab.pc to the Service
    create-db-job.yaml          # runs in ns postgres: creates the "miniflux" database
    bootstrap-feeds-job.yaml    # runs miniflux-initializer to create categories/feeds
  n8n/                         # namespace: watch (except create-db-job.yaml)
    secret.example.yaml         # DB credentials, N8N_ENCRYPTION_KEY
    miniflux-token-secret.example.yaml # Miniflux API token, backup/reference for the n8n credential
    pvc.yaml                     # n8n-data (n8n's own /home/node/.n8n) + n8n-digest-workspace (/data, for Read/Write File nodes)
    deployment.yaml
    service.yaml
    gateway.yaml                 # Gateway API (Envoy), HTTPS listener on 443
    httproute.yaml                # routes n8n.sttlab.pc to the Service
    create-db-job.yaml          # runs in ns postgres: creates the "n8n" database
  digest-files/                 # namespace: watch — read-only nginx file server
    configmap.yaml                # nginx.conf: autoindex on
    deployment.yaml                # mounts n8n-digest-workspace PVC (digests/ subdir), read-only
    service.yaml
    gateway.yaml                   # routes digests.sttlab.pc
    httproute.yaml
miniflux-initializer/           # container image used by bootstrap-feeds-job.yaml
  Dockerfile
  bootstrap_feeds.py
  feeds.yaml
  requirements.txt
prompts/
  digest.md                     # LLM system prompt for the digest workflow; mounted into n8n via
                                 # the n8n-digest-prompt ConfigMap (regenerate after editing, see below)
workflows/
  digest.json                   # "Tech Watch Digest" n8n workflow (daily 06:00 Europe/Paris cron)

Deploy

  1. Namespaces

    kubectl apply -f k8s/namespace.yaml
    kubectl apply -f k8s/postgres/namespace.yaml
    
  2. PostgreSQLk8s/postgres/{namespace,secret.example,statefulset,service}.yaml deploy a standalone instance and are kept for reference/portability, but on this cluster a shared PostgreSQL already runs in the postgres namespace (Bitnami Helm chart, release postgres, service postgres-postgresql). Reuse it instead of applying the StatefulSet:

    cp k8s/postgres/secret.example.yaml k8s/postgres/secret.yaml
    # edit k8s/postgres/secret.yaml: POSTGRES_PASSWORD must match the real
    # instance's password:
    #   kubectl get secret postgres-postgresql -n postgres -o jsonpath='{.data.postgres-password}' | base64 -d
    kubectl apply -f k8s/postgres/secret.yaml
    kubectl apply -f k8s/miniflux/create-db-job.yaml
    kubectl wait --for=condition=complete job/create-db-miniflux -n postgres --timeout=60s
    

    (If deploying to a cluster with no existing shared Postgres, apply statefulset.yaml/service.yaml first and update create-db-job.yaml's host from postgres-postgresql to postgres.)

  3. TLS secret for the Gateway listeners (using the existing wildcard cert for *.sttlab.pc)

    kubectl create secret tls sttlab-pc-tls \
      --cert=~/tls/sttlab.pc.crt --key=~/tls/sttlab.pc.key \
      -n watch
    
  4. Miniflux

    cp k8s/miniflux/secret.example.yaml k8s/miniflux/secret.yaml
    # edit k8s/miniflux/secret.yaml:
    #   - DATABASE_URL password must match k8s/postgres/secret.yaml
    #   - set ADMIN_USERNAME / ADMIN_PASSWORD for the Miniflux admin account
    kubectl apply -f k8s/miniflux/secret.yaml
    kubectl apply -f k8s/miniflux/deployment.yaml
    kubectl apply -f k8s/miniflux/service.yaml
    kubectl apply -f k8s/miniflux/gateway.yaml
    kubectl apply -f k8s/miniflux/httproute.yaml
    

    Exposed via Gateway API (Envoy Gateway, gatewayClassName: envoy) on port 443 — this cluster's ingress-nginx only exposes ports 80/9443 externally, not 443, so a plain Ingress would not actually be reachable over HTTPS. Point DNS for miniflux.sttlab.pc at the Gateway address if not already done (kubectl get gateway miniflux -n watch).

    Feed crawler (fetch original article content instead of the raw RSS/Atom excerpt) is enabled on all feeds except OpenAI's — its blog is behind a Cloudflare bot challenge that always rejects Miniflux's fetcher, so it stays on the RSS excerpt. See crawler: per feed in miniflux-initializer/feeds.yaml.

  5. Bootstrap feeds — runs as a one-off Job using a locally built image (no external registry needed on a single-node k3s cluster):

    cd miniflux-initializer
    docker build --network=host -t miniflux-initializer:latest .
    docker save miniflux-initializer:latest | sudo k3s ctr images import -
    cd ..
    kubectl apply -f k8s/miniflux/bootstrap-feeds-job.yaml
    kubectl wait --for=condition=complete job/bootstrap-feeds -n watch --timeout=60s
    kubectl logs -n watch job/bootstrap-feeds
    

    Safe to re-run: delete the Job (kubectl delete job bootstrap-feeds -n watch) and re-apply after editing miniflux-initializer/feeds.yaml (rebuild + re-import the image first if you changed it). Only adds new feeds — it does not update crawler/settings on feeds that already exist (use the Miniflux API or UI for that).

    Note: --network=host is required for the pip install step during the image build — the local router does not answer DNS queries from Docker's bridge subnet, only the host's own resolver works.

  6. API token for n8n: create it manually in the Miniflux UI under Settings → API Keys — there is no public REST endpoint to create one. Kept as the source of truth in k8s/n8n/miniflux-token-secret.yaml (see .example for the format); the n8n credential itself is created in step 8 below, from the same token.

  7. n8n

    cp k8s/n8n/secret.example.yaml k8s/n8n/secret.yaml
    # edit k8s/n8n/secret.yaml:
    #   - DB_POSTGRESDB_PASSWORD must match k8s/postgres/secret.yaml
    #   - N8N_ENCRYPTION_KEY: generate once with `openssl rand -hex 24` and
    #     never rotate it afterwards (it decrypts every saved credential)
    kubectl apply -f k8s/n8n/secret.yaml
    kubectl apply -f k8s/n8n/create-db-job.yaml
    kubectl wait --for=condition=complete job/create-db-n8n -n postgres --timeout=60s
    kubectl apply -f k8s/n8n/pvc.yaml
    # ConfigMap holding the digest LLM prompt (prompts/digest.md is the source
    # of truth; regenerate and re-apply this any time the file changes):
    kubectl create configmap n8n-digest-prompt --from-file=digest.md=prompts/digest.md \
      -n watch --dry-run=client -o yaml | kubectl apply -f -
    kubectl apply -f k8s/n8n/deployment.yaml
    kubectl apply -f k8s/n8n/service.yaml
    kubectl apply -f k8s/n8n/gateway.yaml
    kubectl apply -f k8s/n8n/httproute.yaml
    

    Same Gateway API setup as Miniflux (TLS terminated by Envoy Gateway on 443, reusing the sttlab-pc-tls secret). Point DNS for n8n.sttlab.pc at the Gateway address once (kubectl get gateway n8n -n watch).

    On first login, n8n prompts to create the owner account (email/password) — there's no CREATE_ADMIN-style env var like Miniflux, so this step is manual in the UI.

    Two PVCs are mounted: n8n-data is n8n's own /home/node/.n8n (config, binary data cache); n8n-digest-workspace is mounted separately at /data for the digest workflow's Read/Write File nodes (prompt + generated digests). They're kept apart because n8n unconditionally blocks file-node access to its own /home/node/.n8n directory (N8N_BLOCK_FILE_ACCESS_TO_N8N_FILES, not overridable via N8N_RESTRICT_FILE_ACCESS_TO) — a dedicated volume avoids that restriction honestly instead of working around it. Note also that N8N_RESTRICT_FILE_ACCESS_TO only accepts a single path, not a list — hence both prompts/ and digests/ live under the same /data root.

  8. Digest workflow — import workflows/digest.json (the "Tech Watch Digest" workflow: daily 06:00 Europe/Paris cron) and wire its credentials via the n8n Public API (Settings → n8n API → create a key first):

    N8N_KEY=<your n8n API key>
    
    # Miniflux credential (Header Auth, X-Auth-Token) — use the same token
    # as k8s/n8n/miniflux-token-secret.yaml
    curl -s https://n8n.sttlab.pc/api/v1/credentials -X POST \
      -H "X-N8N-API-KEY: $N8N_KEY" -H "Content-Type: application/json" \
      -d '{"name":"Miniflux API","type":"httpHeaderAuth","data":{"name":"X-Auth-Token","value":"<token>","allowedHttpRequestDomains":"all"}}'
    
    # LLM credential (Header Auth, Authorization: Bearer <key>) — OpenRouter or
    # any OpenAI-compatible provider
    curl -s https://n8n.sttlab.pc/api/v1/credentials -X POST \
      -H "X-N8N-API-KEY: $N8N_KEY" -H "Content-Type: application/json" \
      -d '{"name":"LLM API","type":"httpHeaderAuth","data":{"name":"Authorization","value":"Bearer <key>","allowedHttpRequestDomains":"all"}}'
    
    # Import the workflow, then edit the two credential IDs in its HTTP
    # Request nodes ("Get Unread Entries" / "Mark Entries As Read" -> Miniflux
    # API; "Summarize Category" -> LLM API) to match the IDs returned above,
    # and set the real llmBaseUrl / llmModel in the "Init" node's code.
    curl -s https://n8n.sttlab.pc/api/v1/workflows -X POST \
      -H "X-N8N-API-KEY: $N8N_KEY" -H "Content-Type: application/json" \
      --data-binary @<(python3 -c "import json; wf=json.load(open('workflows/digest.json')); print(json.dumps({k: wf[k] for k in ('name','nodes','connections','settings')}))")
    
    # Activate once credentials are wired and a manual run succeeds:
    curl -s https://n8n.sttlab.pc/api/v1/workflows/<id>/activate -X POST \
      -H "X-N8N-API-KEY: $N8N_KEY"
    

    The workflow uses Miniflux's internal cluster Service DNS (http://miniflux.watch.svc.cluster.local/v1/entries), not the external HTTPS hostname — miniflux.sttlab.pc isn't resolvable from inside the cluster (it only exists as a Gateway route, no DNS record).

    To test-run before activating (the running Deployment pod can't run n8n execute itself — its Task Broker port is already bound — so use a throwaway Job with the same image/env/volumes instead), or just use the "Test workflow" button in the n8n UI from the Schedule Trigger node.

    Pipeline: fetch unread Miniflux entries from the last 24h → group by category → for each category, call the LLM (system prompt from prompts/digest.md) to pick the 3-5 most relevant items + weak signals, strict JSON output → compose one Markdown file → write it to /data/digests/digest-<date>.md → mark the processed entries as read in Miniflux.

  9. Digest file server — serves /data/digests/ (read-only) so digests can be browsed/downloaded from a browser instead of email:

    kubectl apply -f k8s/digest-files/
    

    Same Gateway/TLS pattern, routes digests.sttlab.pc. Point DNS at the Gateway address (kubectl get gateway digest-files -n watch).

How to add a source

Add an entry under the relevant category (or a new category) in miniflux-initializer/feeds.yaml, verify the URL responds with curl -I <url>, then rebuild the image and re-run the bootstrap Job (see step 5).

Validate manifests

kubectl apply --dry-run=client -f k8s/namespace.yaml
kubectl apply --dry-run=client -f k8s/postgres/
kubectl apply --dry-run=client -f k8s/miniflux/
kubectl apply --dry-run=client -f k8s/n8n/
kubectl apply --dry-run=client -f k8s/digest-files/

(Run against a cluster with the watch/postgres namespaces and secrets already created, since some manifests reference them.)

Smoke-test checklist

  • kubectl get pods -n postgres — postgres pod Running, 1/1 ready
  • kubectl get pods -n watch — miniflux pod Running, 1/1 ready
  • kubectl logs -n watch deploy/miniflux shows migrations applied, no errors
  • https://miniflux.sttlab.pc/healthcheck returns OK
  • Log in to the Miniflux UI with the admin credentials from the secret
  • kubectl logs -n watch job/bootstrap-feeds shows categories/feeds created
  • Categories and feeds appear in the Miniflux UI
  • At least one feed shows fetched entries (Miniflux polls periodically; use "Refresh" in the UI to force it)
  • GET /v1/entries?status=unread (with an API token) returns entries
  • kubectl get pods -n watch — n8n pod Running, 1/1 ready
  • kubectl logs -n watch deploy/n8n shows no DB connection errors
  • https://n8n.sttlab.pc/healthz returns {"status":"ok"}
  • Log in to the n8n UI and create the owner account
  • n8n can reach Miniflux: an HTTP Request node to http://miniflux.watch.svc.cluster.local/v1/me with the API token returns the admin user
  • "Tech Watch Digest" workflow imported, both credentials (Miniflux API, LLM API) attached to their HTTP Request nodes
  • A manual run ("Test workflow" in the UI, or a throwaway n8n execute --id=<id> Job) produces a file under /data/digests/ in the n8n pod, and the corresponding Miniflux entries are marked read
  • Workflow activated ("active": true)
  • kubectl get pods -n watch — digest-files pod Running, 1/1 ready
  • https://digests.sttlab.pc/ lists the generated digest files

Current status

Deployed and active: Miniflux (9 feeds), n8n, "Tech Watch Digest" workflow (daily 06:00 Europe/Paris, OpenRouter deepseek/deepseek-v4-flash), and the digest file server at https://digests.sttlab.pc/. No email delivery (SMTP not configured) — digests are Markdown files served over HTTP instead.

S
Description
No description provided
Readme 57 KiB
Languages
Python 92.1%
Dockerfile 7.9%