From e6699912a540c3330b21b00323b15832cfeb2299 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?St=C3=A9phane=20Tailland?= Date: Mon, 31 Aug 2026 08:43:14 +0200 Subject: [PATCH] Deploy a cluster, not an estate MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The repo had accreted one estate's accounts, topics and prose. An application's account and its topics belong to the application, whose own repo says what it may read and write — so that adding or removing the application adds or removes its access. What is left is the cluster, the admin and smoke-test accounts, the cross-namespace credentials store, and the pattern to copy. Co-Authored-By: Claude Opus 5 --- Makefile | 4 +- README.md | 53 ++++++++++++++------------- kafka-topics.yaml | 56 ---------------------------- kafka-users.yaml | 93 +++++++++++++++++------------------------------ 4 files changed, 62 insertions(+), 144 deletions(-) delete mode 100644 kafka-topics.yaml diff --git a/Makefile b/Makefile index ebe60d5..e07ac05 100644 --- a/Makefile +++ b/Makefile @@ -81,8 +81,7 @@ deploy: @for user in kafka-admin kafka-client; do \ kubectl wait kafkauser/$$user --for=condition=Ready --timeout=120s -n $(NAMESPACE); \ done - @echo "==> [6/6] Deploying topics and the credentials store" - @kubectl apply -f $(SCRIPT_DIR)kafka-topics.yaml + @echo "==> [6/6] Deploying the credentials store" @kubectl apply -f $(SCRIPT_DIR)credentials-access.yaml @echo "" @echo "Cluster ready. Bootstrap: $(BOOTSTRAP)" @@ -116,7 +115,6 @@ upgrade: teardown: @echo "==> Deleting KafkaUsers and Kafka cluster" @kubectl delete -f $(SCRIPT_DIR)credentials-access.yaml --ignore-not-found - @kubectl delete -f $(SCRIPT_DIR)kafka-topics.yaml --ignore-not-found @kubectl delete -f $(SCRIPT_DIR)kafka-users.yaml --ignore-not-found @kubectl delete -f $(SCRIPT_DIR)kafka.yaml --ignore-not-found @echo "==> Waiting for pods to terminate" diff --git a/README.md b/README.md index 96b44d6..b3ac8a3 100644 --- a/README.md +++ b/README.md @@ -40,18 +40,19 @@ kafka-kafka-bootstrap.kafka.svc.cluster.local:9093 |---|---|---| | `kafka-admin` | `kafka-admin` | Super user — unrestricted access | | `kafka-client` | `kafka-client` | Application user — Read/Write all topics and groups | -| `classic-models-api` | `classic-models-api` | Producer for `classic-models.*` — **Write and Describe only**, no Read | -| `classic-models-agent-approvisionnement` | same name | Consumer of `classic-models.catalog` and group prefix `approvisionnement-` — **Read and Describe only**, no Write | -Each application account is scoped to what it actually does. The producer -publishes domain events and consumes nothing, so it is write-only: one that -could also read every topic exfiltrates every other domain's events if it is -ever compromised. The consumer is the mirror — it cannot see sales or payments, -and it cannot write, so a compromised workflow can raise no event that another -agent would then act on. +These two are all this repo declares. **An application's account is not added +here** — it belongs to the application, whose own repo says what it may read and +write, so that adding or removing the application adds or removes its access. +This repo deploys a cluster and holds no opinion about who uses it. -Duplicate one of these per application. `kafka-client` is the wide-open -template, useful for a smoke test and for nothing that runs unattended. +`kafka-users.yaml` carries the shape to copy. Grant **one direction, on one +prefix**: a producer that can also read every topic exfiltrates every other +application's data if it is ever compromised, and a consumer that can write can +raise an event someone else acts on. + +`kafka-client` is the wide-open template — useful for a smoke test, and for +nothing that runs unattended. Strimzi stores credentials in Kubernetes secrets. Each secret contains two keys: - `password` — the SCRAM password @@ -59,22 +60,21 @@ Strimzi stores credentials in Kubernetes secrets. Each secret contains two keys: ### Topics -Declared in `kafka-topics.yaml`, never auto-created — an application account has -no `Create` ACL, so a producer would simply be denied, and an auto-created topic -takes broker defaults nobody chose. +This repo declares none, for the same reason it declares no application account: +a topic belongs to whoever publishes on it, and its partition count, retention +and cleanup policy are that application's decisions, not the broker's. -| Topic | Partitions | Replicas | Retention | -|---|---|---|---| -| `classic-models.catalog` | 3 | 1 | 7 days | -| `classic-models.sales` | 3 | 1 | 7 days | -| `classic-models.payments` | 3 | 1 | 7 days | +Declare them as `KafkaTopic` CRs from the producer's own repo, **in this +namespace** — Strimzi runs with `watchAnyNamespace=false`, so one created +anywhere else is ignored, silently. -Partition count is the ceiling on consumer parallelism and **cannot be lowered -later**, so it is set for the traffic expected rather than the traffic today. +Declare them rather than relying on auto-creation: an application account has no +`Create` ACL, so a producer would simply be denied, and an auto-created topic +takes broker defaults nobody chose. Partition count is the ceiling on consumer +parallelism and **cannot be lowered later**. -The events these carry are specified by their producer, in -`classic-models-api/docs/DOMAIN_EVENTS.md`. This repo owns the cluster, not the -contract. +`make topic-create TOPIC=… PARTITIONS=…` exists for a scratch topic, not for +anything an application depends on. ### Getting credentials into another namespace @@ -179,8 +179,7 @@ The target performs the following steps: 3. Waits for the Strimzi CRDs to be fully established 4. Applies `kafka.yaml` (KafkaNodePool + Kafka CRs) and waits for `Ready` 5. Applies `kafka-users.yaml` (KafkaUser CRs) and waits for `Ready` -6. Applies `kafka-topics.yaml` (the declared topics) and `credentials-access.yaml` - (the cross-namespace credentials store) +6. Applies `credentials-access.yaml` (the cross-namespace credentials store) Expected duration: **4–6 minutes** on a single-node k3s cluster. @@ -279,7 +278,9 @@ The job is cleaned up automatically after 1 minute (`ttlSecondsAfterFinished: 60 ## Adding a new user -Create a new `KafkaUser` manifest following the `kafka-client` pattern in `kafka-users.yaml`, then apply it: +Copy the pattern documented at the bottom of `kafka-users.yaml` into the +application's **own** repo, then apply it — into this namespace, since Strimzi +watches no other: ```bash kubectl apply -f kafka-users.yaml diff --git a/kafka-topics.yaml b/kafka-topics.yaml deleted file mode 100644 index dc18cbe..0000000 --- a/kafka-topics.yaml +++ /dev/null @@ -1,56 +0,0 @@ ---- -# Topics are declared, never auto-created. -# -# Two reasons. An application account has no Create ACL — a producer that can -# invent topics can also invent a typo and publish into it for weeks with nobody -# noticing. And auto-created topics take the broker defaults, which is how a -# topic that should be compacted and retained for a year ends up deleted after -# seven days. -# -# One topic per aggregate. Partition count is the ceiling on consumer -# parallelism and cannot be lowered later, so it is set for the traffic -# expected, not for the traffic today. -apiVersion: kafka.strimzi.io/v1 -kind: KafkaTopic -metadata: - name: classic-models.catalog - namespace: kafka - labels: - strimzi.io/cluster: kafka -spec: - partitions: 3 - replicas: 1 - config: - retention.ms: "604800000" # 7 days - cleanup.policy: delete - min.insync.replicas: "1" ---- -apiVersion: kafka.strimzi.io/v1 -kind: KafkaTopic -metadata: - name: classic-models.sales - namespace: kafka - labels: - strimzi.io/cluster: kafka -spec: - partitions: 3 - replicas: 1 - config: - retention.ms: "604800000" - cleanup.policy: delete - min.insync.replicas: "1" ---- -apiVersion: kafka.strimzi.io/v1 -kind: KafkaTopic -metadata: - name: classic-models.payments - namespace: kafka - labels: - strimzi.io/cluster: kafka -spec: - partitions: 3 - replicas: 1 - config: - retention.ms: "604800000" - cleanup.policy: delete - min.insync.replicas: "1" diff --git a/kafka-users.yaml b/kafka-users.yaml index 1c3e7e6..84ea579 100644 --- a/kafka-users.yaml +++ b/kafka-users.yaml @@ -52,64 +52,39 @@ spec: operations: - Describe - Write ---- -# The classic-models API, which publishes domain events and consumes nothing. -# -# Write-only and topic-scoped: a producer that can also read every topic is a -# producer that can exfiltrate every other domain's events if it is ever -# compromised. Describe is needed for metadata; it does not grant reads. -apiVersion: kafka.strimzi.io/v1 -kind: KafkaUser -metadata: - name: classic-models-api - namespace: kafka - labels: - strimzi.io/cluster: kafka -spec: - authentication: - type: scram-sha-512 - authorization: - type: simple - acls: - - resource: - type: topic - name: classic-models. - patternType: prefix - operations: - - Write - - Describe ---- -# The Approvisionnement agent, which is woken by catalog events. +# --- adding an application account ---------------------------------------- # -# Read on one topic and one consumer-group prefix, nothing else. It cannot see -# sales or payments — an agent whose job is stock has no business reading who -# paid what — and it cannot write, so a compromised workflow can raise no event -# that another agent would then act on. -apiVersion: kafka.strimzi.io/v1 -kind: KafkaUser -metadata: - name: classic-models-agent-approvisionnement - namespace: kafka - labels: - strimzi.io/cluster: kafka -spec: - authentication: - type: scram-sha-512 - authorization: - type: simple - acls: - - resource: - type: topic - name: classic-models.catalog - patternType: literal - operations: - - Read - - Describe - - resource: - type: group - name: approvisionnement- - patternType: prefix - operations: - - Read - - Describe +# Do not add one here. An account belongs to the application that uses it: its +# own repo says what it may read and write, and adding or removing the +# application adds or removes its access. This repo deploys a cluster and holds +# no opinion about who uses it. +# +# Create the CR from the application's own repo, in **this** namespace — Strimzi +# runs with `watchAnyNamespace=false`, so a KafkaUser anywhere else is ignored, +# silently. Copy the shape below and cut it down to what the application does: +# +# apiVersion: kafka.strimzi.io/v1 +# kind: KafkaUser +# metadata: +# name: +# namespace: kafka +# labels: +# strimzi.io/cluster: kafka +# spec: +# authentication: +# type: scram-sha-512 +# authorization: +# type: simple +# acls: +# - resource: {type: topic, name: , patternType: prefix} +# operations: [Write, Describe] # a producer +# - resource: {type: group, name: , patternType: prefix} +# operations: [Read, Describe] # a consumer's group +# +# A producer that can also read every topic exfiltrates every other +# application's data if it is ever compromised, and a consumer that can write +# can raise an event someone else acts on. Grant one direction, on one prefix. +# +# `kafka-client` above is the wide-open template: useful for a smoke test, and +# for nothing that runs unattended.