diff --git a/Makefile b/Makefile index ebe60d5..e07ac05 100644 --- a/Makefile +++ b/Makefile @@ -81,8 +81,7 @@ deploy: @for user in kafka-admin kafka-client; do \ kubectl wait kafkauser/$$user --for=condition=Ready --timeout=120s -n $(NAMESPACE); \ done - @echo "==> [6/6] Deploying topics and the credentials store" - @kubectl apply -f $(SCRIPT_DIR)kafka-topics.yaml + @echo "==> [6/6] Deploying the credentials store" @kubectl apply -f $(SCRIPT_DIR)credentials-access.yaml @echo "" @echo "Cluster ready. Bootstrap: $(BOOTSTRAP)" @@ -116,7 +115,6 @@ upgrade: teardown: @echo "==> Deleting KafkaUsers and Kafka cluster" @kubectl delete -f $(SCRIPT_DIR)credentials-access.yaml --ignore-not-found - @kubectl delete -f $(SCRIPT_DIR)kafka-topics.yaml --ignore-not-found @kubectl delete -f $(SCRIPT_DIR)kafka-users.yaml --ignore-not-found @kubectl delete -f $(SCRIPT_DIR)kafka.yaml --ignore-not-found @echo "==> Waiting for pods to terminate" diff --git a/README.md b/README.md index 96b44d6..b3ac8a3 100644 --- a/README.md +++ b/README.md @@ -40,18 +40,19 @@ kafka-kafka-bootstrap.kafka.svc.cluster.local:9093 |---|---|---| | `kafka-admin` | `kafka-admin` | Super user — unrestricted access | | `kafka-client` | `kafka-client` | Application user — Read/Write all topics and groups | -| `classic-models-api` | `classic-models-api` | Producer for `classic-models.*` — **Write and Describe only**, no Read | -| `classic-models-agent-approvisionnement` | same name | Consumer of `classic-models.catalog` and group prefix `approvisionnement-` — **Read and Describe only**, no Write | -Each application account is scoped to what it actually does. The producer -publishes domain events and consumes nothing, so it is write-only: one that -could also read every topic exfiltrates every other domain's events if it is -ever compromised. The consumer is the mirror — it cannot see sales or payments, -and it cannot write, so a compromised workflow can raise no event that another -agent would then act on. +These two are all this repo declares. **An application's account is not added +here** — it belongs to the application, whose own repo says what it may read and +write, so that adding or removing the application adds or removes its access. +This repo deploys a cluster and holds no opinion about who uses it. -Duplicate one of these per application. `kafka-client` is the wide-open -template, useful for a smoke test and for nothing that runs unattended. +`kafka-users.yaml` carries the shape to copy. Grant **one direction, on one +prefix**: a producer that can also read every topic exfiltrates every other +application's data if it is ever compromised, and a consumer that can write can +raise an event someone else acts on. + +`kafka-client` is the wide-open template — useful for a smoke test, and for +nothing that runs unattended. Strimzi stores credentials in Kubernetes secrets. Each secret contains two keys: - `password` — the SCRAM password @@ -59,22 +60,21 @@ Strimzi stores credentials in Kubernetes secrets. Each secret contains two keys: ### Topics -Declared in `kafka-topics.yaml`, never auto-created — an application account has -no `Create` ACL, so a producer would simply be denied, and an auto-created topic -takes broker defaults nobody chose. +This repo declares none, for the same reason it declares no application account: +a topic belongs to whoever publishes on it, and its partition count, retention +and cleanup policy are that application's decisions, not the broker's. -| Topic | Partitions | Replicas | Retention | -|---|---|---|---| -| `classic-models.catalog` | 3 | 1 | 7 days | -| `classic-models.sales` | 3 | 1 | 7 days | -| `classic-models.payments` | 3 | 1 | 7 days | +Declare them as `KafkaTopic` CRs from the producer's own repo, **in this +namespace** — Strimzi runs with `watchAnyNamespace=false`, so one created +anywhere else is ignored, silently. -Partition count is the ceiling on consumer parallelism and **cannot be lowered -later**, so it is set for the traffic expected rather than the traffic today. +Declare them rather than relying on auto-creation: an application account has no +`Create` ACL, so a producer would simply be denied, and an auto-created topic +takes broker defaults nobody chose. Partition count is the ceiling on consumer +parallelism and **cannot be lowered later**. -The events these carry are specified by their producer, in -`classic-models-api/docs/DOMAIN_EVENTS.md`. This repo owns the cluster, not the -contract. +`make topic-create TOPIC=… PARTITIONS=…` exists for a scratch topic, not for +anything an application depends on. ### Getting credentials into another namespace @@ -179,8 +179,7 @@ The target performs the following steps: 3. Waits for the Strimzi CRDs to be fully established 4. Applies `kafka.yaml` (KafkaNodePool + Kafka CRs) and waits for `Ready` 5. Applies `kafka-users.yaml` (KafkaUser CRs) and waits for `Ready` -6. Applies `kafka-topics.yaml` (the declared topics) and `credentials-access.yaml` - (the cross-namespace credentials store) +6. Applies `credentials-access.yaml` (the cross-namespace credentials store) Expected duration: **4–6 minutes** on a single-node k3s cluster. @@ -279,7 +278,9 @@ The job is cleaned up automatically after 1 minute (`ttlSecondsAfterFinished: 60 ## Adding a new user -Create a new `KafkaUser` manifest following the `kafka-client` pattern in `kafka-users.yaml`, then apply it: +Copy the pattern documented at the bottom of `kafka-users.yaml` into the +application's **own** repo, then apply it — into this namespace, since Strimzi +watches no other: ```bash kubectl apply -f kafka-users.yaml diff --git a/kafka-topics.yaml b/kafka-topics.yaml deleted file mode 100644 index dc18cbe..0000000 --- a/kafka-topics.yaml +++ /dev/null @@ -1,56 +0,0 @@ ---- -# Topics are declared, never auto-created. -# -# Two reasons. An application account has no Create ACL — a producer that can -# invent topics can also invent a typo and publish into it for weeks with nobody -# noticing. And auto-created topics take the broker defaults, which is how a -# topic that should be compacted and retained for a year ends up deleted after -# seven days. -# -# One topic per aggregate. Partition count is the ceiling on consumer -# parallelism and cannot be lowered later, so it is set for the traffic -# expected, not for the traffic today. -apiVersion: kafka.strimzi.io/v1 -kind: KafkaTopic -metadata: - name: classic-models.catalog - namespace: kafka - labels: - strimzi.io/cluster: kafka -spec: - partitions: 3 - replicas: 1 - config: - retention.ms: "604800000" # 7 days - cleanup.policy: delete - min.insync.replicas: "1" ---- -apiVersion: kafka.strimzi.io/v1 -kind: KafkaTopic -metadata: - name: classic-models.sales - namespace: kafka - labels: - strimzi.io/cluster: kafka -spec: - partitions: 3 - replicas: 1 - config: - retention.ms: "604800000" - cleanup.policy: delete - min.insync.replicas: "1" ---- -apiVersion: kafka.strimzi.io/v1 -kind: KafkaTopic -metadata: - name: classic-models.payments - namespace: kafka - labels: - strimzi.io/cluster: kafka -spec: - partitions: 3 - replicas: 1 - config: - retention.ms: "604800000" - cleanup.policy: delete - min.insync.replicas: "1" diff --git a/kafka-users.yaml b/kafka-users.yaml index 1c3e7e6..84ea579 100644 --- a/kafka-users.yaml +++ b/kafka-users.yaml @@ -52,64 +52,39 @@ spec: operations: - Describe - Write ---- -# The classic-models API, which publishes domain events and consumes nothing. -# -# Write-only and topic-scoped: a producer that can also read every topic is a -# producer that can exfiltrate every other domain's events if it is ever -# compromised. Describe is needed for metadata; it does not grant reads. -apiVersion: kafka.strimzi.io/v1 -kind: KafkaUser -metadata: - name: classic-models-api - namespace: kafka - labels: - strimzi.io/cluster: kafka -spec: - authentication: - type: scram-sha-512 - authorization: - type: simple - acls: - - resource: - type: topic - name: classic-models. - patternType: prefix - operations: - - Write - - Describe ---- -# The Approvisionnement agent, which is woken by catalog events. +# --- adding an application account ---------------------------------------- # -# Read on one topic and one consumer-group prefix, nothing else. It cannot see -# sales or payments — an agent whose job is stock has no business reading who -# paid what — and it cannot write, so a compromised workflow can raise no event -# that another agent would then act on. -apiVersion: kafka.strimzi.io/v1 -kind: KafkaUser -metadata: - name: classic-models-agent-approvisionnement - namespace: kafka - labels: - strimzi.io/cluster: kafka -spec: - authentication: - type: scram-sha-512 - authorization: - type: simple - acls: - - resource: - type: topic - name: classic-models.catalog - patternType: literal - operations: - - Read - - Describe - - resource: - type: group - name: approvisionnement- - patternType: prefix - operations: - - Read - - Describe +# Do not add one here. An account belongs to the application that uses it: its +# own repo says what it may read and write, and adding or removing the +# application adds or removes its access. This repo deploys a cluster and holds +# no opinion about who uses it. +# +# Create the CR from the application's own repo, in **this** namespace — Strimzi +# runs with `watchAnyNamespace=false`, so a KafkaUser anywhere else is ignored, +# silently. Copy the shape below and cut it down to what the application does: +# +# apiVersion: kafka.strimzi.io/v1 +# kind: KafkaUser +# metadata: +# name: +# namespace: kafka +# labels: +# strimzi.io/cluster: kafka +# spec: +# authentication: +# type: scram-sha-512 +# authorization: +# type: simple +# acls: +# - resource: {type: topic, name: , patternType: prefix} +# operations: [Write, Describe] # a producer +# - resource: {type: group, name: , patternType: prefix} +# operations: [Read, Describe] # a consumer's group +# +# A producer that can also read every topic exfiltrates every other +# application's data if it is ever compromised, and a consumer that can write +# can raise an event someone else acts on. Grant one direction, on one prefix. +# +# `kafka-client` above is the wide-open template: useful for a smoke test, and +# for nothing that runs unattended.