Administer the Kafka topology centrally

Topics and accounts on the shared cluster belong to the platform team, for the
same reason the gateway defaults do: an ACL is not a team's decision to make
about itself, and a partition count is not a team's decision to make about the
cluster.

They had been pushed into the application repos on the argument that ownership
follows use. That argument holds for a contract, not for a grant.

Self-service is a different thing and does not come from moving these files: it
comes from an event gateway that virtualises topics and a developer portal that
socialises them.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

flux/kafka-topology.yaml

@@ -0,0 +1,21 @@
+# Topics and accounts on the shared Kafka cluster.
+#
+# Platform-owned, like the domain defaults beside it: an ACL is not a team's
+# decision to make about itself, and a partition count is not a team's decision
+# to make about the cluster.
+#
+# No targetNamespace — every object names `kafka` explicitly, because that is
+# the only namespace Strimzi watches and putting one anywhere else fails in
+# silence.
+apiVersion: kustomize.toolkit.fluxcd.io/v1
+kind: Kustomization
+metadata:
+  name: kafka-topology
+  namespace: flux-system
+spec:
+  interval: 10m
+  sourceRef:
+    kind: GitRepository
+    name: k8s-platform
+  path: ./kafka-topology
+  prune: true

flux/kustomization.yaml

@@ -3,3 +3,4 @@ kind: Kustomization
 resources:
   - fluxinstance.yaml
   - gitrepository.yaml
+  - kafka-topology.yaml

kafka-topology/README.md

@@ -0,0 +1,48 @@
+# Kafka topology
+
+Platform-owned. Topics and accounts on the shared Kafka cluster are administered
+here, not by the teams that use them.
+
+That is the same reason gateway defaults are here: **an ACL is not a team's
+decision to make about itself.** A team that declares its own `KafkaUser` grants
+its own permissions, and a team that declares its own `KafkaTopic` sets a
+partition count and a retention that the cluster then has to live with. Both are
+platform decisions, taken with a view of the whole cluster.
+
+It is also why this is not self-service, and should not be mistaken for it. See
+"Self-service" below.
+
+## What lives here
+
+| | |
+|---|---|
+| `classic-models.yaml` | the `classic-models` estate: three topics, one producer account, one consumer account |
+
+The **cluster** is not here — `infra-kafka` is a generic, reusable Strimzi
+deployment and names no estate. This repo says who may publish what on it.
+
+The **events** are not here either: their contract belongs to the application
+that raises them, in `classic-models-api/docs/DOMAIN_EVENTS.md`. This repo grants
+access; it does not define meaning.
+
+## Two things that bite
+
+**Every CR goes in the operator's namespace.** Strimzi runs with
+`watchAnyNamespace=false`, so a `KafkaUser` or `KafkaTopic` created anywhere else
+is ignored **silently** — `kubectl apply` says `created`, `kubectl get` shows it,
+and it never gets a `status`, a Secret, or a line in the operator's log. It looks
+exactly like one still being reconciled.
+
+**Grant one direction, on one prefix.** A producer that can also read every topic
+exfiltrates every other application's data if it is ever compromised; a consumer
+that can write can raise an event someone else acts on.
+
+## Self-service, when it comes
+
+Not by letting teams declare CRs here or in their own repos. Through an **event
+gateway**, which virtualises topics so a consumer is given a contract rather than
+a broker address and an ACL, and through the **developer portal**, which
+socialises those topics the way it already socialises the REST APIs.
+
+Until that exists, this directory is the request queue: a team asks, the platform
+team writes the entry.

kafka-topology/classic-models.yaml

@@ -0,0 +1,123 @@
+# The classic-models estate on the shared Kafka cluster.
+#
+# Everything here is created in the `kafka` namespace, because that is the only
+# one Strimzi watches (`watchAnyNamespace=false`). A CR elsewhere is ignored in
+# silence — no status, no Secret, no log line.
+#
+# The events these topics carry are specified by their producer, in
+# classic-models-api/docs/DOMAIN_EVENTS.md. This file grants access; it does not
+# define meaning.
+
+# --- topics -----------------------------------------------------------------
+#
+# One per aggregate, not one per event type: per type would lose the order
+# between product.created and product.stock_changed for the same product, which
+# is the one ordering a consumer actually needs.
+#
+# Declared rather than auto-created — the accounts below have no Create ACL, so
+# a producer would simply be denied, and an auto-created topic takes broker
+# defaults nobody chose. Partition count is the ceiling on consumer parallelism
+# and cannot be lowered later.
+---
+apiVersion: kafka.strimzi.io/v1
+kind: KafkaTopic
+metadata:
+  name: classic-models.catalog
+  namespace: kafka
+  labels:
+    strimzi.io/cluster: kafka
+spec:
+  partitions: 3
+  replicas: 1
+  config:
+    retention.ms: "604800000"   # 7 days
+    cleanup.policy: delete
+    min.insync.replicas: "1"
+---
+apiVersion: kafka.strimzi.io/v1
+kind: KafkaTopic
+metadata:
+  name: classic-models.sales
+  namespace: kafka
+  labels:
+    strimzi.io/cluster: kafka
+spec:
+  partitions: 3
+  replicas: 1
+  config:
+    retention.ms: "604800000"
+    cleanup.policy: delete
+    min.insync.replicas: "1"
+---
+apiVersion: kafka.strimzi.io/v1
+kind: KafkaTopic
+metadata:
+  name: classic-models.payments
+  namespace: kafka
+  labels:
+    strimzi.io/cluster: kafka
+spec:
+  partitions: 3
+  replicas: 1
+  config:
+    retention.ms: "604800000"
+    cleanup.policy: delete
+    min.insync.replicas: "1"
+
+# --- accounts ---------------------------------------------------------------
+#
+# One direction each, on one prefix. A producer that could also read every topic
+# exfiltrates every other application's data if it is ever compromised, and a
+# consumer that could write can raise an event someone else acts on.
+---
+# classic-models-api. Publishes domain events, consumes nothing.
+apiVersion: kafka.strimzi.io/v1
+kind: KafkaUser
+metadata:
+  name: classic-models-api
+  namespace: kafka
+  labels:
+    strimzi.io/cluster: kafka
+spec:
+  authentication:
+    type: scram-sha-512
+  authorization:
+    type: simple
+    acls:
+      - resource:
+          type: topic
+          name: classic-models.
+          patternType: prefix
+        operations:
+          - Write
+          - Describe
+---
+# The Approvisionnement agent. Reads the catalog topic only — an agent whose job
+# is stock has no business reading who paid what.
+apiVersion: kafka.strimzi.io/v1
+kind: KafkaUser
+metadata:
+  name: classic-models-agent-approvisionnement
+  namespace: kafka
+  labels:
+    strimzi.io/cluster: kafka
+spec:
+  authentication:
+    type: scram-sha-512
+  authorization:
+    type: simple
+    acls:
+      - resource:
+          type: topic
+          name: classic-models.catalog
+          patternType: literal
+        operations:
+          - Read
+          - Describe
+      - resource:
+          type: group
+          name: approvisionnement-
+          patternType: prefix
+        operations:
+          - Read
+          - Describe

kafka-topology/kustomization.yaml

@@ -0,0 +1,4 @@
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+resources:
+  - classic-models.yaml