diff --git a/flux/kafka-topology.yaml b/flux/kafka-topology.yaml new file mode 100644 index 0000000..f4d0df0 --- /dev/null +++ b/flux/kafka-topology.yaml @@ -0,0 +1,21 @@ +# Topics and accounts on the shared Kafka cluster. +# +# Platform-owned, like the domain defaults beside it: an ACL is not a team's +# decision to make about itself, and a partition count is not a team's decision +# to make about the cluster. +# +# No targetNamespace — every object names `kafka` explicitly, because that is +# the only namespace Strimzi watches and putting one anywhere else fails in +# silence. +apiVersion: kustomize.toolkit.fluxcd.io/v1 +kind: Kustomization +metadata: + name: kafka-topology + namespace: flux-system +spec: + interval: 10m + sourceRef: + kind: GitRepository + name: k8s-platform + path: ./kafka-topology + prune: true diff --git a/flux/kustomization.yaml b/flux/kustomization.yaml index 1b19eac..6f66056 100644 --- a/flux/kustomization.yaml +++ b/flux/kustomization.yaml @@ -3,3 +3,4 @@ kind: Kustomization resources: - fluxinstance.yaml - gitrepository.yaml + - kafka-topology.yaml diff --git a/kafka-topology/README.md b/kafka-topology/README.md new file mode 100644 index 0000000..23396a8 --- /dev/null +++ b/kafka-topology/README.md @@ -0,0 +1,48 @@ +# Kafka topology + +Platform-owned. Topics and accounts on the shared Kafka cluster are administered +here, not by the teams that use them. + +That is the same reason gateway defaults are here: **an ACL is not a team's +decision to make about itself.** A team that declares its own `KafkaUser` grants +its own permissions, and a team that declares its own `KafkaTopic` sets a +partition count and a retention that the cluster then has to live with. Both are +platform decisions, taken with a view of the whole cluster. + +It is also why this is not self-service, and should not be mistaken for it. See +"Self-service" below. + +## What lives here + +| | | +|---|---| +| `classic-models.yaml` | the `classic-models` estate: three topics, one producer account, one consumer account | + +The **cluster** is not here — `infra-kafka` is a generic, reusable Strimzi +deployment and names no estate. This repo says who may publish what on it. + +The **events** are not here either: their contract belongs to the application +that raises them, in `classic-models-api/docs/DOMAIN_EVENTS.md`. This repo grants +access; it does not define meaning. + +## Two things that bite + +**Every CR goes in the operator's namespace.** Strimzi runs with +`watchAnyNamespace=false`, so a `KafkaUser` or `KafkaTopic` created anywhere else +is ignored **silently** — `kubectl apply` says `created`, `kubectl get` shows it, +and it never gets a `status`, a Secret, or a line in the operator's log. It looks +exactly like one still being reconciled. + +**Grant one direction, on one prefix.** A producer that can also read every topic +exfiltrates every other application's data if it is ever compromised; a consumer +that can write can raise an event someone else acts on. + +## Self-service, when it comes + +Not by letting teams declare CRs here or in their own repos. Through an **event +gateway**, which virtualises topics so a consumer is given a contract rather than +a broker address and an ACL, and through the **developer portal**, which +socialises those topics the way it already socialises the REST APIs. + +Until that exists, this directory is the request queue: a team asks, the platform +team writes the entry. diff --git a/kafka-topology/classic-models.yaml b/kafka-topology/classic-models.yaml new file mode 100644 index 0000000..36d605e --- /dev/null +++ b/kafka-topology/classic-models.yaml @@ -0,0 +1,123 @@ +# The classic-models estate on the shared Kafka cluster. +# +# Everything here is created in the `kafka` namespace, because that is the only +# one Strimzi watches (`watchAnyNamespace=false`). A CR elsewhere is ignored in +# silence — no status, no Secret, no log line. +# +# The events these topics carry are specified by their producer, in +# classic-models-api/docs/DOMAIN_EVENTS.md. This file grants access; it does not +# define meaning. + +# --- topics ----------------------------------------------------------------- +# +# One per aggregate, not one per event type: per type would lose the order +# between product.created and product.stock_changed for the same product, which +# is the one ordering a consumer actually needs. +# +# Declared rather than auto-created — the accounts below have no Create ACL, so +# a producer would simply be denied, and an auto-created topic takes broker +# defaults nobody chose. Partition count is the ceiling on consumer parallelism +# and cannot be lowered later. +--- +apiVersion: kafka.strimzi.io/v1 +kind: KafkaTopic +metadata: + name: classic-models.catalog + namespace: kafka + labels: + strimzi.io/cluster: kafka +spec: + partitions: 3 + replicas: 1 + config: + retention.ms: "604800000" # 7 days + cleanup.policy: delete + min.insync.replicas: "1" +--- +apiVersion: kafka.strimzi.io/v1 +kind: KafkaTopic +metadata: + name: classic-models.sales + namespace: kafka + labels: + strimzi.io/cluster: kafka +spec: + partitions: 3 + replicas: 1 + config: + retention.ms: "604800000" + cleanup.policy: delete + min.insync.replicas: "1" +--- +apiVersion: kafka.strimzi.io/v1 +kind: KafkaTopic +metadata: + name: classic-models.payments + namespace: kafka + labels: + strimzi.io/cluster: kafka +spec: + partitions: 3 + replicas: 1 + config: + retention.ms: "604800000" + cleanup.policy: delete + min.insync.replicas: "1" + +# --- accounts --------------------------------------------------------------- +# +# One direction each, on one prefix. A producer that could also read every topic +# exfiltrates every other application's data if it is ever compromised, and a +# consumer that could write can raise an event someone else acts on. +--- +# classic-models-api. Publishes domain events, consumes nothing. +apiVersion: kafka.strimzi.io/v1 +kind: KafkaUser +metadata: + name: classic-models-api + namespace: kafka + labels: + strimzi.io/cluster: kafka +spec: + authentication: + type: scram-sha-512 + authorization: + type: simple + acls: + - resource: + type: topic + name: classic-models. + patternType: prefix + operations: + - Write + - Describe +--- +# The Approvisionnement agent. Reads the catalog topic only — an agent whose job +# is stock has no business reading who paid what. +apiVersion: kafka.strimzi.io/v1 +kind: KafkaUser +metadata: + name: classic-models-agent-approvisionnement + namespace: kafka + labels: + strimzi.io/cluster: kafka +spec: + authentication: + type: scram-sha-512 + authorization: + type: simple + acls: + - resource: + type: topic + name: classic-models.catalog + patternType: literal + operations: + - Read + - Describe + - resource: + type: group + name: approvisionnement- + patternType: prefix + operations: + - Read + - Describe diff --git a/kafka-topology/kustomization.yaml b/kafka-topology/kustomization.yaml new file mode 100644 index 0000000..069895f --- /dev/null +++ b/kafka-topology/kustomization.yaml @@ -0,0 +1,4 @@ +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +resources: + - classic-models.yaml