stt/infra-kafka
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Adding a Makefile

Browse Source
This commit is contained in:
sttlab
2026-05-15 08:10:45 +00:00
parent 569dff28a7
commit e99cc49b8a
4 changed files with 279 additions and 56 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+67 -55
View File
@@ -60,17 +60,59 @@ This certificate must be trusted by any Kafka client connecting over TLS.
- `kubectl` configured against the target cluster
- `helm` v3
- `make`
- `cert-manager` is **not** required — Strimzi manages its own CA
---
## Makefile
All operations are available via `make`. Run `make help` for the full reference.
```
Cluster lifecycle:
deploy Install Strimzi operator + Kafka cluster + users
smoke-test Run produce/consume end-to-end test
upgrade Upgrade Kafka (KAFKA_VER=) or Strimzi operator
teardown Delete the Kafka cluster and namespace
Topics:
topic-list List all topics
topic-create Create TOPIC (idempotent)
topic-delete Delete TOPIC
Messages:
produce Produce MESSAGE into TOPIC
consume Consume MAX messages from TOPIC (from latest)
list-messages Consume MAX messages from TOPIC (from beginning)
Consumer groups:
group-list List all consumer groups
group-offsets Show offsets and lag for GROUP
```
Key variables (all overridable on the command line):
| Variable | Default | Description |
|---|---|---|
| `TOPIC` | `smoke-test` | Topic name |
| `MESSAGE` | `hello` | Message to produce |
| `GROUP` | `my-group` | Consumer group |
| `MAX` | `10` | Max messages to consume |
| `PARTITIONS` | `1` | Partitions for topic-create |
| `REPLICAS` | `1` | Replication factor for topic-create |
| `KAFKA_VER` | `4.1.0` | Kafka version for upgrade |
| `METADATA_VER` | `4.1-IV0` | KRaft metadata version for upgrade |
---
## Installation
```bash
./deploy.sh
make deploy
```
The script performs the following steps:
The target performs the following steps:
1. Creates the `kafka` namespace
2. Installs the Strimzi operator via Helm (scoped to the `kafka` namespace)
@@ -143,24 +185,11 @@ sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule require
## Testing
A smoke test job is provided that validates the full produce/consume cycle over SASL_SSL.
### Run the smoke test
```bash
kubectl apply -f kafka-smoke-test.yaml
kubectl logs -n kafka -l job-name=kafka-smoke-test -f
make smoke-test
```
### Re-run
```bash
kubectl delete job kafka-smoke-test -n kafka
kubectl apply -f kafka-smoke-test.yaml
kubectl logs -n kafka -l job-name=kafka-smoke-test -f
```
### What the test does
Runs a Kubernetes Job that validates the full produce/consume cycle over SASL_SSL:
1. **Topic creation** — creates topic `smoke-test` with `--if-not-exists` (idempotent)
2. **Produce** — sends 5 messages (`message-1` … `message-5`) via `kafka-console-producer`
@@ -179,9 +208,10 @@ message-3
message-4
message-5
==> SMOKE TEST PASSED (5/5 messages)
==> Smoke test PASSED
```
The job is cleaned up automatically after 10 minutes (`ttlSecondsAfterFinished: 600`).
The job is cleaned up automatically after 1 minute (`ttlSecondsAfterFinished: 60`).
---
@@ -199,59 +229,41 @@ Strimzi will create the corresponding secret in the `kafka` namespace within sec
## Upgrade
### Upgrade the Kafka version
> **Rule:** always upgrade the Strimzi operator before upgrading the Kafka version.
> Check the [Strimzi upgrade guide](https://strimzi.io/docs/operators/latest/deploying.html#assembly-upgrade-str) for supported upgrade paths.
Supported versions are dictated by the installed Strimzi operator. Check available versions:
### Upgrade Kafka version
`KAFKA_VER` and `METADATA_VER` are independent because Strimzi recommends upgrading
the broker binary first, then migrating the metadata format — this allows rollback of
the binary without being blocked by an already-migrated metadata version.
```bash
kubectl get kafka kafka -n kafka \
-o jsonpath='{.status.kafkaVersion}'
# Step 1: upgrade broker binary only
make upgrade KAFKA_VER=4.1.1 METADATA_VER=4.1-IV0
helm show chart strimzi/strimzi-kafka-operator | grep appVersion
# Step 2: once confirmed stable, migrate metadata format
make upgrade KAFKA_VER=4.1.1 METADATA_VER=4.1-IV1
```
To upgrade Kafka from e.g. `4.1.0` to `4.1.1`:
1. Update `spec.kafka.version` in `kafka.yaml`
2. Update `spec.kafka.metadataVersion` if the new version introduces a new metadata version
3. Apply the change:
```bash
kubectl apply -f kafka.yaml
kubectl wait kafka/kafka --for=condition=Ready --timeout=10m -n kafka
```
Strimzi performs a rolling restart of the broker pod automatically.
### Upgrade the Strimzi operator
```bash
helm repo update strimzi
helm upgrade strimzi-kafka-operator strimzi/strimzi-kafka-operator \
--namespace kafka \
--set watchAnyNamespace=false \
--wait --timeout 5m
make upgrade
```
> **Note:** always upgrade the operator before upgrading the Kafka version. Check the [Strimzi upgrade guide](https://strimzi.io/docs/operators/latest/deploying.html#assembly-upgrade-str) for supported upgrade paths.
Running `upgrade` without overriding `KAFKA_VER` upgrades only the Strimzi operator
via `helm upgrade` and leaves the Kafka CR unchanged.
---
## Uninstallation
```bash
# Delete Kafka cluster and users (PVC is deleted because deleteClaim: true)
kubectl delete -f kafka-users.yaml
kubectl delete -f kafka.yaml
# Wait for pods to terminate
kubectl wait --for=delete pod -l strimzi.io/cluster=kafka -n kafka --timeout=120s
# Uninstall Strimzi operator
helm uninstall strimzi-kafka-operator -n kafka
# Delete namespace (removes all remaining secrets and CRDs bindings)
kubectl delete namespace kafka
make teardown
```
> The `deleteClaim: true` flag in `kafka.yaml` ensures the 10 Gi PVC is deleted together with the KafkaNodePool, leaving no orphaned volumes.
Deletes in order: KafkaUsers → Kafka cluster → Strimzi operator → namespace.
> The `deleteClaim: true` flag in `kafka.yaml` ensures the 10 Gi PVC is deleted
> together with the KafkaNodePool, leaving no orphaned volumes.