configmap-realm.yaml was a first-install seed: `start --import-realm`
skips a realm that already exists and logs "Import skipped", so editing
the file changed nothing on a restart and the truth drifted into the
database. It already had — four scope grants existed only in the running
realm.
There is no option to make the startup import overwrite. The --override
flag exists only on the standalone `import` command, and the Keycloak
documentation is explicit that re-creating realms means running that
command before starting the server. So the import moves into an init
container that runs `import --dir ... --override true`, and --import-realm
comes off the main container's args.
The init container empties JAVA_OPTS_APPEND for itself. It inherits the
main container's env, which loads the OpenTelemetry agent from a volume
the other init container downloads into and this one does not mount;
inherited verbatim, the JVM fails before the import runs.
What this costs, deliberately: the realm is replaced at every start, so
anything created at runtime and not written back to this file is lost, and
active sessions are dropped. Checked before enabling it — the five users
in the realm are the five declared here, and the only live sessions were
two LibreChat ones. Nothing is to be created through the admin console
expecting it to last.
Verified after the switch: the import logs "Strategy: OVERWRITE_EXISTING",
the realm still has its 21 clients with no optionalClientScopes
divergence, and the full identity chain still returns 200 — pm-user token,
exchanged by the Catalogue agent, accepted by catalog-mcp.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file is a bootstrap seed, not the live realm. Keycloak starts with
--import-realm under IGNORE_EXISTING and logs "Realm 'classic-models'
already exists. Import skipped", so editing it changes nothing on a
restart and the truth lives in Keycloak's database.
That divergence had already opened, in the direction that hurts: four
authorisations existed only in the running realm, granted through the
admin API while working out which audiences the gateway actually needs.
classic-models-agent-catalogue +catalog-api-audience
+sales-api-audience
+kong-token-exchange-broker-audience
librechat +classic-models-agent-catalogue-audience
Rebuilding the realm from this file would have left the Catalogue agent
with two of its five scopes and no indication why it stopped working.
Compared object by object against the live realm: the inventories already
matched — 21 clients, 38 client scopes, none missing either way — so the
drift was in contents, not in the lists. optionalClientScopes are now
copied back for every client that differed, and a re-comparison reports no
remaining gap.
This closes today's drift; it does not fix the mechanism that allows it.
Keeping a seed honest by hand is the same arrangement the Flux entrypoints
had before they were made self-reconciling, and it will drift again.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
First of the six agents to need an identity. It exchanges the end user's
token for one narrowly-scoped token per MCP server, so `azp` names the
agent while `sub` stays the human — which is what makes the audit trail
able to answer "which agent asked, on whose behalf".
Deliberately narrower than the librechat client it is modelled on:
* no browser flow at all. standardFlowEnabled, implicitFlow and
directAccessGrants are off, because the agent never logs a user in —
it receives an already-issued token over A2A. Any of those three
would be a way to obtain a token it has no business obtaining.
* no service account. It has no standing access of its own; everything
it can reach, it reaches on behalf of a user who asked.
* two optional scopes, mcp-catalog-audience and mcp-sales-audience, and
nothing else. librechat carries all of them because it fronts every
domain; an agent serving the product manager must not be able to mint
a payments token.
Adds classic-models-agent-catalogue-audience, the scope whoever issues
the user's token requests so the agent appears in that token's audience.
Keycloak's Standard Token Exchange refuses otherwise, with "Client is not
within the token audience" — which reads as a permissions problem on the
target rather than a missing audience on the source.
Verified against the running realm: catalog and sales exchange
successfully and return aud=[mcp-catalog] / aud=[mcp-sales] with
azp=classic-models-agent-catalogue, while payments is refused with
invalid_scope. The confinement is enforced by Keycloak, not by the
agent's own restraint.
Note this file is a bootstrap seed, not the live realm: Keycloak starts
with --import-realm under IGNORE_EXISTING and logs "Realm
'classic-models' already exists. Import skipped", so editing it changes
nothing on a restart. The client and scope were created through the admin
API and recorded here so a fresh install matches. That divergence is
worth closing separately.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Point the OTel Java agent at Tempo instead of Jaeger, so Keycloak's traces
land in the same backend as the rest of the stack and correlate with the
Loki logs of the same requests.
Annotate the pod for scraping. KC_METRICS_ENABLED already served /metrics
on the management port, so this only makes Prometheus discover it — on
port 9000, never on the application port.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Stéphane Tailland