Capture request and response bodies on the AI gateway

The audit trail recorded metadata only for this gateway, so the prompts
sent to the LLM and the completions coming back — the one thing worth
auditing on an AI gateway — were never written down. The five business
domains have captured bodies since they were built; this gateway declares
its own copy of the observability plugins and never got them.

Adds the same pre-function/post-function pair the domains use, bound to
the proxy service, and the custom_fields_by_lua that reads them back in
the log phase. If the capture plugins are ever removed the fields go nil
and are silently omitted, so the log degrades to metadata rather than
breaking.

The response accumulator earns its keep here more than elsewhere: LLM
answers stream as SSE, so body_filter runs once per chunk and the
complete answer only exists after the last one.

Still dev-only, for the same reason as the rest of this file: it puts
prompt and completion content in the log stream.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Stéphane Tailland
2026-08-27 12:48:00 +02:00
parent 9893221da0
commit 11edfa8ed1
+77
View File
@@ -66,6 +66,12 @@ config:
http_endpoint: ${AUDIT_LOG_ENDPOINT}
method: POST
content_type: application/json
# Reads what the two capture plugins below stashed in kong.ctx.shared. Both
# are nil -- and silently omitted from the record -- if those plugins are not
# applied, so this degrades to metadata-only logging rather than breaking.
custom_fields_by_lua:
request_body: "return kong.ctx.shared.audit_request_body"
response_body: "return kong.ctx.shared.audit_response_body"
---
# prometheus and opentelemetry are bound control-plane-wide, matching the
# shared template: they describe the gateway itself, not one service.
@@ -116,3 +122,74 @@ spec:
group: configuration.konghq.com
kind: KongService
name: ${NAME}-llm-proxy
---
# dev only: captures the raw request/response bodies into kong.ctx.shared for
# the http-log plugin above to read back in the log phase. Same mechanism the
# five business domains use (<domain>-api/kong/mcp/audit-log-body-capture.yaml).
# Without it this gateway logged metadata only, so the prompts and completions
# going through the LLM were the one thing the audit trail did not record.
#
# The response accumulator matters more here than elsewhere: LLM answers stream
# as SSE, so body_filter runs once per chunk and the full completion only exists
# after the last one.
apiVersion: configuration.konghq.com/v1
kind: KongPlugin
metadata:
name: ${NAME}-llm-audit-capture-request
namespace: ${NAMESPACE}
plugin: pre-function
config:
access:
- "kong.ctx.shared.audit_request_body = kong.request.get_raw_body()"
---
apiVersion: configuration.konghq.com/v1
kind: KongPlugin
metadata:
name: ${NAME}-llm-audit-capture-response
namespace: ${NAMESPACE}
plugin: post-function
config:
body_filter:
- |
local chunk = ngx.arg[1]
if chunk then
kong.ctx.shared.audit_response_body = (kong.ctx.shared.audit_response_body or "") .. chunk
end
---
# Bound to the proxy service, like the audit log itself, so only real gateway
# traffic is captured.
apiVersion: configuration.konghq.com/v1alpha1
kind: KongPluginBinding
metadata:
name: ${NAME}-llm-audit-capture-request-binding
namespace: ${NAMESPACE}
spec:
controlPlaneRef:
type: konnectNamespacedRef
konnectNamespacedRef:
name: ${CONTROL_PLANE_REF_NAME}
pluginRef:
name: ${NAME}-llm-audit-capture-request
targets:
serviceRef:
group: configuration.konghq.com
kind: KongService
name: ${NAME}-llm-proxy
---
apiVersion: configuration.konghq.com/v1alpha1
kind: KongPluginBinding
metadata:
name: ${NAME}-llm-audit-capture-response-binding
namespace: ${NAMESPACE}
spec:
controlPlaneRef:
type: konnectNamespacedRef
konnectNamespacedRef:
name: ${CONTROL_PLANE_REF_NAME}
pluginRef:
name: ${NAME}-llm-audit-capture-response
targets:
serviceRef:
group: configuration.konghq.com
kind: KongService
name: ${NAME}-llm-proxy