Let the AI gateway read the bodies it is meant to log
Two ceilings had to clear 30 KB prompts, and neither alone was enough: nginx's client_body_buffer_size, which decides whether the body stays in memory at all, and ai-proxy's max_request_body_size, which decides whether the plugin will introspect it. With the first at 8k the body went to a temp file and kong.request.get_raw_body() returned nil, so log_payloads and log_statistics were enabled and empty, and no kong_ai_llm_* metric ever appeared. 256k here against the 8k the domain gateways keep. Affordable because this gateway is low-traffic; the buffer is per connection carrying a body, so copying the value to the domain gateways needs that arithmetic redone for their volume. Storage cost measured rather than assumed: 18 LLM requests a day at roughly 60 KB of prompt and completion is 1.1 MiB/day, against 425 MiB/day of Kong audit records already flowing. My earlier warning that the audit trail would grow fast was wrong at this volume. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -5,6 +5,10 @@ metadata:
|
||||
namespace: ${NAMESPACE}
|
||||
plugin: ai-proxy-advanced
|
||||
config:
|
||||
# The plugin's own ceiling on what it will introspect, distinct from nginx's
|
||||
# client_body_buffer_size — both have to clear 30 KB prompts or the payload
|
||||
# and usage logging stay empty. Raised together; neither alone is enough.
|
||||
max_request_body_size: 262144
|
||||
targets:
|
||||
- route_type: llm/v1/chat
|
||||
auth:
|
||||
|
||||
Reference in New Issue
Block a user