Stéphane Tailland
86d56dc4ea
ai-proxy's log_payloads and log_statistics produced nothing on the AI
gateway despite being enabled. The cause was not the plugin's own
max_request_body_size but nginx's client_body_buffer_size, left at its 8k
default: past it the body goes to a temp file, and
kong.request.get_raw_body() returns nil for a body on disk, so any plugin
reading the body silently stops working. Prompts on that gateway run to 30
KB. Measured, not guessed — request.size was 30369 on the last call.
Expose it as ${CLIENT_BODY_BUFFER_SIZE} with the 8k default every domain
keeps, rather than raising it globally. The buffer is allocated per
connection carrying a body, so a value that is free on a gateway seeing 18
requests a day is not free on one serving real traffic.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
57 lines
2.2 KiB
YAML
57 lines
2.2 KiB
YAML
apiVersion: gateway-operator.konghq.com/v2beta1
|
|
kind: GatewayConfiguration
|
|
metadata:
|
|
name: ${NAME}
|
|
namespace: ${NAMESPACE}
|
|
spec:
|
|
dataPlaneOptions:
|
|
deployment:
|
|
replicas: ${REPLICAS}
|
|
podTemplateSpec:
|
|
spec:
|
|
containers:
|
|
- name: proxy
|
|
image: ${GATEWAY_IMAGE}
|
|
env:
|
|
- name: KONG_TRACING_INSTRUMENTATIONS
|
|
value: "all"
|
|
- name: KONG_TRACING_SAMPLING_RATE
|
|
value: "1.0"
|
|
- name: KONG_OPENTELEMETRY_TRACING
|
|
value: "on"
|
|
- name: KONG_NGINX_PROXY_PROXY_BUFFER_SIZE
|
|
value: "128k"
|
|
- name: KONG_NGINX_PROXY_PROXY_BUFFERS
|
|
value: "8 128k"
|
|
- name: KONG_NGINX_PROXY_PROXY_BUSY_BUFFERS_SIZE
|
|
value: "256k"
|
|
- name: KONG_NGINX_PROXY_LARGE_CLIENT_HEADER_BUFFERS
|
|
value: "4 32k"
|
|
- name: KONG_NGINX_HTTP_PROXY_BUFFER_SIZE
|
|
value: "128k"
|
|
- name: KONG_NGINX_HTTP_PROXY_BUFFERS
|
|
value: "8 128k"
|
|
- name: KONG_NGINX_HTTP_PROXY_BUSY_BUFFERS_SIZE
|
|
value: "256k"
|
|
- name: KONG_NGINX_HTTP_LARGE_CLIENT_HEADER_BUFFERS
|
|
value: "4 32k"
|
|
# How much of a request body nginx keeps in memory. Past it the body
|
|
# goes to a temp file, and kong.request.get_raw_body() returns nil for
|
|
# a body on disk — so any plugin that reads the body silently stops
|
|
# working. That is what kept ai-proxy's log_payloads and log_statistics
|
|
# empty on the AI gateway, whose prompts run to 30 KB against this 8k
|
|
# default.
|
|
#
|
|
# Per domain, not global: the buffer is allocated per connection carrying
|
|
# a body, so a value that is free on a gateway seeing 18 requests a day
|
|
# is not free on one serving real traffic.
|
|
- name: KONG_NGINX_HTTP_CLIENT_BODY_BUFFER_SIZE
|
|
value: "${CLIENT_BODY_BUFFER_SIZE}"
|
|
konnect:
|
|
authRef:
|
|
name: ${NAME}
|
|
source: Mirror
|
|
mirror:
|
|
konnect:
|
|
id: ${CONTROL_PLANE_ID}
|