Turn on Kong's own AI telemetry instead of improvising it

Four settings, all of them off, which is why the AI gateway produced
HTTP-level telemetry and nothing about the model.

  logging.log_payloads     the prompt and the completion, in the log
                           records and — since OTel instrumentation is on
                           here — in the spans too
  logging.log_statistics   model usage and token counts
  prometheus.ai_metrics    kong_ai_llm_* : requests, tokens by type, cost,
                           provider latency, per model and per consumer
  input_cost / output_cost cost per 1M tokens, which is what makes
                           kong_ai_llm_cost_total mean anything

Also removes this morning's pre-function/post-function body capture from
this gateway. It was the wrong tool for the LLM route: Kong's own
implementation reassembles a streamed SSE answer, while accumulating raw
chunks in kong.ctx.shared does not. The five business domains keep theirs
— their MCP traffic is not streamed the same way.

The costs are a snapshot, checked against OpenRouter's price list for
google/gemini-3.7-flash on 2026-08-27: $0.375 in, $1.875 out per 1M
tokens. They are substitution variables rather than literals precisely
because a stale figure produces confidently wrong money rather than an
error.

Note what this does NOT fix: usage is missing on streamed responses
because the OpenAI-compatible API only reports it when the caller asks
for stream_options.include_usage, and that is LibreChat's request to
make, not Kong's. Expect token counts to stay partial until then.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Stéphane Tailland
2026-08-27 14:12:32 +02:00
parent 11edfa8ed1
commit 05cf2bcce1
3 changed files with 31 additions and 77 deletions
+7
View File
@@ -21,6 +21,13 @@ spec:
OIDC_ISSUER: https://keycloak.sttlab.eu/realms/sttlab
AGENT_AUDIENCE: '["llm-proxy"]'
OPENROUTER_UPSTREAM_URL: https://openrouter.ai/api/v1/chat/completions
# Cost per 1M tokens for the model behind OPENROUTER_UPSTREAM_URL, used by
# ai-proxy-advanced to compute kong_ai_llm_cost_total. Checked against
# OpenRouter's price list for google/gemini-3.7-flash on 2026-08-27
# ($0.375 in / $1.875 out). These are a snapshot, not a live rate: change
# the model or the contract and they silently become wrong money.
LLM_INPUT_COST: "0.375"
LLM_OUTPUT_COST: "1.875"
OPENROUTER_EMBEDDINGS_UPSTREAM_URL: https://openrouter.ai/api/v1/embeddings
VECTORDB_HOST: llm-dev-vectordb
VECTORDB_PORT: "6379"