Turn on Kong's own AI telemetry instead of improvising it
Four settings, all of them off, which is why the AI gateway produced
HTTP-level telemetry and nothing about the model.
logging.log_payloads the prompt and the completion, in the log
records and — since OTel instrumentation is on
here — in the spans too
logging.log_statistics model usage and token counts
prometheus.ai_metrics kong_ai_llm_* : requests, tokens by type, cost,
provider latency, per model and per consumer
input_cost / output_cost cost per 1M tokens, which is what makes
kong_ai_llm_cost_total mean anything
Also removes this morning's pre-function/post-function body capture from
this gateway. It was the wrong tool for the LLM route: Kong's own
implementation reassembles a streamed SSE answer, while accumulating raw
chunks in kong.ctx.shared does not. The five business domains keep theirs
— their MCP traffic is not streamed the same way.
The costs are a snapshot, checked against OpenRouter's price list for
google/gemini-3.7-flash on 2026-08-27: $0.375 in, $1.875 out per 1M
tokens. They are substitution variables rather than literals precisely
because a stale figure produces confidently wrong money rather than an
error.
Note what this does NOT fix: usage is missing on streamed responses
because the OpenAI-compatible API only reports it when the caller asks
for stream_options.include_usage, and that is LibreChat's request to
make, not Kong's. Expect token counts to stay partial until then.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -10,9 +10,28 @@ config:
|
||||
auth:
|
||||
header_name: Authorization
|
||||
header_value: ${OPENROUTER_BEARER}
|
||||
# dev only, like the rest of this gateway's audit trail: log_payloads
|
||||
# puts the prompt and the completion in the log stream, and -- because
|
||||
# OpenTelemetry instrumentation is on here -- in the spans as well.
|
||||
#
|
||||
# This is what the pre-function/post-function capture in
|
||||
# observability.yaml stood in for on this route. Kong's own
|
||||
# implementation reassembles a streamed SSE answer; accumulating raw
|
||||
# chunks does not.
|
||||
logging:
|
||||
log_payloads: true
|
||||
# Model usage and token counts in the log records. With this off Kong
|
||||
# counted nothing of its own, which is half of why usage was missing.
|
||||
log_statistics: true
|
||||
model:
|
||||
provider: openai
|
||||
# name deliberately unset: the client's own `model` field is passed
|
||||
# through to OpenRouter.
|
||||
options:
|
||||
upstream_url: ${OPENROUTER_UPSTREAM_URL}
|
||||
# Cost per 1M tokens, in the provider's currency. Kong multiplies
|
||||
# these by the token counts to produce kong_ai_llm_cost_total, so a
|
||||
# stale figure yields confidently wrong money rather than an error.
|
||||
# Re-check them whenever the model or the contract changes.
|
||||
input_cost: ${LLM_INPUT_COST}
|
||||
output_cost: ${LLM_OUTPUT_COST}
|
||||
|
||||
Reference in New Issue
Block a user