fix(llm): double the per-client token budget to 100k/min

A tool-calling agent re-sends the whole tool result on every iteration of its
loop, so a single question against a tool returning a full collection burns the
old budget in three turns and answers 429 in the middle of reasoning. Pagination
on those tools is the real fix; this buys room until then and stays well inside
the 500k global ceiling.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Stéphane Tailland
2026-08-29 07:57:53 +02:00
parent f9b73fda9f
commit 2d4debd12a
+7 -1
View File
@@ -40,7 +40,13 @@ spec:
VECTORDB_HOST: llm-dev-vectordb
VECTORDB_PORT: "6379"
RATE_LIMIT_GLOBAL_TOKENS_PER_MINUTE: "500000"
RATE_LIMIT_PER_CLIENT_TOKENS_PER_MINUTE: "50000"
# Doubled from 50000. A tool-calling agent re-sends the whole tool
# result on every iteration of its loop, so one question against a
# tool that returns a full collection (list-products is ~23 KB) can
# burn the old budget in three turns and answer 429 mid-reasoning.
# Pagination on those tools is the real fix; this buys room until
# then, and stays well inside the global ceiling.
RATE_LIMIT_PER_CLIENT_TOKENS_PER_MINUTE: "100000"
# Looked up after consumer.yaml's llm-anonymous KongConsumer first
# reconciled — config.anonymous needs the real Consumer ID, not the
# username (Konnect stores it as a literal string otherwise, silently