admin/trailofbits-buttercup
1
0
Fork 0
mirror of https://github.com/trailofbits/buttercup synced 2026-06-21 14:11:39 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(common): use budgeted LITELLM_API_KEY so LITELLM_MAX_BUDGET is enforced (#550)

Browse Source 
* fix(common): use budgeted LITELLM_API_KEY for the LLM client

create_llm() authenticated with BUTTERCUP_LITELLM_KEY, which
litellm_config.yaml defines as general_settings.master_key. LiteLLM
master keys bypass all budget/rate enforcement, so LITELLM_MAX_BUDGET
had no effect — an e2e run with --budget 1 spent ~$4.77 unthrottled
with every /chat/completions returning 200.

The budgeted virtual key (llm-user, created with max_budget=
$LITELLM_MAX_BUDGET by litellm-user-keys-setup) is already plumbed:
seed-gen/patcher entrypoints export LITELLM_API_KEY from the mounted
key file. create_llm() just never read it.

Prefer LITELLM_API_KEY when set; fall back to BUTTERCUP_LITELLM_KEY so
environments without a provisioned budgeted key are unaffected.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(compose): document LITELLM_MAX_BUDGET in env.template (complements #550)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Riccardo Schirone
2026-05-18 15:52:14 +02:00
committed by GitHub
parent 4cc95c6659
commit ec7031c5ef
2 changed files with 15 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
common/src/buttercup/common/llm.py
+10 -2
View File
@@ -133,9 +133,17 @@ def create_default_llm_with_temperature(**kwargs: Any) -> Runnable:
def create_llm(**kwargs: Any) -> BaseChatModel:
"""Create an LLM object with the given configuration."""
"""Create an LLM object with the given configuration.
Prefer ``LITELLM_API_KEY`` (the per-deployment budgeted virtual key created
by litellm-user-keys-setup / the k8s key Job) over ``BUTTERCUP_LITELLM_KEY``.
``BUTTERCUP_LITELLM_KEY`` is the litellm *master* key, which bypasses all
budget/rate enforcement, so using it directly makes ``LITELLM_MAX_BUDGET``
ineffective. Fall back to it only when no budgeted key is provided.
"""
api_key = os.environ.get("LITELLM_API_KEY") or os.environ["BUTTERCUP_LITELLM_KEY"]
return ChatOpenAI(
openai_api_base=os.environ["BUTTERCUP_LITELLM_HOSTNAME"],
openai_api_key=SecretStr(os.environ["BUTTERCUP_LITELLM_KEY"]),
openai_api_key=SecretStr(api_key),
**kwargs,
)
dev/docker-compose/env.template
+5
View File
@@ -2,6 +2,11 @@
# LiteLLM variables
BUTTERCUP_LITELLM_KEY=sk-1234
# Max spend (USD) enforced on the generated llm-user LiteLLM key. Once the
# cumulative spend on that key exceeds this value, LiteLLM rejects further
# requests. Picked up by compose interpolation in compose.yaml; defaults to 100
# if unset. Kept low here to cap spend during local development.
LITELLM_MAX_BUDGET=15
# You only need to change variables for APIs you will use, but you need all these variables set
ANTHROPIC_API_KEY=<INSERT_KEY>
AZURE_API_BASE=<INSERT_HOST>