BUTTERCUP_PATCHER_TASK_STORAGE_DIR was /node_data/crs_scratch. The
patcher's get_clean_task() resolves <task_storage>/<task_id>; under
crs_scratch that path is already the fuzzer/scheduler per-task scratch
dir (corpus/crashes/build subdirs, no task_meta.json). remote_archive_
to_dir() early-returns because the dir exists, so the pristine task is
never materialized and TaskMeta.load() raises FileNotFoundError on
task_meta.json during the patcher 'find_tests' node.
Point it at /node_data/tasks_storage (same as
BUTTERCUP_DOWNLOADER_DOWNLOAD_DIR / BUTTERCUP_SCHEDULER_TASKS_STORAGE_DIR),
where the downloaded task with task_meta.json actually lives.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds an overlay for dev/docker-compose that resets the `build:` block
of every locally-built component and points it at the matching prebuilt
image from GHCR, so developers can run the stack without local builds.
Image tag defaults to "main" and is overridable via BUTTERCUP_IMAGE_TAG.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* ci: allow Docker Compose !reset/!override tags in YAML check
The static-checks YAML step uses yaml.safe_load_all, which rejects the
Compose-specific !reset/!override merge tags used by
compose.prebuilt.yaml. Register no-op constructors for those tags so
compose overlays validate without masking real YAML errors elsewhere.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The scheduler/task-server competition API URL is
http://buttercup-ui:31323, but the UI listened on the container-internal
port 1323 (31323 was only the host-published mapping). Inside the
compose network, scheduler -> http://buttercup-ui:31323 was refused,
so POV/patch/bundle submissions failed with "Failed to submit POV"
even though the host-side e2e.sh task POST (localhost:31323) worked.
Set BUTTERCUP_UI_PORT=31323 and map 31323:31323 so host, internal,
and the *_COMPETITION_API_URL env all use one consistent port.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(common): use budgeted LITELLM_API_KEY for the LLM client
create_llm() authenticated with BUTTERCUP_LITELLM_KEY, which
litellm_config.yaml defines as general_settings.master_key. LiteLLM
master keys bypass all budget/rate enforcement, so LITELLM_MAX_BUDGET
had no effect — an e2e run with --budget 1 spent ~$4.77 unthrottled
with every /chat/completions returning 200.
The budgeted virtual key (llm-user, created with max_budget=
$LITELLM_MAX_BUDGET by litellm-user-keys-setup) is already plumbed:
seed-gen/patcher entrypoints export LITELLM_API_KEY from the mounted
key file. create_llm() just never read it.
Prefer LITELLM_API_KEY when set; fall back to BUTTERCUP_LITELLM_KEY so
environments without a provisioned budgeted key are unaffected.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore(compose): document LITELLM_MAX_BUDGET in env.template (complements #550)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Before this commit, the fuzzer dependend on clusterfuzz, which uses
protobuf 3.20. Since fuzzer-bot depended on common subpackage as well
and also common (and the other packages) require protobuf, we had to use
protobuf 3.20 everywhere. This old dependency however means that a lot
of packages can't be used in their "newer" versions, because they depend
on newer protobuf versions.
This commit splits the fuzzer into a separate fuzzer-runner that is
executed in a separate process inside a separate venv. fuzzer-runner
executes the clusterfuzz-heavy operations (e.g. fuzzing) and isolates the
clusterfuzz dependency, so that the rest of the system can use newer
protobuf version.
* split `fuzzer-bot` in `fuzzer-runner` and `fuzzer-bot`
* have a "full" optional dependency group in `common`, including `openlit`
and `protobuf`, so that `fuzzer-runner` can use the lite version of `common`
without bringing those heavy deps
* move `FuzzConfiguration`/`BuildConfiguration` in a separate common file
that doesn't require to load all protobuf files. Again, in this way
other components can just depend on the `common` "lite" version and not
require protobuf stuff
* add `fuzzer-runner` as a separate venv inside the `fuzzer-bot` container
* add `RunnerProxy` class in fuzzer package to provide an interface to
interact with the fuzzer-runner binary.
* Test configuration for Azure deployment
* ci: re-enable docker build/push
* ci: docker push labeled PRs
* Use `make deploy` for both local and azure deployments
* doc: update AKS_DEPLOYMENT doc
* add confirmation in Makefile
* Make resource group location configurable
* fix: standardize packaging configuration across components
- Fix fuzzer package name from 'fuzzing-infra' to 'fuzzer' for consistency
- Add missing README field to common/pyproject.toml
- Standardize seed-gen script naming to follow buttercup-* convention
- Remove redundant flake8 dependency from orchestrator and patcher (ruff handles linting)
- Update lock files to reflect dependency changes
- Auto-format code in common and seed-gen components
* refactor: standardize dev dependencies across all components
- Add pytest-cov to all components for consistent test coverage reporting
- Standardize types-redis version to 4.6.0 across all components
- Standardize types-requests version to 2.32.0 where used
- Remove unused pytest-asyncio from orchestrator and patcher (no async tests found)
- Remove unused types-python-dateutil from patcher (only used in orchestrator)
- Keep component-specific dependencies where justified:
- orchestrator: pytest-xdist for parallel testing, API testing tools
- patcher: pytest-xdist for parallel testing
- common: types-PyYAML, dirty-equals for testing
- seed-gen: tqdm for progress bars
All components now have consistent base dev dependencies with optional extras
as needed. This improves maintainability and reduces confusion.
* fix: update deployment configs for renamed seed-gen command
- Update docker-compose to use buttercup-seed-gen command
- Update Kubernetes/Helm deployment to use buttercup-seed-gen command
- These are critical fixes to ensure deployments work with the renamed command
* feat: Add simplified SigNoz integration using official Helm chart
- Add SigNoz as a Helm dependency with conditional deployment
- Enable SigNoz by default for minikube environments
- Auto-configure OTEL to use internal SigNoz when enabled
- Update Docker Compose to include existing SigNoz stack
- Minimal configuration following the official chart approach
Co-authored-by: Riccardo Schirone <ret2libc@users.noreply.github.com>
* Configure local signoz
* do not deploy signoz in ci
* makefile: ensure signoz is available
* feat: Simplify SigNoz integration for improved user experience
- Remove external SigNoz configuration prompts from setup-local script
- Make local SigNoz deployment the default for quickstart experience
- Move external SigNoz configuration to MANUAL_SETUP.md for advanced users
- Streamline README log access section to focus on local SigNoz UI
- Relocate kubectl commands to QUICK_REFERENCE.md as alternative method
- Improve documentation structure for better user onboarding
Addresses feedback from @michaelbrownuc to simplify the integration
before merging.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-authored-by: Riccardo Schirone <ret2libc@users.noreply.github.com>
* Update MANUAL_SETUP.md
---------
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Riccardo Schirone <ret2libc@users.noreply.github.com>
* Move docker compose files to dev/docker-compose/ directory
- Moved compose.yaml, env.template, and env.dev.compose from root to dev/docker-compose/
- Updated all relative path references in compose.yaml to point back to root directory
- Removed docker compose section from README.md
- Added README.md in dev/docker-compose/ with usage instructions
- Docker compose remains available for developers but is no longer prominently featured
Addresses #181🤖 Generated with [Claude Code](https://claude.ai/code)
Co-authored-by: Riccardo Schirone <ret2libc@users.noreply.github.com>
* Move env.dev.compose and env.template to dev/docker-compose/ directory
- Restored env.dev.compose and env.template from git history
- Moved both files to dev/docker-compose/ directory alongside compose.yaml
- Files were previously deleted but should have been moved with other compose files
- All references in compose.yaml and README.md are already correct
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-authored-by: Riccardo Schirone <ret2libc@users.noreply.github.com>
* fix compose
---------
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Riccardo Schirone <ret2libc@users.noreply.github.com>
Riccardo Schirone