* feat(llm): add newer Anthropic and OpenAI models to LiteLLM
Add 9 model entries across the LiteLLM proxy config, k8s values, and
ButtercupLLM enum so callers can opt into newer models without code
changes: openai-o4-mini, openai-gpt-5/-mini/-nano, claude-4-opus,
claude-4.1-opus, claude-4.5-haiku, claude-4.6-sonnet, claude-4.7-opus.
Verified end-to-end with docker compose: LiteLLM v1.57.8 boots cleanly,
/v1/models lists all 27 entries, and chat completions for the new
model IDs are dispatched to api.anthropic.com / api.openai.com.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* feat(llm): narrow new model set to opus 4.6/4.7, sonnet 4.6, haiku 4.5 + gpt-5.4/5.4-mini/5.5
Replace the previously added openai-o4-mini and gpt-5/-mini/-nano entries
with gpt-5.4, gpt-5.4-mini, gpt-5.5; drop claude opus 4 and 4.1 in favor
of opus 4.6. Final new entries (7): claude-4.5-haiku, claude-4.6-sonnet,
claude-4.6-opus, claude-4.7-opus, openai-gpt-5.4-mini, openai-gpt-5.4,
openai-gpt-5.5.
Re-verified with docker compose: LiteLLM v1.57.8 boots, /v1/models lists
all 25 entries, and chat completions for claude-4.6-opus and
openai-gpt-5.5 are dispatched to api.anthropic.com / api.openai.com.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
macOS users using Colima instead of Docker Desktop were hitting
missing buildx errors and had no documentation to guide them.
Add Colima as a supported Docker runtime, document buildx
installation, and ensure setup-local installs buildx automatically.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Add optional tmpfs storage for fuzzing corpus
Add support for storing the node-local fuzzing corpus in tmpfs (RAM-based
filesystem) for improved I/O performance during fuzzing operations.
Key changes:
- Add CORPUS_TMPFS_PATH environment variable to configure tmpfs location
- Add cross-filesystem safe operations (copy+delete fallback for EXDEV)
- Update Corpus class to use tmpfs path when enabled while maintaining
correct remote path calculation for rsync synchronization
- Add Helm chart configuration using /dev/shm (requires no host setup)
- Update fuzzer-bot, coverage-bot, merger-bot, seed-gen deployments
The feature is disabled by default. Enable by setting
global.volumes.corpusTmpfs.enabled=true in values.yaml.
Uses hostPath to /dev/shm/buttercup-corpus which is available on
Linux systems without requiring any Kubernetes host configuration.
Apply automated fixes from shellcheck and shfmt to all shell scripts:
- Quote variables to prevent word splitting and globbing
- Use $(...) instead of backticks for command substitution
- Add proper shebang and set -euo pipefail where missing
- Fix array handling and iteration patterns
- Consistent indentation (4 spaces)
- Remove unnecessary curly braces and simplify expressions
Files fixed:
- deployment/*.sh
- scripts/*.sh
- orchestrator/scripts/*.sh
- fuzzer_runner/runner.sh
- protoc.sh
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add comprehensive documentation for local development deployment:
- Docker Desktop resource configuration (CPU, memory, disk)
- Common errors and solutions (resource insufficiency, pending pods, GHCR auth)
- Resource expectations table for different memory allocations
- macOS ARM64 (Apple Silicon) notes
- Verification steps and useful debugging commands
Also update env.template with inline comments explaining minikube resource
settings and their relationship to Docker Desktop resources.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Use project specified in oss-fuzz
This will use the contents of the oss-fuzz provided container as the
projects contents instead of a user supplied git url. This will allow to
run projects very similar to how oss-fuzz does it. It will enable
analysis of e.g. libmodbus who was previously not possible due to the
fuzzing harnesses being overwritten/shadowed by the mount of project
directory.
Use project_name for focusdir when in doubt
Before this commit, the fuzzer dependend on clusterfuzz, which uses
protobuf 3.20. Since fuzzer-bot depended on common subpackage as well
and also common (and the other packages) require protobuf, we had to use
protobuf 3.20 everywhere. This old dependency however means that a lot
of packages can't be used in their "newer" versions, because they depend
on newer protobuf versions.
This commit splits the fuzzer into a separate fuzzer-runner that is
executed in a separate process inside a separate venv. fuzzer-runner
executes the clusterfuzz-heavy operations (e.g. fuzzing) and isolates the
clusterfuzz dependency, so that the rest of the system can use newer
protobuf version.
* split `fuzzer-bot` in `fuzzer-runner` and `fuzzer-bot`
* have a "full" optional dependency group in `common`, including `openlit`
and `protobuf`, so that `fuzzer-runner` can use the lite version of `common`
without bringing those heavy deps
* move `FuzzConfiguration`/`BuildConfiguration` in a separate common file
that doesn't require to load all protobuf files. Again, in this way
other components can just depend on the `common` "lite" version and not
require protobuf stuff
* add `fuzzer-runner` as a separate venv inside the `fuzzer-bot` container
* add `RunnerProxy` class in fuzzer package to provide an interface to
interact with the fuzzer-runner binary.
* Test configuration for Azure deployment
* ci: re-enable docker build/push
* ci: docker push labeled PRs
* Use `make deploy` for both local and azure deployments
* doc: update AKS_DEPLOYMENT doc
* add confirmation in Makefile
* Make resource group location configurable
* Add gemini api key option during setup and in deployment environments
* Add gemini pro as a fallback model in all components
* Lint
* Set rate limits for gemini models
* Add fallback models in more places
* Fix typo
* Fix kwargs expansion
* Fix instantiation of llm with callbacks
* Fixed formatting after merge
* Fix instantiation of default models
* Lint
* Fix llm creation
* Fix
* Lint
* Lint
---------
Co-authored-by: Michael D Brown <michael.brown@trailofbits.com>
* fix: standardize packaging configuration across components
- Fix fuzzer package name from 'fuzzing-infra' to 'fuzzer' for consistency
- Add missing README field to common/pyproject.toml
- Standardize seed-gen script naming to follow buttercup-* convention
- Remove redundant flake8 dependency from orchestrator and patcher (ruff handles linting)
- Update lock files to reflect dependency changes
- Auto-format code in common and seed-gen components
* refactor: standardize dev dependencies across all components
- Add pytest-cov to all components for consistent test coverage reporting
- Standardize types-redis version to 4.6.0 across all components
- Standardize types-requests version to 2.32.0 where used
- Remove unused pytest-asyncio from orchestrator and patcher (no async tests found)
- Remove unused types-python-dateutil from patcher (only used in orchestrator)
- Keep component-specific dependencies where justified:
- orchestrator: pytest-xdist for parallel testing, API testing tools
- patcher: pytest-xdist for parallel testing
- common: types-PyYAML, dirty-equals for testing
- seed-gen: tqdm for progress bars
All components now have consistent base dev dependencies with optional extras
as needed. This improves maintainability and reduces confusion.
* fix: update deployment configs for renamed seed-gen command
- Update docker-compose to use buttercup-seed-gen command
- Update Kubernetes/Helm deployment to use buttercup-seed-gen command
- These are critical fixes to ensure deployments work with the renamed command
* feat: Add simplified SigNoz integration using official Helm chart
- Add SigNoz as a Helm dependency with conditional deployment
- Enable SigNoz by default for minikube environments
- Auto-configure OTEL to use internal SigNoz when enabled
- Update Docker Compose to include existing SigNoz stack
- Minimal configuration following the official chart approach
Co-authored-by: Riccardo Schirone <ret2libc@users.noreply.github.com>
* Configure local signoz
* do not deploy signoz in ci
* makefile: ensure signoz is available
* feat: Simplify SigNoz integration for improved user experience
- Remove external SigNoz configuration prompts from setup-local script
- Make local SigNoz deployment the default for quickstart experience
- Move external SigNoz configuration to MANUAL_SETUP.md for advanced users
- Streamline README log access section to focus on local SigNoz UI
- Relocate kubectl commands to QUICK_REFERENCE.md as alternative method
- Improve documentation structure for better user onboarding
Addresses feedback from @michaelbrownuc to simplify the integration
before merging.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-authored-by: Riccardo Schirone <ret2libc@users.noreply.github.com>
* Update MANUAL_SETUP.md
---------
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Riccardo Schirone <ret2libc@users.noreply.github.com>
* Scale down minikube specs and runner
* Scale down registry-cache and minikube again
* Use runner which can be used in a public repo
* Increase minikube size for services without resource requests
Also rename github action
* ci: try to reenable example-libpng integration test
* ci: try to use GITHUB_TOKEN
* ci: deploy the cluster
* ci: try private runner
* ci: no need for kubeconfig setting anymore
* Makefile: make all targets namespace flexible
* ci: use the right github token secret
* Makefile: make send-* targets return 0
* ci: use buttercup-ui and don't kill port-forward
* orchestrator/ui: add /webhook/sarif endpoint
* orchestrator/ui: model dump the broadcast
* ui: implement a few other competition-api endpoints
* ui: fix bundle apis
* feat: save CRS artifacts to run-data directories
Add functionality to automatically save POVs, patches, and bundles
received from the CRS into organized run-data-<timestamp> directories
structured by task-id as requested in issue #205.
Changes:
- Add run_data_dir configuration setting to Settings
- Implement save_artifact() utility function with proper base64 decoding
- Modify bundle, patch, and POV endpoints to save artifacts to disk
- Create directory structure: run-data-YYYYMMDDHHMMSS/<task-id>/{povs,patches,bundles}/
- Handle different file types: .json for bundles, .patch for patches, .bin for POVs
Co-authored-by: Riccardo Schirone <ret2libc@users.noreply.github.com>
* feat: add hostPath volume for buttercup-ui run data access
- Add run-data-volume hostPath mount to UI deployment for minikube
- Maps container /tmp/buttercup-run-data to host /data/buttercup-run-data
- Enables external access to CRS artifacts (POVs, patches, bundles)
- Falls back to emptyDir for non-minikube environments
Co-authored-by: Riccardo Schirone <ret2libc@users.noreply.github.com>
* fix: add BUTTERCUP_UI_RUN_DATA_DIR environment variable to UI deployment
- Set BUTTERCUP_UI_RUN_DATA_DIR to /tmp/buttercup-run-data to match volume mount
- Ensures UI application saves run data artifacts to the correct directory
- Run data will now be accessible outside Kubernetes cluster via hostPath volume
Co-authored-by: Riccardo Schirone <ret2libc@users.noreply.github.com>
* fix save-artifact feature
* orchestrator: fix lint
---------
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Riccardo Schirone <ret2libc@users.noreply.github.com>
Co-authored-by: Michael D Brown <michael.brown@trailofbits.com>
* Fix integration-test run
* use the proper oss-fuzz-aixcc commit to run integration-test
* determine OSS_FUZZ_CONTAINER_ORG at runtime
* use `git diff` in the patcher to create the patch to fix a problem
with patches affecting non-newline terminated files
* fix parsing
* fix program-model lint
* Modify CRS to work on MacOS/ARM
* Use LibUCL for testing
* use example-libpng
* fix linting
* common: fix tests
* Fix integration-test run
* use the proper oss-fuzz-aixcc commit to run integration-test
* determine OSS_FUZZ_CONTAINER_ORG at runtime
* use `git diff` in the patcher to create the patch to fix a problem
with patches affecting non-newline terminated files
* fix parsing
* fix program-model lint
* deployment: just use the value in values.template
* ci: disable integration tests and private settings
* Download trailofbits cscope, not aixcc-finals one
* ci: fix docker login to ghcr.io
* Changes to allow non aixcc deployment
* buttercup-ui: basic skeleton for a CRS interface
* file-server implementation
* add ui to k8s
* other apis
* try to fix k8s
* remove some labels
* fix ui
* small fixes to doc
* ui: support for cloning private repos
* Add setup scripts for easy deployment
* update scripts
* update make/readme
* small adj
* address review
* we need 0.0.0.0 for k8s
* deployment: just use the value in values.template
* ci: disable integration tests and private settings
* Download trailofbits cscope, not aixcc-finals one
* ci: fix docker login to ghcr.io
Riccardo Schirone