IsaacTrost
311085c3c5
* changed configuration to focus on seed-gen * added nomminally working debug_subagent_task * debugging process * debugging process * debugging process * debugging process * got debug subagent to work more reliably * debugging why it cant find the seed file * added more tests * more debugging of debugginf functionality * more debugging of debugger * debugging working, scripts bad because some symbols are not found * mostly working live debugging, still a bit pricy though * Added hybrid mode that will do batch, then try interactive if that fails * fixes * fixes, updating docker interactive * updating mi parsing for gdb * modifications to the mi parser * Improve coverage tracking precision (#401) * Improve coverage tracking precision Previously coverage tracking included all regions. This changes coverage tracking to be more precise. Macros that result in code are represented as a single line in the function, and any executed line in the macro will make that line 'covered'. Beyound that, only CodeRegions will count. This will prevent code guarded by #if that aren't in the binary from being considered reachable for example. The idea is that seed-gen would have more accurate information when selecting functions to target. I've done some testing and it appears as if the new implementation in general reaches more lines and covers more functions. * Fix macro coverage leaking across files due to missing filename in key The expansion_coverage map used (line, col) as key without filename, causing coverage from one file to incorrectly appear in another when macros were at the same coordinates. * Recursively expand macros and add named types for coverage data - Process ExpansionRegion target_regions recursively instead of counting only the call site line - Add coordinate index for O(1) expansion lookups - Cache computed expansion lines to avoid recomputation across functions - Use bulk set operations for better performance - Prevent infinite loops with visited set for circular macro references - Add named types for coverage data structures: - RegionCoords, ExpansionKey, CachedExpansionLines - Type aliases: ExpansionMap, CoordToFilenames, ExpansionLinesCache - Add comprehensive tests for nested macros and edge cases * fix: fixing `line 110: set: -g: invalid option` for Fish shell (#408) Co-authored-by: kevin-valerio <kevin-valerio@users.noreply.github.com> * working state * working state for monolith, still needs refactor * limit grep output (whoops) and fix building to force optimization flags * added function lookup tool * added better build selection logic, and avoided c ode duplication * fix fuzzer selection to ignore debug * added debug builds as a seperate 'sanitizor' * fixed build system, more in line with other dependancies now * adding new better debug targets, and logging * final changes before testing * improving test coverage * improving test coverage * feat: add extract_povs command to buttercup-util (#410) Add new CLI subcommand to extract PoVs, stack traces, and patches from Redis submissions into a structured directory format for easy analysis. Features: - Extracts crash inputs (PoV files) via kubectl cp from cluster pods - Writes fuzzer and tracer stack traces to text files - Exports associated patches with metadata - Organizes output by project/task_id/vulnerability - Supports filtering by task_id and passed_only options - Skips empty patch trackers (placeholders that never received content) * Use git-lfs when downloading challenges * improving test coverage * Delete b.txt * final changes before run hopefully * feat: add extract_povs command to buttercup-util (#410) Add new CLI subcommand to extract PoVs, stack traces, and patches from Redis submissions into a structured directory format for easy analysis. Features: - Extracts crash inputs (PoV files) via kubectl cp from cluster pods - Writes fuzzer and tracer stack traces to text files - Exports associated patches with metadata - Organizes output by project/task_id/vulnerability - Supports filtering by task_id and passed_only options - Skips empty patch trackers (placeholders that never received content) * Sanitize exception messages for git clone command (#411) * Sanitize exception messages for git clone command * Update test case * Don't sanitize exception if there is no PAT * merging with main and fixing tests which tested old build system * fixing erronous changes from testing * fix silly linting errors * reformatted for linter * fixing failing tests * fixing failing tests * linting * restore scripts to remove debugging changes --------- Co-authored-by: Henrik Brodin <90325907+hbrodin@users.noreply.github.com> Co-authored-by: Kevin Valerio <24193167+kevin-valerio@users.noreply.github.com> Co-authored-by: kevin-valerio <kevin-valerio@users.noreply.github.com> Co-authored-by: Ronald Eytchison <58823072+reytchison@users.noreply.github.com>
Buttercup Cyber Reasoning System (CRS)
Buttercup is a Cyber Reasoning System (CRS) developed by Trail of Bits for the DARPA AIxCC (AI Cyber Challenge). Buttercup finds and patches software vulnerabilities in open-source code repositories like example-libpng. It starts by running an AI/ML-assisted fuzzing campaign (built on oss-fuzz) for the program. When vulnerabilities are found, Buttercup analyzes them and uses a multi-agent AI-driven patcher to repair the vulnerability. Buttercup system consists of several components:
- Orchestrator: Coordinates the overall task process and manages the workflow
- Seed Generator: Creates inputs for vulnerability discovery
- Fuzzer: Discovers vulnerabilities through intelligent fuzzing techniques
- Program Model: Analyzes code structure and semantics for better understanding
- Patcher: Generates and applies security patches to fix vulnerabilities
System Requirements
Minimum Requirements
- CPU: 8 cores
- Memory: 16 GB RAM
- Storage: 100 GB available disk space
- Network: Stable internet connection for downloading dependencies
Note: Buttercup uses third-party AI providers (LLMs from companies like OpenAI, Anthropic and Google), which cost money. Please ensure that you manage per-deployment costs by using the built-in LLM budget setting.
Note: Buttercup works best with access to models from OpenAI and Anthropic, but can be run with at least one API key from one third-party provider. You can use a combination of OpenAI, Anthropic, and Google LLMs.
Supported Systems
- Linux x86_64 (fully supported)
- ARM64 (partial support for upstream Google OSS-Fuzz projects)
Required System Packages
Before setup, ensure you have these packages installed:
# Ubuntu/Debian
sudo apt-get update
sudo apt-get install -y make curl git
# RHEL/CentOS/Fedora
sudo yum install -y make curl git
# or
sudo dnf install -y make curl git
# MacOS
brew install make curl git
Supported Targets
Buttercup works with:
- C source code repositories that are OSS-Fuzz compatible
- Java source code repositories that are OSS-Fuzz compatible
- Projects that build successfully and have existing fuzzing harnesses
Quick Start
- Clone the repository with submodules:
git clone --recurse-submodules https://github.com/trailofbits/buttercup.git
cd buttercup
- Run automated setup (Recommended)
make setup-local
This script will install all dependencies, configure the environment, and guide you through the setup process.
Note: If you prefer manual setup, see the Manual Setup Guide.
- Start Buttercup locally
make deploy
- Verify local deployment:
make status
When a deployment is successful, you should see all pods in "Running" or "Completed" status.
- Send Buttercup a simple task
Note: When tasked, Buttercup will start consuming third-party AI resources.
This command will make Buttercup pull down an example repo example-libpng with a known vulnerability. Buttercup will start fuzzing it to find and patch vulnerabilities.
make send-libpng-task
- Access Buttercup's web-based GUI
Run:
make web-ui
Then navigate to http://localhost:31323 in your web browser.
In the GUI you can monitor active tasks and see when Buttercup finds bugs and generates patches for them.
- Stop Buttercup
Note: This is an important step to ensure Buttercup shuts down and stops consuming third-party AI resources.
make undeploy
Accessing Logs
Buttercup includes local SigNoz deployment by default for comprehensive system observability. You can access logs, traces, and metrics through the SigNoz UI:
make signoz-ui
Then navigate to http://localhost:33301 in your web browser to view:
- Distributed traces
- Application metrics
- Error monitoring
- Performance insights
If you configured LangFuse during setup, you can also monitor LLM usage and costs there.
For additional log access methods, see the Quick Reference Guide.
Additional Resources
- Quick Reference Guide - Common commands and troubleshooting
- Manual Setup Guide - Detailed manual installation steps
- AKS Deployment Guide - Production deployment on Azure
- Contributing Guidelines - Development workflow and standards
- Deployment Documentation - Advanced deployment configuration
- Writing Custom Challenges - Custom project configuration and setup
- Unscored rounds - Running unscored round challenges
- Scored round - Parsing post-final round results