Henrik Brodin
58c52cf6e2
* Improve coverage tracking precision Previously coverage tracking included all regions. This changes coverage tracking to be more precise. Macros that result in code are represented as a single line in the function, and any executed line in the macro will make that line 'covered'. Beyound that, only CodeRegions will count. This will prevent code guarded by #if that aren't in the binary from being considered reachable for example. The idea is that seed-gen would have more accurate information when selecting functions to target. I've done some testing and it appears as if the new implementation in general reaches more lines and covers more functions. * Fix macro coverage leaking across files due to missing filename in key The expansion_coverage map used (line, col) as key without filename, causing coverage from one file to incorrectly appear in another when macros were at the same coordinates. * Recursively expand macros and add named types for coverage data - Process ExpansionRegion target_regions recursively instead of counting only the call site line - Add coordinate index for O(1) expansion lookups - Cache computed expansion lines to avoid recomputation across functions - Use bulk set operations for better performance - Prevent infinite loops with visited set for circular macro references - Add named types for coverage data structures: - RegionCoords, ExpansionKey, CachedExpansionLines - Type aliases: ExpansionMap, CoordToFilenames, ExpansionLinesCache - Add comprehensive tests for nested macros and edge cases
Buttercup Cyber Reasoning System (CRS)
Buttercup is a Cyber Reasoning System (CRS) developed by Trail of Bits for the DARPA AIxCC (AI Cyber Challenge). Buttercup finds and patches software vulnerabilities in open-source code repositories like example-libpng. It starts by running an AI/ML-assisted fuzzing campaign (built on oss-fuzz) for the program. When vulnerabilities are found, Buttercup analyzes them and uses a multi-agent AI-driven patcher to repair the vulnerability. Buttercup system consists of several components:
- Orchestrator: Coordinates the overall task process and manages the workflow
- Seed Generator: Creates inputs for vulnerability discovery
- Fuzzer: Discovers vulnerabilities through intelligent fuzzing techniques
- Program Model: Analyzes code structure and semantics for better understanding
- Patcher: Generates and applies security patches to fix vulnerabilities
System Requirements
Minimum Requirements
- CPU: 8 cores
- Memory: 16 GB RAM
- Storage: 100 GB available disk space
- Network: Stable internet connection for downloading dependencies
Note: Buttercup uses third-party AI providers (LLMs from companies like OpenAI, Anthropic and Google), which cost money. Please ensure that you manage per-deployment costs by using the built-in LLM budget setting.
Note: Buttercup works best with access to models from OpenAI and Anthropic, but can be run with at least one API key from one third-party provider. You can use a combination of OpenAI, Anthropic, and Google LLMs.
Supported Systems
- Linux x86_64 (fully supported)
- ARM64 (partial support for upstream Google OSS-Fuzz projects)
Required System Packages
Before setup, ensure you have these packages installed:
# Ubuntu/Debian
sudo apt-get update
sudo apt-get install -y make curl git
# RHEL/CentOS/Fedora
sudo yum install -y make curl git
# or
sudo dnf install -y make curl git
# MacOS
brew install make curl git
Supported Targets
Buttercup works with:
- C source code repositories that are OSS-Fuzz compatible
- Java source code repositories that are OSS-Fuzz compatible
- Projects that build successfully and have existing fuzzing harnesses
Quick Start
- Clone the repository with submodules:
git clone --recurse-submodules https://github.com/trailofbits/buttercup.git
cd buttercup
- Run automated setup (Recommended)
make setup-local
This script will install all dependencies, configure the environment, and guide you through the setup process.
Note: If you prefer manual setup, see the Manual Setup Guide.
- Start Buttercup locally
make deploy
- Verify local deployment:
make status
When a deployment is successful, you should see all pods in "Running" or "Completed" status.
- Send Buttercup a simple task
Note: When tasked, Buttercup will start consuming third-party AI resources.
This command will make Buttercup pull down an example repo example-libpng with a known vulnerability. Buttercup will start fuzzing it to find and patch vulnerabilities.
make send-libpng-task
- Access Buttercup's web-based GUI
Run:
make web-ui
Then navigate to http://localhost:31323 in your web browser.
In the GUI you can monitor active tasks and see when Buttercup finds bugs and generates patches for them.
- Stop Buttercup
Note: This is an important step to ensure Buttercup shuts down and stops consuming third-party AI resources.
make undeploy
Accessing Logs
Buttercup includes local SigNoz deployment by default for comprehensive system observability. You can access logs, traces, and metrics through the SigNoz UI:
make signoz-ui
Then navigate to http://localhost:33301 in your web browser to view:
- Distributed traces
- Application metrics
- Error monitoring
- Performance insights
If you configured LangFuse during setup, you can also monitor LLM usage and costs there.
For additional log access methods, see the Quick Reference Guide.
Additional Resources
- Quick Reference Guide - Common commands and troubleshooting
- Manual Setup Guide - Detailed manual installation steps
- AKS Deployment Guide - Production deployment on Azure
- Contributing Guidelines - Development workflow and standards
- Deployment Documentation - Advanced deployment configuration
- Writing Custom Challenges - Custom project configuration and setup
- Unscored rounds - Running unscored round challenges
- Scored round - Parsing post-final round results