Henrik Brodin
a3d805cb35
* ci: shell-quote env values for system integration tests Two bugs in the env file step prevented the weekly System Integration Tests from ever passing since #427 (2026-01-26): 1. The deletion regex `/^KEY=/` did not match the template's `export KEY=replace-me` lines, leaving stale `replace-me` placeholders in the file alongside the appended real values. 2. Appended values were not shell-quoted, so `source ./env` in deployment/crs-architecture.sh treated whitespace and metacharacters as shell syntax. The `OTEL_TOKEN` secret (a `Bearer <token>` header) tripped this with `command not found` on the token component. The same gap was a code-execution sink: a secret containing `$(...)` or backticks would execute on the runner at source time. Fix: strip with `(export[[:space:]]+)?` so template lines are actually removed, and write with `printf 'export %s=%q\n'` so every value is shell-escaped — defending against both whitespace and injection. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * ci: drop placeholder block from ci-env.template The placeholder lines (`export KEY=replace-me`) only existed because the pre-rewrite step substituted into them with `sed -i "s|KEY=.*|...|"`. The new `printf %q` write path doesn't need them, so removing them at the source eliminates the entire delete-then-rewrite dance. Drops `strip_var` and the for-loop. The workflow step is now: copy the static template, then append shell-quoted assignments for every secret the env: block injects. GHCR_AUTH is no longer defined at all, so the `[ -n "$GHCR_AUTH" ]` warning branch in crs-architecture.sh:63 fires as intended. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Buttercup Cyber Reasoning System (CRS)
Buttercup is a Cyber Reasoning System (CRS) developed by Trail of Bits for the DARPA AIxCC (AI Cyber Challenge). Buttercup finds and patches software vulnerabilities in open-source code repositories like example-libpng. It starts by running an AI/ML-assisted fuzzing campaign (built on oss-fuzz) for the program. When vulnerabilities are found, Buttercup analyzes them and uses a multi-agent AI-driven patcher to repair the vulnerability. Buttercup system consists of several components:
- Orchestrator: Coordinates the overall task process and manages the workflow
- Seed Generator: Creates inputs for vulnerability discovery
- Fuzzer: Discovers vulnerabilities through intelligent fuzzing techniques
- Program Model: Analyzes code structure and semantics for better understanding
- Patcher: Generates and applies security patches to fix vulnerabilities
System Requirements
Minimum Requirements
- CPU: 8 cores
- Memory: 16 GB RAM
- Storage: 100 GB available disk space
- Network: Stable internet connection for downloading dependencies
Note: Buttercup uses third-party AI providers (LLMs from companies like OpenAI, Anthropic and Google), which cost money. Please ensure that you manage per-deployment costs by using the built-in LLM budget setting.
Note: Buttercup works best with access to models from OpenAI and Anthropic, but can be run with at least one API key from one third-party provider. You can use a combination of OpenAI, Anthropic, and Google LLMs.
Supported Systems
- Linux x86_64 (fully supported)
- macOS (local development supported via Docker Desktop or Colima)
- ARM64 (partial support for upstream Google OSS-Fuzz projects)
Required System Packages
Before setup, ensure you have these packages installed:
# Ubuntu/Debian
sudo apt-get update
sudo apt-get install -y make curl git
# RHEL/CentOS/Fedora
sudo yum install -y make curl git
# or
sudo dnf install -y make curl git
# MacOS
brew install make curl git
Supported Targets
Buttercup works with:
- C source code repositories that are OSS-Fuzz compatible
- Java source code repositories that are OSS-Fuzz compatible
- Projects that build successfully and have existing fuzzing harnesses
Quick Start
- Clone the repository with submodules:
git clone --recurse-submodules https://github.com/trailofbits/buttercup.git
cd buttercup
- Run automated setup (Recommended)
make setup-local
This script will install all dependencies, configure the environment, and guide you through the setup process.
Note: If you prefer manual setup, see the Manual Setup Guide.
- Ensure Docker is running before deploying:
# Docker Desktop: start the application
# Colima (macOS/Linux alternative):
colima start --cpu 6 --memory 10 --disk 80
- Start Buttercup locally
make deploy
- Verify local deployment:
make status
When a deployment is successful, you should see all pods in "Running" or "Completed" status.
- Send Buttercup a simple task
Note: When tasked, Buttercup will start consuming third-party AI resources.
This command will make Buttercup pull down an example repo example-libpng with a known vulnerability. Buttercup will start fuzzing it to find and patch vulnerabilities.
make send-libpng-task
- Access Buttercup's web-based GUI
Run:
make web-ui
Then navigate to http://localhost:31323 in your web browser.
In the GUI you can monitor active tasks and see when Buttercup finds bugs and generates patches for them.
- Stop Buttercup
Note: This is an important step to ensure Buttercup shuts down and stops consuming third-party AI resources.
make undeploy
Accessing Logs
Buttercup includes local SigNoz deployment by default for comprehensive system observability. You can access logs, traces, and metrics through the SigNoz UI:
make signoz-ui
Then navigate to http://localhost:33301 in your web browser to view:
- Distributed traces
- Application metrics
- Error monitoring
- Performance insights
If you configured LangFuse during setup, you can also monitor LLM usage and costs there.
For additional log access methods, see the Quick Reference Guide.
Additional Resources
- Quick Reference Guide - Common commands and troubleshooting
- Manual Setup Guide - Detailed manual installation steps
- AKS Deployment Guide - Production deployment on Azure
- Contributing Guidelines - Development workflow and standards
- Deployment Documentation - Advanced deployment configuration
- Writing Custom Challenges - Custom project configuration and setup
- Unscored rounds - Running unscored round challenges
- Scored round - Parsing post-final round results