mirror of
https://github.com/trailofbits/buttercup
synced 2026-06-21 14:11:39 +00:00
IsaacTrost
311085c3c5
* changed configuration to focus on seed-gen * added nomminally working debug_subagent_task * debugging process * debugging process * debugging process * debugging process * got debug subagent to work more reliably * debugging why it cant find the seed file * added more tests * more debugging of debugginf functionality * more debugging of debugger * debugging working, scripts bad because some symbols are not found * mostly working live debugging, still a bit pricy though * Added hybrid mode that will do batch, then try interactive if that fails * fixes * fixes, updating docker interactive * updating mi parsing for gdb * modifications to the mi parser * Improve coverage tracking precision (#401) * Improve coverage tracking precision Previously coverage tracking included all regions. This changes coverage tracking to be more precise. Macros that result in code are represented as a single line in the function, and any executed line in the macro will make that line 'covered'. Beyound that, only CodeRegions will count. This will prevent code guarded by #if that aren't in the binary from being considered reachable for example. The idea is that seed-gen would have more accurate information when selecting functions to target. I've done some testing and it appears as if the new implementation in general reaches more lines and covers more functions. * Fix macro coverage leaking across files due to missing filename in key The expansion_coverage map used (line, col) as key without filename, causing coverage from one file to incorrectly appear in another when macros were at the same coordinates. * Recursively expand macros and add named types for coverage data - Process ExpansionRegion target_regions recursively instead of counting only the call site line - Add coordinate index for O(1) expansion lookups - Cache computed expansion lines to avoid recomputation across functions - Use bulk set operations for better performance - Prevent infinite loops with visited set for circular macro references - Add named types for coverage data structures: - RegionCoords, ExpansionKey, CachedExpansionLines - Type aliases: ExpansionMap, CoordToFilenames, ExpansionLinesCache - Add comprehensive tests for nested macros and edge cases * fix: fixing `line 110: set: -g: invalid option` for Fish shell (#408) Co-authored-by: kevin-valerio <kevin-valerio@users.noreply.github.com> * working state * working state for monolith, still needs refactor * limit grep output (whoops) and fix building to force optimization flags * added function lookup tool * added better build selection logic, and avoided c ode duplication * fix fuzzer selection to ignore debug * added debug builds as a seperate 'sanitizor' * fixed build system, more in line with other dependancies now * adding new better debug targets, and logging * final changes before testing * improving test coverage * improving test coverage * feat: add extract_povs command to buttercup-util (#410) Add new CLI subcommand to extract PoVs, stack traces, and patches from Redis submissions into a structured directory format for easy analysis. Features: - Extracts crash inputs (PoV files) via kubectl cp from cluster pods - Writes fuzzer and tracer stack traces to text files - Exports associated patches with metadata - Organizes output by project/task_id/vulnerability - Supports filtering by task_id and passed_only options - Skips empty patch trackers (placeholders that never received content) * Use git-lfs when downloading challenges * improving test coverage * Delete b.txt * final changes before run hopefully * feat: add extract_povs command to buttercup-util (#410) Add new CLI subcommand to extract PoVs, stack traces, and patches from Redis submissions into a structured directory format for easy analysis. Features: - Extracts crash inputs (PoV files) via kubectl cp from cluster pods - Writes fuzzer and tracer stack traces to text files - Exports associated patches with metadata - Organizes output by project/task_id/vulnerability - Supports filtering by task_id and passed_only options - Skips empty patch trackers (placeholders that never received content) * Sanitize exception messages for git clone command (#411) * Sanitize exception messages for git clone command * Update test case * Don't sanitize exception if there is no PAT * merging with main and fixing tests which tested old build system * fixing erronous changes from testing * fix silly linting errors * reformatted for linter * fixing failing tests * fixing failing tests * linting * restore scripts to remove debugging changes --------- Co-authored-by: Henrik Brodin <90325907+hbrodin@users.noreply.github.com> Co-authored-by: Kevin Valerio <24193167+kevin-valerio@users.noreply.github.com> Co-authored-by: kevin-valerio <kevin-valerio@users.noreply.github.com> Co-authored-by: Ronald Eytchison <58823072+reytchison@users.noreply.github.com>
Buttercup Common utilities
Protobufs
The protos directory contains the protobuf definitions for various messages used by the Buttercup system.
Buttercup Util utility
The buttercup-util script is a utility for interacting with various components of the Buttercup CRS.
$ buttercup-util --help
usage: buttercup-util [-h] [--redis_url str] [--log_level str] {send_queue,read_queue,list_queues,delete_queue,add_harness,add_build,read_harnesses,read_builds} ...
options:
-h, --help show this help message and exit
--redis_url str Redis URL (default: redis://localhost:6379)
--log_level str Log level (default: info)
subcommands:
{send_queue,read_queue,list_queues,delete_queue,add_harness,add_build,read_harnesses,read_builds}
send_queue
read_queue
list_queues
delete_queue
add_harness
add_build
read_harnesses
read_builds
Send messages to a specific queue
$ buttercup-util send_queue orchestrator_download_tasks_queue ./examples/task_download.txt
2025-03-12 10:49:46,342 - buttercup.common.util_cli - INFO - Reading TaskDownload message from file 'examples/task_download.txt'
2025-03-12 10:49:46,342 - buttercup.common.util_cli - INFO - Pushing message to queue 'orchestrator_download_tasks_queue': task {
message_time: 1739917788000
task_id: "my-task-id"
task_type: TASK_TYPE_DELTA
sources {
sha256: "c516e2b73f58fe163be48f5bc0ca36995ee100c752e30883f9acaa0a95ca2bb6"
url: "https://challengesact.blob.core.windows.net/challenges/c516e2b73f58fe163be48f5bc0ca36995ee100c752e30883f9acaa0a95ca2bb6.tar.gz?se=2025-08-18T22%3A29%3A44Z&sp=r&sv=2022-11-02&sr=b&sig=7lj49Z6vXsFuKp4DqVrVVMwHU4xEAQJ%2BSCZ7BAQnbvY%3D"
}
sources {
sha256: "910913fd13eb2e7cb7ca9a39fce4cc753d54579c938a5c60d478788101fdde3e"
source_type: SOURCE_TYPE_FUZZ_TOOLING
url: "https://challengesact.blob.core.windows.net/challenges/910913fd13eb2e7cb7ca9a39fce4cc753d54579c938a5c60d478788101fdde3e.tar.gz?se=2025-08-18T22%3A29%3A47Z&sp=r&sv=2022-11-02&sr=b&sig=M6JoI0pGccbSARTqVLm23yQZUbUwsQsFyBpRMoADnYc%3D"
}
sources {
sha256: "04ffd1402d868846d6812112c4bc2ec50722aa1adfaf02aab7233ad20bd1b495"
source_type: SOURCE_TYPE_DIFF
url: "https://challengesact.blob.core.windows.net/challenges/04ffd1402d868846d6812112c4bc2ec50722aa1adfaf02aab7233ad20bd1b495.tar.gz?se=2025-08-18T22%3A29%3A42Z&sp=r&sv=2022-11-02&sr=b&sig=M63mfyTls1CJhxelj%2BdtGmmO9fIVimybM6yqOMCkRac%3D"
}
deadline: 1739932188000
project_name: "libpng"
focus: "example-libpng"
}
Read an entire queue
$ buttercup-util read_queue orchestrator_download_tasks_queue
[...]
Or, if you want to simulate a consumer in a consumer group popping an element out of the queue:
$ buttercup-util read_queue orchestrator_download_tasks_queue --group_name orchestrator_group
Add an harness to the fuzzer map
$ buttercup-util add_harness ./examples/weighted_harness.txt
2025-03-12 10:53:10,256 - buttercup.common.util_cli - INFO - Added harness weight for libpng | libpng_read_fuzzer | my-task-id
$ buttercup-util read_harnesses
weight: 1.0
package_name: "libpng"
harness_name: "libpng_read_fuzzer"
task_id: "my-task-id"
2025-03-12 10:55:35,483 - buttercup.common.util_cli - INFO - Done
Add a build to the build map
$ buttercup-util add_build ./examples/build.txt
2025-03-12 10:55:07,273 - buttercup.common.util_cli - INFO - Added build for my-task-id | fuzzer | address
$ buttercup-util read_builds my-task-id fuzzer
engine: "libfuzzer"
sanitizer: "address"
task_dir: "/crs_scratch/my-task-id/"
task_id: "my-task-id"
build_type: "FUZZER"
apply_diff: true
2025-03-12 10:55:20,298 - buttercup.common.util_cli - INFO - Done