Add new CLI subcommand to extract PoVs, stack traces, and patches from
Redis submissions into a structured directory format for easy analysis.
Features:
- Extracts crash inputs (PoV files) via kubectl cp from cluster pods
- Writes fuzzer and tracer stack traces to text files
- Exports associated patches with metadata
- Organizes output by project/task_id/vulnerability
- Supports filtering by task_id and passed_only options
- Skips empty patch trackers (placeholders that never received content)
* Improve coverage tracking precision
Previously coverage tracking included all regions. This changes coverage tracking to be more precise.
Macros that result in code are represented as a single line in the function, and any executed line in the macro will make that line 'covered'. Beyound that, only CodeRegions will count. This will prevent code guarded by #if that aren't in the binary from being considered reachable for example. The idea is that seed-gen would have more accurate information when selecting functions to target.
I've done some testing and it appears as if the new implementation in general reaches more lines and covers more functions.
* Fix macro coverage leaking across files due to missing filename in key
The expansion_coverage map used (line, col) as key without filename,
causing coverage from one file to incorrectly appear in another when
macros were at the same coordinates.
* Recursively expand macros and add named types for coverage data
- Process ExpansionRegion target_regions recursively instead of counting
only the call site line
- Add coordinate index for O(1) expansion lookups
- Cache computed expansion lines to avoid recomputation across functions
- Use bulk set operations for better performance
- Prevent infinite loops with visited set for circular macro references
- Add named types for coverage data structures:
- RegionCoords, ExpansionKey, CachedExpansionLines
- Type aliases: ExpansionMap, CoordToFilenames, ExpansionLinesCache
- Add comprehensive tests for nested macros and edge cases
Previously, `send-integration-task` and `send-libpng-task` used
`pkill -f "kubectl port-forward"` which killed ALL kubectl port-forward
processes. This was problematic when other port-forwards were running.
Now each target captures the PID of the port-forward it starts and kills
only that specific process when done.
When the web-ui server stops while a browser window is open, the UI
was showing fake task data (libpng-analysis, libxml2-fuzzing) instead
of indicating the server was unavailable.
Changes:
- Remove getMockTasks() function that returned hardcoded fake data
- Add apiError state to track API connection status
- Update loadTasks() to set empty array and error flag on failure
- Add updateApiErrorBanner() to show/hide error banner
- Add CSS styles for the error banner
Fixes#395🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* Fix CRS_KEY_TOKEN_HASH truncation in setup-local
- auth_tool.py: Use plain print() when piped; Rich wraps at 80 chars
- common.sh: Escape $ in hash before sed (interpreted as backreferences)
* Revert unneeded change
OSS-Fuzz helper.py reproduce_impl passes err_result as positional arg
to docker_run(), interpreted as print_output=False, redirecting output
to /dev/null, breaking test_real_reproduce_pov on x86_64 CI.
Refactor _hack_oss_fuzz_aarch64_runner into unified _hack_oss_fuzz_runner
that takes architecture parameter and applies:
- Common fixes (reproduce_impl, debug tag, tag handling) to all arches
- ARM64-specific fixes (:manifest-arm64v8 tags) conditionally
Update get_rw_copy to call helper.py patching for all architectures.
The tracer bot failed to reproduce crashes on ARM64 due to incorrect
parameter passing in the reproduce_impl patch. The err_result value was
being passed as a positional argument to docker_run(), which interpreted
it as print_output=False, redirecting all output to /dev/null.
Changes:
- Fix reproduce_impl to pass architecture as keyword arg only
- Refactor ARM64 patching into separate dockerfile and runner functions
- Add comprehensive helper.py patching for ARM64:
* Patch image_name variables to use :manifest-arm64v8 tag
* Patch BASE_RUNNER_IMAGE assignment with tag stripping
* Fix debug mode tag insertion (insert -debug before tag, not after)
* Prevent double-tagging in _get_base_runner_image()
- Default CLI architecture parameters to ARCHITECTURE constant
This enables the tracer bot to see fuzzer output and successfully detect
crash reproduction on ARM64 systems.
Co-authored-by: Riccardo Schirone <562321+ret2libc@users.noreply.github.com>
libpng has changed the default branch to libpng18, but that doesn't
contain the oss-fuzz files. Let's use our own fork of oss-fuzz for now,
until https://github.com/google/oss-fuzz/pull/14080 is ready.
* Split C and C++ projects
* Add support for C++ files in tree-sitter
* Remove patch_validation step because it does not support c++ and it was anyway a leftover from the AIxCC competition. It can actually happen that a patch needs to fix a harness and checking if the patch is only in the right type of file was never that good.
* Remove language-identifier binaries because not needed anymore (due to previous point).
Before this commit, the fuzzer dependend on clusterfuzz, which uses
protobuf 3.20. Since fuzzer-bot depended on common subpackage as well
and also common (and the other packages) require protobuf, we had to use
protobuf 3.20 everywhere. This old dependency however means that a lot
of packages can't be used in their "newer" versions, because they depend
on newer protobuf versions.
This commit splits the fuzzer into a separate fuzzer-runner that is
executed in a separate process inside a separate venv. fuzzer-runner
executes the clusterfuzz-heavy operations (e.g. fuzzing) and isolates the
clusterfuzz dependency, so that the rest of the system can use newer
protobuf version.
* split `fuzzer-bot` in `fuzzer-runner` and `fuzzer-bot`
* have a "full" optional dependency group in `common`, including `openlit`
and `protobuf`, so that `fuzzer-runner` can use the lite version of `common`
without bringing those heavy deps
* move `FuzzConfiguration`/`BuildConfiguration` in a separate common file
that doesn't require to load all protobuf files. Again, in this way
other components can just depend on the `common` "lite" version and not
require protobuf stuff
* add `fuzzer-runner` as a separate venv inside the `fuzzer-bot` container
* add `RunnerProxy` class in fuzzer package to provide an interface to
interact with the fuzzer-runner binary.
* Test configuration for Azure deployment
* ci: re-enable docker build/push
* ci: docker push labeled PRs
* Use `make deploy` for both local and azure deployments
* doc: update AKS_DEPLOYMENT doc
* add confirmation in Makefile
* Make resource group location configurable
There is now an additional button available in the Submit Task dialogue.
This enables users to easily modify values such as timeouts etc and keeps
the main ui clean.
* renaming system integration tests to something more appropriate
* WIP to isolate unit tests from integration tests
* WIP
* fix typo
* fix after merge from main
* Merged changes from prior PRs into comp-integration
* separated integration tests for components into separate workflow
* fixed label
* cleanup
* put system integration tests back on nightly schedule
* Put component integration tests back on for push to main
* Disable some tests because target is not publicly available.
* appease linter
* trying HTTPS instead of SSH
* disable test with non-public target.
* disable finicky tests
* update label for component integration tests.
---------
Co-authored-by: Michael D. Brown <michaeldbrown@Michaels-MacBook-Pro-4.local>
* Add gemini api key option during setup and in deployment environments
* Add gemini pro as a fallback model in all components
* Lint
* Set rate limits for gemini models
* Add fallback models in more places
* Fix typo
* Fix kwargs expansion
* Fix instantiation of llm with callbacks
* Fixed formatting after merge
* Fix instantiation of default models
* Lint
* Fix llm creation
* Fix
* Lint
* Lint
---------
Co-authored-by: Michael D Brown <michael.brown@trailofbits.com>
* feat: add minimal pre-commit infrastructure
- Add pre-commit configuration with essential checks only
- Check YAML, TOML, JSON syntax
- Check for merge conflicts and large files
- Enforce LF line endings (fix 2 test files with CRLF)
- Add debug statement detection
- Integrate ruff for Python formatting and linting
- Add GitHub workflow for pre-commit CI
- No unnecessary Python code changes
* fix: add explicit permissions to pre-commit workflow
- Set GITHUB_TOKEN permissions to read-only for contents
- Follows principle of least privilege
- Addresses CodeQL security recommendation
- Pre-commit checks only need to read code, not write
* fix: exclude test data from line ending modifications
- Exclude .proto files and test/data directories from mixed-line-ending hook
- Revert changes to test data files (traced_crash.proto, java_stacktrace.txt)
- These files need to preserve their original format for test integrity
- Binary proto files could be corrupted by line ending changes
* docs: add pre-commit hooks documentation to CONTRIBUTING.md
- Add pre-commit installation instructions to development setup
- Document pre-commit hooks in Code Quality Standards section
- Update submission workflow to include pre-commit checks
- Provide manual pre-commit run commands for contributors
* docs: streamline CONTRIBUTING.md for better readability
- Reduce from 214 to 124 lines (42% reduction) while keeping all essential info
- Consolidate setup instructions into concise Quick Start section
- Convert component descriptions to scannable table format
- Streamline testing strategy with clear requirements and timing
- Add back critical testing prerequisites (codequery, ripgrep, cscope)
- Create actionable Getting Help section with common troubleshooting
- Remove redundant command listings and verbose explanations
- Maintain all security requirements and essential workflows
* style: apply ruff auto-fixes and formatting across entire codebase
Applied safe auto-fixes from ruff v0.12.9 with --select ALL to improve code quality:
- Reorder imports (stdlib → third-party → local)
- Use modern type hints (collections.abc.Generator instead of typing.Generator)
- Add trailing commas for better diffs
- Format multi-line function parameters for readability
- Add strict=False to zip() calls for explicit behavior
- Simplify redundant elif to if after return statements
- Consistent code formatting with ruff format
These are all mechanical, non-controversial changes that improve code consistency
without altering functionality. Changes affect 180 files across all modules:
common, fuzzer, orchestrator, patcher, program-model, and seed-gen.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* re-applt ruff after merge
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Michael D Brown <michael.brown@trailofbits.com>
Henrik Brodin