Commit Graph

132 Commits

Author SHA1 Message Date
Henrik Brodin 0268403c44 Update protobuf to 6.33.5 to fix high severity security vulnerability (#463)
Resolves Dependabot alerts #62, #71, #81 by updating protobuf from
6.33.4 to 6.33.5 across all components.
2026-02-04 20:28:24 +01:00
Ronald Eytchison e826b53530 Fix CoverageBot.run_task and add test cases (#458)
* Use openlit <1.36.6

Openlit >=1.36.6 has a bug with langgraph instrumentation.

Also add a test to seed-gen for when openlit is enabled

* Link to issue

* Fix CoverageBot.run_task and add test cases

---------

Co-authored-by: Henrik Brodin <90325907+hbrodin@users.noreply.github.com>
2026-01-27 23:13:34 +01:00
Ronald Eytchison a31b670a2e Fix: Use Openlit <1.36.6 to avoid bug with langgraph instrumentation (#457)
* Use openlit <1.36.6

Openlit >=1.36.6 has a bug with langgraph instrumentation.

Also add a test to seed-gen for when openlit is enabled

* Link to issue
2026-01-27 16:28:41 -05:00
Henrik Brodin b94a23eedb build(deps): update openlit and pydantic to unblock mcp upgrade (#455)
Update dependency constraints to allow mcp 1.26.0:
- common: openlit ==1.35.0 -> ~=1.36.0
- orchestrator: pydantic ~=2.10.5 -> ~=2.11.0

The mcp upgrade was blocked because:
1. openlit 1.35.0 constrained openai-agents to older versions
2. pydantic <2.11.0 blocked mcp 1.23.0+ (requires pydantic>=2.11.0)

Resulting upgrades:
- mcp: 1.12.4 -> 1.26.0
- openlit: 1.35.0 -> 1.36.7
- pydantic: 2.10.6 -> 2.11.10
- openai-agents: 0.2.8 -> 0.3.3

Closes #450

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 14:15:53 +01:00
dependabot[bot] 678b6657f3 build(deps): bump python-multipart from 0.0.20 to 0.0.22 in /fuzzer
Bumps [python-multipart](https://github.com/Kludex/python-multipart) from 0.0.20 to 0.0.22.
- [Release notes](https://github.com/Kludex/python-multipart/releases)
- [Changelog](https://github.com/Kludex/python-multipart/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Kludex/python-multipart/compare/0.0.20...0.0.22)

---
updated-dependencies:
- dependency-name: python-multipart
  dependency-version: 0.0.22
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-01-27 11:11:22 +01:00
Riccardo Schirone 2257d3eaa6 fix: resolve Dockerfile hadolint issues
- Add `# hadolint ignore=DL3006` for ARG-based FROM statements
- Add `# hadolint ignore=SC1091` for sourcing /etc/os-release
- Add `--no-install-recommends` to apt-get install commands
- Add `ca-certificates` package for HTTPS curl operations
- Add `automake` to cscope-builder stage (required for autoreconf)
- Consolidate consecutive RUN instructions in fuzzer Dockerfile

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-26 17:06:33 +01:00
Riccardo Schirone 47f38b8863 fix: apply ruff import sorting fixes to all source files
Run `ruff check --fix` across all components to fix I001 import sorting
violations in src/ and test/ directories.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-26 15:17:12 +01:00
Dan Guido 00283a31a5 fix: apply ruff and ty fixes to all Python source files
Apply automated fixes from ruff (import ordering, formatting) and ty (type
checker) to all Python source files across all components:

- Reorder imports according to isort/ruff rules
- Remove unnecessary type: ignore comments
- Fix minor type issues identified by ty
- Add atexit.register(logging.shutdown) where needed
- Consistent formatting

Components affected:
- common/src/
- orchestrator/src/
- fuzzer/src/
- fuzzer_runner/src/
- patcher/src/
- program-model/src/
- seed-gen/src/

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-26 15:17:12 +01:00
Dan Guido e3d0d29356 fix: apply ruff fixes to all test files
Apply automated fixes from ruff (import ordering, formatting) to all test
files across all components:

- Reorder imports according to isort/ruff rules
- Remove unnecessary imports
- Consistent formatting

Components affected:
- fuzzer/tests/
- orchestrator/test/
- patcher/tests/
- program-model/tests/
- seed-gen/test/

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-26 11:52:59 +01:00
Dan Guido 0206c0f6e1 chore: update Python dependencies and pyproject.toml configs
Update dependencies across all components:
- Update ruff to v0.12.0
- Add ty (Astral type checker) as dev dependency
- Standardize pyproject.toml configurations
- Regenerate uv.lock files

Components updated:
- common
- orchestrator
- fuzzer
- fuzzer_runner
- patcher
- program-model
- seed-gen

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-26 11:44:48 +01:00
Dan Guido 1d47e6347f fix: apply hadolint fixes to all Dockerfiles
Apply hadolint recommendations to all Dockerfiles:
- Add SHELL instruction with pipefail for proper error handling
- Use COPY instead of ADD where appropriate
- Add hadolint ignore comments for intentional violations
- Consistent formatting

Files fixed:
- fuzzer/Dockerfile
- orchestrator/Dockerfile
- patcher/Dockerfile
- program-model/Dockerfile
- seed-gen/Dockerfile

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-26 11:23:54 +01:00
Henrik Brodin 1d83c2daf4 Improve fuzzer logging (#416)
To help when integrating new harnesses with buttercup, this provides
more detailed logging about harness execution and exit.
2026-01-21 13:03:00 +01:00
Henrik Brodin 58c52cf6e2 Improve coverage tracking precision (#401)
* Improve coverage tracking precision

Previously coverage tracking included all regions. This changes coverage tracking to be more precise.

Macros that result in code are represented as a single line in the function, and any executed line in the macro will make that line 'covered'. Beyound that, only CodeRegions will count. This will prevent code guarded by #if that aren't in the binary from being considered reachable for example. The idea is that seed-gen would have more accurate information when selecting functions to target.

I've done some testing and it appears as if the new implementation in general reaches more lines and covers more functions.

* Fix macro coverage leaking across files due to missing filename in key

The expansion_coverage map used (line, col) as key without filename,
causing coverage from one file to incorrectly appear in another when
macros were at the same coordinates.

* Recursively expand macros and add named types for coverage data

- Process ExpansionRegion target_regions recursively instead of counting
  only the call site line
- Add coordinate index for O(1) expansion lookups
- Cache computed expansion lines to avoid recomputation across functions
- Use bulk set operations for better performance
- Prevent infinite loops with visited set for circular macro references
- Add named types for coverage data structures:
  - RegionCoords, ExpansionKey, CachedExpansionLines
  - Type aliases: ExpansionMap, CoordToFilenames, ExpansionLinesCache
- Add comprehensive tests for nested macros and edge cases
2025-12-18 08:36:07 +01:00
Riccardo Schirone ff998721c2 fuzzer: demangle function names when storing Coverage info (#384)
* fuzzer: demangle function names when storing Coverage info

* check if demangled name is empty
2025-11-11 16:58:04 +01:00
Riccardo Schirone 0d71570949 Add support for C++ projects (#357)
* Split C and C++ projects
* Add support for C++ files in tree-sitter
* Remove patch_validation step because it does not support c++ and it was anyway a leftover from the AIxCC competition. It can actually happen that a patch needs to fix a harness and checking if the patch is only in the right type of file was never that good.
* Remove language-identifier binaries because not needed anymore (due to previous point).
2025-09-16 17:57:46 +02:00
Riccardo Schirone 143a59c236 fuzzer_runner: split fuzzer's dependencies (#320)
Before this commit, the fuzzer dependend on clusterfuzz, which uses
protobuf 3.20. Since fuzzer-bot depended on common subpackage as well
and also common (and the other packages) require protobuf, we had to use
protobuf 3.20 everywhere. This old dependency however means that a lot
of packages can't be used in their "newer" versions, because they depend
on newer protobuf versions.

This commit splits the fuzzer into a separate fuzzer-runner that is
executed in a separate process inside a separate venv. fuzzer-runner
executes the clusterfuzz-heavy operations (e.g. fuzzing) and isolates the
clusterfuzz dependency, so that the rest of the system can use newer
protobuf version.

* split `fuzzer-bot` in `fuzzer-runner` and `fuzzer-bot`
* have a "full" optional dependency group in `common`, including `openlit`
  and `protobuf`, so that `fuzzer-runner` can use the lite version of `common`
  without bringing those heavy deps
* move `FuzzConfiguration`/`BuildConfiguration` in a separate common file
  that doesn't require to load all protobuf files. Again, in this way
  other components can just depend on the `common` "lite" version and not
  require protobuf stuff
* add `fuzzer-runner` as a separate venv inside the `fuzzer-bot` container
* add `RunnerProxy` class in fuzzer package to provide an interface to
  interact with the fuzzer-runner binary.
2025-09-10 09:52:08 +02:00
Dan Guido 7bc3dfdc43 chore: standardize ruff configuration across all components (#310) 2025-08-22 14:08:07 -04:00
Dan Guido 42691e50b4 style: apply ruff auto-fixes and formatting across entire codebase (#309)
* style: apply ruff auto-fixes and formatting across entire codebase

Applied safe auto-fixes from ruff v0.12.9 with --select ALL to improve code quality:
- Reorder imports (stdlib → third-party → local)
- Use modern type hints (collections.abc.Generator instead of typing.Generator)
- Add trailing commas for better diffs
- Format multi-line function parameters for readability
- Add strict=False to zip() calls for explicit behavior
- Simplify redundant elif to if after return statements
- Consistent code formatting with ruff format

These are all mechanical, non-controversial changes that improve code consistency
without altering functionality. Changes affect 180 files across all modules:
common, fuzzer, orchestrator, patcher, program-model, and seed-gen.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* re-applt ruff after merge

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Michael D Brown <michael.brown@trailofbits.com>
2025-08-22 10:30:12 -04:00
Dan Guido 37597105ed Standardize Python packaging and project configuration (#285)
* fix: respect OSS_FUZZ_CONTAINER_ORG environment variable

The oss_fuzz_container_org property now checks the OSS_FUZZ_CONTAINER_ORG
environment variable first before falling back to parsing the helper file.
This fixes the failing test_container_image_custom_org integration test
that was caught in nightly CI.

The test was expecting that setting OSS_FUZZ_CONTAINER_ORG=myorg would
result in container images using that organization, but the code was
ignoring the environment variable entirely.

* fix: standardize Python packaging and project configuration

- Fix critical Python version inconsistency (common was 3.10+, dependents required 3.12+)
- Standardize project metadata: add descriptions, licenses, consistent author emails
- Implement consistent dependency management using compatible release (~=) strategy
- Modernize all components to use [project.optional-dependencies] instead of [dependency-groups]
- Restore essential tool configurations (ruff lint rules, pytest settings, coverage config)
- Remove redundant component Makefiles (orchestrator, program-model)
- Add project URLs for better package discoverability

This resolves packaging inconsistencies introduced in PR #271 while maintaining
modern Python packaging standards.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: restore accidentally removed Makefiles with unique functionality

- Restore orchestrator/Makefile: contains unique API management targets (update-apis, generate-competition-api)
- Restore program-model/Makefile: contains specific integration test commands not in root Makefile
- These Makefiles provide component-specific functionality not available elsewhere

* fix: standardize program-model Dockerfile to use Python 3.12

Ensures consistency with pyproject.toml requirement of Python >=3.12,<3.13.
This aligns with the other components and standardizes Python version
across all Dockerfiles in the project.

* refactor: improve Dockerfile consistency and layer caching

- Combine consecutive apt operations to reduce layers
- Add missing DEBIAN_FRONTEND=noninteractive declarations
- Ensure all apt operations include cleanup with rm -rf /var/lib/apt/lists/*
- Reduces image size and improves build consistency

* fix: use [dependency-groups] instead of [project.optional-dependencies]

Addresses reviewer feedback that [dependency-groups] is the semantically
correct approach for development dependencies like test, lint, and typing tools.

Per PEP 735 and packaging.python.org guidance:
- [dependency-groups] for development workflow dependencies
- [project.optional-dependencies] for optional user-facing features

This standardizes all components to use the modern PEP 735 approach.

* fix: move dependencies from [project.urls] to [project] section

Fixes TOML syntax error where dependencies array was incorrectly placed
under [project.urls] instead of [project], causing build failures:
'TypeError: URL  of field  must be a string'

This resolves CI failures across all components during uv sync.

* fix: temporarily disable ruff rules to resolve CI failures

Temporarily disabled the following ruff rules for standardization PR:
- I001: Import block is un-sorted or un-formatted
- E501: Line too long
- W291: Trailing whitespace
- UP006: Use built-in collection types for type annotations
- UP015: Unnecessary mode argument
- UP035: Import from modern locations instead of deprecated typing
- UP046: Use modern generic class syntax

These rules were disabled in common/ and fuzzer/ components where they
were causing CI failures. Rules are commented with intention to re-enable
after refactoring work is complete.

Also fixed ruff formatting issues in seed-gen component.

* refactor: simplify dependency groups per maintainer feedback

Address maintainer feedback by consolidating dependency groups:
- Combined dev/test/typing/lint groups into single practical 'dev' group
- Ensures mypy gets all required type stubs (no more missing dependencies)
- Eliminates duplication and reduces cognitive overhead
- Users now only need: uv sync --group dev

Also removed pytest configuration from common/ as requested, since most
settings were defaults and only existed in one component.

Changes provide better developer experience with simpler, working
dependency management.

* fix: add missing UP045 rule to ruff ignore lists

Addresses CI failure: 'UP045 Use X | None for type annotations'

Added UP045 to ruff ignore lists in:
- common/pyproject.toml
- fuzzer/pyproject.toml
- orchestrator/pyproject.toml
- seed-gen/pyproject.toml

This completes the temporary rule disabling for the standardization PR.
UP045 enforces modern union syntax (X | None vs Optional[X]) - will
re-enable after refactoring.

* fix: correct argon2-cffi version constraint in orchestrator

Changes argon2-cffi from ~=21.0.0 to ~=21.3.0 to resolve dependency
resolution failure. Version 21.0.0 never existed on PyPI - available
versions jump from 20.1.0 directly to 21.1.0.

This restores the previously working constraint and resolves:
'No solution found when resolving dependencies: argon2-cffi>=21.0.0,<21.1.dev0'

* chore: update mypy to latest version 1.17.1

Updates mypy from 1.15.0 to 1.17.1 across all components to ensure
we're using the latest type checker features and bug fixes.

ruff is already on the latest version (0.12.8).

This keeps the linting tools current and prevents them from becoming
outdated over time.

* fix: add necessary ruff ignore rules to orchestrator

Add the specific ruff rules that orchestrator needs disabled
based on CI lint failures. Other components passed linting,
so only orchestrator needs these additional rules.

Rules added:
- I001: Import block formatting
- E501: Line too long
- UP006: Built-in collection types
- UP007: Union type annotations
- UP009: UTF-8 encoding declarations
- UP015: Unnecessary mode argument
- UP035: Modern import locations

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* feat: add missing ruff config and project metadata

- Add ruff configuration to program-model for consistency
- Add project URLs to 4 components for discoverability
- Add types-redis to fuzzer dev dependencies for type checking
- Ensure all components have consistent ignore rules

* fix: add additional ruff ignore rules for patcher and program-model

- Add W293, UP012, UP031 to patcher ignore list
- Add UP032 to program-model ignore list
- Ensures ruff checks pass for both components

* fix: resolve ruff formatting and dependency issues

- Format all program-model files with ruff (28 files reformatted)
- Standardize patcher dependencies to use ~= version specifiers
- Update langgraph-checkpoint to ~=2.1.0 to resolve conflicts
- Fix Docker PYTHON_VERSION variable usage in patcher/Dockerfile
- Update patcher lock file with new dependency constraints

* Update tree-sitter-language-pack and refresh uv.lock

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Michael D Brown <michael.brown@trailofbits.com>
Co-authored-by: Riccardo Schirone <riccardo.schirone@trailofbits.com>
2025-08-18 09:12:53 -04:00
Riccardo Schirone a533a28d47 Revert "refactor: standardize packaging across all components (#266)"
This reverts commit cc938e105f.
2025-08-09 13:13:28 -05:00
Riccardo Schirone 465d05fa9c Revert "Standardize packaging and dev dependencies across components (#268)"
This reverts commit 1c574962b3.
2025-08-09 13:13:28 -05:00
Dan Guido 1c574962b3 Standardize packaging and dev dependencies across components (#268)
* fix: standardize packaging configuration across components

- Fix fuzzer package name from 'fuzzing-infra' to 'fuzzer' for consistency
- Add missing README field to common/pyproject.toml
- Standardize seed-gen script naming to follow buttercup-* convention
- Remove redundant flake8 dependency from orchestrator and patcher (ruff handles linting)
- Update lock files to reflect dependency changes
- Auto-format code in common and seed-gen components

* refactor: standardize dev dependencies across all components

- Add pytest-cov to all components for consistent test coverage reporting
- Standardize types-redis version to 4.6.0 across all components
- Standardize types-requests version to 2.32.0 where used
- Remove unused pytest-asyncio from orchestrator and patcher (no async tests found)
- Remove unused types-python-dateutil from patcher (only used in orchestrator)
- Keep component-specific dependencies where justified:
  - orchestrator: pytest-xdist for parallel testing, API testing tools
  - patcher: pytest-xdist for parallel testing
  - common: types-PyYAML, dirty-equals for testing
  - seed-gen: tqdm for progress bars

All components now have consistent base dev dependencies with optional extras
as needed. This improves maintainability and reduces confusion.

* fix: update deployment configs for renamed seed-gen command

- Update docker-compose to use buttercup-seed-gen command
- Update Kubernetes/Helm deployment to use buttercup-seed-gen command
- These are critical fixes to ensure deployments work with the renamed command
2025-08-09 01:09:36 -07:00
Dan Guido cc938e105f refactor: standardize packaging across all components (#266)
* refactor: standardize packaging across all components

- Standardize Python version to >=3.12,<3.13 for all components
- Migrate from [project.optional-dependencies] to modern [dependency-groups] (PEP 735)
- Standardize ruff line-length to 120 characters across all components
- Add consistent project metadata:
  - AGPL-3.0 license field
  - Repository and Issues URLs
  - Improved, descriptive description fields
  - Fix email addresses to include .com domain

This improves consistency, maintainability, and follows modern Python
packaging best practices with uv/pip standards.

* fix: address PR review comments

- Move requires-python field to standard position (after license) in seed-gen
- Update all components to use latest ruff version (>=0.12.8)
- Ensure consistent dependency ordering across all components

* refactor: standardize dependency pinning strategy

Apply consistent dependency versioning across all components:

- Use ~= (compatible release) for core dependencies:
  - Infrastructure: redis, pydantic, fastapi, uvicorn, sqlalchemy
  - AI/LLM: openai, langchain-community, langgraph-checkpoint
  - Parsing: tree-sitter, tree-sitter-language-pack
  - Web: requests, urllib3, pyyaml
  - Utils: python-dotenv, unidiff, argon2-cffi, pymongo, six

- Keep >= for stable dev tools:
  - pytest, mypy, ruff, flake8 (want latest versions)
  - types-* packages (want latest type definitions)
  - rich, beautifulsoup4 (stable, backwards compatible)

- Keep exact pins for known issues:
  - protobuf (narrow range for compatibility)
  - openlit==1.32.12 (documented issue with 1.33)
  - clusterfuzz==2.6.0 (complex, version-sensitive)

This provides predictable builds with automatic patch updates while
preventing unexpected breaking changes from major/minor version bumps.

* feat: add project metadata for discoverability

Add comprehensive metadata to all components:

Keywords:
- common: cybersecurity, crs, utilities, protobuf, redis, telemetry
- fuzzer: fuzzing, oss-fuzz, libfuzzer, vulnerability-discovery, coverage
- orchestrator: orchestration, task-management, scheduler, api, fastapi
- patcher: patching, vulnerability-repair, llm, ai, code-generation
- program-model: static-analysis, codequery, tree-sitter, semantic-analysis
- seed-gen: test-generation, input-generation, fuzzing, seed-corpus, llm

Classifiers:
- Development Status :: 4 - Beta (all components)
- License :: OSI Approved :: GNU Affero General Public License v3
- Programming Language :: Python :: 3.12
- Topic :: Security (all components)
- Component-specific topics (Testing, AI, Distributed Computing, etc.)
- Operating System :: POSIX :: Linux

URLs:
- Added Documentation URL pointing to README for all components

This improves package discoverability, provides clear metadata for tools,
and gives the project a more professional appearance.

* Standardize tool configurations across all components

- Add pytest.ini_options configuration to all components
- Add coverage configuration with consistent exclude patterns
- Standardize ruff configuration with target-version and lint rules
- Fix missing readme field in fuzzer/pyproject.toml
- Fix python-dotenv spacing inconsistency in seed-gen
- Standardize all dev dependencies to use ~= operator for consistency

* Fix trailing whitespace and line length issues

- Remove trailing whitespace from tree-sitter query strings
- Remove trailing whitespace from test output strings
- Fix line length issues in logger.info() calls by splitting format strings
- Fix line length in datetime formatting by extracting variables
- Split long Pydantic Field descriptions and docstrings
- Leave test data strings unchanged to avoid breaking tests

* Fix dependency resolution issues

- Update argon2-cffi from ~=21.0.0 to ~=21.3.0 (21.0.x doesn't exist on PyPI)
- Update langgraph-checkpoint from ~=2.0.25 to ~=2.1.0 to match langgraph requirements
- Standardize spacing around ~= operators in all dependency specifications
- All components now successfully resolve dependencies with uv

* Apply ruff auto-fixes across project

- Fix import sorting (I001) in fuzzer, orchestrator, and patcher
- Update to PEP 585 type annotations (List->list, Dict->dict, etc.)
- Update to PEP 604 union syntax (Optional[X] -> X | None)
- Remove unnecessary UTF-8 encoding declarations
- Remove redundant file open modes
- Modernize type annotations throughout the codebase

Remaining issues are primarily line length (E501) which require manual review

* Fix line length issues in program-model component

- Break up long Java code strings in test assertions using implicit concatenation
- Split long constructor and method definitions across multiple lines
- Add noqa: E501 comment for 10,977 character struct definition test data
- All program-model line length issues resolved

* fix: revert protobuf enum type annotations to Optional

Protobuf enums (EnumTypeWrapper) don't support the | operator for type unions.
The ruff UP035 rule converted Optional[ProtobufEnum] to ProtobufEnum | None,
but this causes TypeError at runtime. Reverted these specific changes while
keeping the modern type union syntax for regular Python types.

* chore: add ruff protection for protobuf enum type annotations

- Configure ruff to ignore UP045 rule in test_submissions.py
- Add inline noqa comments to document why Optional is needed
- Protobuf enums (EnumTypeWrapper) don't support the | operator
- This prevents future automated fixes from breaking the code

* fix: modernize Python syntax and fix formatting issues

- Convert printf-style formatting to f-strings (UP031)
- Remove trailing whitespace from blank lines (W293)
- Use PEP 695 generic class syntax for Python 3.12+ (UP046)
- Use PEP 695 type alias syntax with 'type' keyword (UP040)

These changes modernize the codebase to use Python 3.12+ features
and fix formatting inconsistencies detected by ruff.

* fix: resolve undefined MsgType reference after PEP 695 conversion

When converting to PEP 695 generic class syntax, the MsgType TypeVar
was removed but was still referenced in overloaded method signatures.
Changed the generic fallback overload to use Message directly.

* fix: resolve line length violations across entire codebase

Applied Black formatter and manual fixes to resolve E501 line length violations:
- Fixed 178 line length issues across common, fuzzer, orchestrator, patcher, and program-model components
- Used Black formatter for automatic reformatting where possible
- Manually split long strings, function calls, and complex expressions
- All files now comply with 120-character line limit

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: revert PEP 695 type alias syntax in node_local.py

The PEP 695 syntax (type X = Y) creates TypeAliasType objects that cannot
be used as constructors at runtime. Since node_local.py uses NodeLocalPath
and RemotePath as constructors (e.g., NodeLocalPath(path)), we must use
the old TypeAlias syntax to maintain runtime functionality.

Added noqa comments to prevent ruff from attempting to modernize these
aliases in the future.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* chore: configure ruff to ignore UP040 for node_local.py

Added per-file configuration to prevent ruff from attempting to convert
TypeAlias annotations to PEP 695 syntax in node_local.py. This protects
the runtime functionality that relies on these type aliases being usable
as constructors.

Also removed redundant inline noqa comments since the ignore is now
configured at the project level.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: correct MsgType reference in static method decorator

The _ensure_group_name static method decorator was incorrectly
referencing MsgType in the wrapper function signature. Since MsgType
is a class-level type parameter and not accessible in static method
scope, changed it to Message which is the appropriate bound type.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* docs: add explanatory comment for Message type in decorator

Added a comment explaining why we must use Message instead of MsgType
in the _ensure_group_name decorator's wrapper function. This prevents
future confusion and protects against accidental "fixes" that would
break the code.

The MsgType parameter is a class-level type variable that's not in
scope within the static method decorator context.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-08-08 19:59:56 -07:00
Dan Guido f24e5fa971 Revert "refactor: standardize packaging across all components"
This reverts commit 1efa75836f.
2025-08-08 17:18:54 -07:00
Dan Guido 1efa75836f refactor: standardize packaging across all components
- Standardize Python version to >=3.12,<3.13 for all components
- Migrate from [project.optional-dependencies] to modern [dependency-groups] (PEP 735)
- Standardize ruff line-length to 120 characters across all components
- Add consistent project metadata:
  - AGPL-3.0 license field
  - Repository and Issues URLs
  - Improved, descriptive description fields
  - Fix email addresses to include .com domain

This improves consistency, maintainability, and follows modern Python
packaging best practices with uv/pip standards.
2025-08-08 17:15:50 -07:00
Ronald Eytchison 36ade4e8f8 Fix merger-bot bug by using PathLike[str] interface (#256)
Also add integration test for merger bot

Co-authored-by: Brad Swain <brad.swain@trailofbits.com>
2025-08-08 12:13:46 -05:00
Brad Swain c350d95221 Enforce mypy on fuzzer component (#229)
* Enforce mypy on fuzzer component

- Add mypy type checking enforcement for fuzzer component in Makefile
- Fix 39 type errors across 9 files in fuzzer component:
  - Add missing return type annotations (-> None, -> bool, -> int)
  - Fix Beautiful Soup union type issues with proper casting and None checks
  - Fix TemporaryDirectory path usage by using .name attribute
  - Add type annotations for function parameters and variables
  - Handle potential None values in registry and XML parsing

All fuzzer component files now pass strict mypy type checking.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-authored-by: Brad Swain <BradSwain@users.noreply.github.com>

* fix tests

* Update fuzzer/src/buttercup/fuzzing_infra/builder_bot.py

Co-authored-by: Riccardo Schirone <562321+ret2libc@users.noreply.github.com>

* Update fuzzer/src/buttercup/fuzzing_infra/builder_bot.py

Co-authored-by: Riccardo Schirone <562321+ret2libc@users.noreply.github.com>

* use pathlike[str] instead of tmpdir

* update test to use Path directly

* fix type annotation

---------

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Brad Swain <BradSwain@users.noreply.github.com>
Co-authored-by: Riccardo Schirone <562321+ret2libc@users.noreply.github.com>
2025-08-04 16:49:36 -04:00
Riccardo Schirone 5cfa9bd26e Add support for ARM (#207)
* Fix integration-test run

* use the proper oss-fuzz-aixcc commit to run integration-test
* determine OSS_FUZZ_CONTAINER_ORG at runtime
* use `git diff` in the patcher to create the patch to fix a problem
  with patches affecting non-newline terminated files

* fix parsing

* fix program-model lint

* Modify CRS to work on MacOS/ARM

* Use LibUCL for testing

* use example-libpng

* fix linting

* common: fix tests
2025-07-31 09:55:36 -04:00
Riccardo Schirone a08cbf1050 Make CRS more flexible (#202)
* Fix integration-test run

* use the proper oss-fuzz-aixcc commit to run integration-test
* determine OSS_FUZZ_CONTAINER_ORG at runtime
* use `git diff` in the patcher to create the patch to fix a problem
  with patches affecting non-newline terminated files

* fix parsing

* fix program-model lint
2025-07-31 09:34:28 -04:00
Brad Swain c7fc0f7c9a remove refs to aixcc resources (#195)
* remove refs to aixcc resources

* remove aixcc.tech

* change aixcc-finals to tob-challenges
2025-07-30 09:34:58 -05:00
Riccardo Schirone 1d385cf89f Make deployment easier (#158)
* deployment: just use the value in values.template

* ci: disable integration tests and private settings

* Download trailofbits cscope, not aixcc-finals one

* ci: fix docker login to ghcr.io

* Changes to allow non aixcc deployment

* buttercup-ui: basic skeleton for a CRS interface

* file-server implementation

* add ui to k8s

* other apis

* try to fix k8s

* remove some labels

* fix ui

* small fixes to doc

* ui: support for cloning private repos

* Add setup scripts for easy deployment

* update scripts

* update make/readme

* small adj

* address review

* we need 0.0.0.0 for k8s
2025-07-11 07:31:57 -04:00
Riccardo Schirone 581458f67d Update components dependencies 2025-07-09 18:17:03 +02:00
Riccardo Schirone cdb67b0b60 Changes to allow non aixcc deployment (#147) 2025-07-09 12:22:54 +02:00
Riccardo Schirone 32849ffa6f Timeout fixes (#930) 2025-06-24 19:55:03 +02:00
Ronald Eytchison 5fe43e0b0a Enforce size limits for PoVs and seeds (#905)
* Don't process PoVs that exceed the max size

* WIP enforce seed and PoV size limits in seed-gen

* Revert "Don't process PoVs that exceed the max size"

This reverts commit 8cee343160ba299dccd12284f65934efda63b2ef.

* Enforce max PoV size in fuzzer

* Set max seed size to 64 KiB

* Fixup

* Pass copy_corpus_max_size to Corpus and InputDir

* Fixup
2025-06-23 09:10:53 -04:00
Riccardo Schirone 20b9fd61e1 Use 'git apply' to apply the diff, instead of 'patch' (#893) 2025-06-19 17:57:12 +02:00
Henrik Brodin 6409b5be0c Use an internal_patch_id instead of indices (#867)
* Use an internal_patch_id instead of indices

This is the first step in being able to merge sets of PoVs and test
patches against all PoVs within a task.

* Discard redundant builds

* Compare relevant fields
2025-06-18 14:08:50 +02:00
Riccardo Schirone bea67eaa09 Update oss-fuzz base to v1.3.0 (#879) 2025-06-18 13:51:30 +02:00
Riccardo Schirone a4a846df22 Clean tasks (#844)
implement scratch-cleaner daemonset
2025-06-13 19:10:44 +02:00
Riccardo Schirone 512a66f021 CrashDir: store povs in <crash-token>/<sanitizer> (#829)
In Redis we were already differentiating between different sanitizers,
however we were storing all PoVs with the same crash-token in the same
directory, indipendently of the sanitizer that triggered the bug.

However, this information is necessary for the patcher that has to
reproduce the PoVs.
2025-06-11 14:37:50 -04:00
Evan Downing 0f68d6795f Propagate log max line length (#813) 2025-06-11 09:27:30 -04:00
Riccardo Schirone d877b0cdd7 orchestrator: patched build requests (#820) 2025-06-11 15:12:28 +02:00
Michael D Brown 17e60bf8d9 Updates to stack parsing (#817)
* (612) Make crash tokens a combo of crash data and inst key

* minor refactor
2025-06-10 14:16:29 -04:00
Riccardo Schirone b81135bf66 Make builder-bot build patched versions (#819)
* Make builder-bot build patched versions

* orchestrator: test new build outputs

* address review

* builder-bot: use times_delivered logic in case of failures
2025-06-10 09:58:38 -04:00
Riccardo Schirone 9cedaccb4e fuzzer: add simple utility to get clusterfuzz crash_state (#804)
* fuzzer: add simple utility to get clusterfuzz crash_state

* fix lint
2025-06-06 09:39:42 -04:00
Evan Downing d50d21015a Increase memory limits (#800) 2025-06-06 09:01:29 -04:00
Riccardo Schirone 2540e2357c common: provide unified_language property in ProjectYaml (#774)
* common: provide unified_language property in ProjectYaml

* common: use also cpp

* Applying fixup to missed location

* codequery: fix how unified_language is used

* program-model: fix filter_project_context for new Language enum

* program-model: fix tests

* fix program model tests

---------

Co-authored-by: Michael D. Brown <michael.brown@trailofbits.com>
2025-06-04 23:33:32 -04:00
Evan Downing 10d83ba66d Bump versions to v1.2.1 (#780) 2025-06-04 07:25:48 -04:00
Evan Downing 07f9ce5310 Output fuzz options to help debug (#777) 2025-06-03 16:17:37 -04:00
Henrik Brodin 16d429823f Bumping the runner image to v1.2.0 (#767)
* Bumping the runner image to v1.2.0

Also, ensures it is preloaded into the image cache and increases the
size of the image cache storage to account for more data. Removed an
unused configuration section for an old dind-sidecar.

* Fix incorrect version
2025-06-03 10:59:50 +02:00