25 Commits

Author SHA1 Message Date
Riccardo Schirone 47f38b8863 fix: apply ruff import sorting fixes to all source files
Run `ruff check --fix` across all components to fix I001 import sorting
violations in src/ and test/ directories.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-26 15:17:12 +01:00
Dan Guido 42691e50b4 style: apply ruff auto-fixes and formatting across entire codebase (#309)
* style: apply ruff auto-fixes and formatting across entire codebase

Applied safe auto-fixes from ruff v0.12.9 with --select ALL to improve code quality:
- Reorder imports (stdlib → third-party → local)
- Use modern type hints (collections.abc.Generator instead of typing.Generator)
- Add trailing commas for better diffs
- Format multi-line function parameters for readability
- Add strict=False to zip() calls for explicit behavior
- Simplify redundant elif to if after return statements
- Consistent code formatting with ruff format

These are all mechanical, non-controversial changes that improve code consistency
without altering functionality. Changes affect 180 files across all modules:
common, fuzzer, orchestrator, patcher, program-model, and seed-gen.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* re-applt ruff after merge

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Michael D Brown <michael.brown@trailofbits.com>
2025-08-22 10:30:12 -04:00
Riccardo Schirone a533a28d47 Revert "refactor: standardize packaging across all components (#266)"
This reverts commit cc938e105f.
2025-08-09 13:13:28 -05:00
Dan Guido cc938e105f refactor: standardize packaging across all components (#266)
* refactor: standardize packaging across all components

- Standardize Python version to >=3.12,<3.13 for all components
- Migrate from [project.optional-dependencies] to modern [dependency-groups] (PEP 735)
- Standardize ruff line-length to 120 characters across all components
- Add consistent project metadata:
  - AGPL-3.0 license field
  - Repository and Issues URLs
  - Improved, descriptive description fields
  - Fix email addresses to include .com domain

This improves consistency, maintainability, and follows modern Python
packaging best practices with uv/pip standards.

* fix: address PR review comments

- Move requires-python field to standard position (after license) in seed-gen
- Update all components to use latest ruff version (>=0.12.8)
- Ensure consistent dependency ordering across all components

* refactor: standardize dependency pinning strategy

Apply consistent dependency versioning across all components:

- Use ~= (compatible release) for core dependencies:
  - Infrastructure: redis, pydantic, fastapi, uvicorn, sqlalchemy
  - AI/LLM: openai, langchain-community, langgraph-checkpoint
  - Parsing: tree-sitter, tree-sitter-language-pack
  - Web: requests, urllib3, pyyaml
  - Utils: python-dotenv, unidiff, argon2-cffi, pymongo, six

- Keep >= for stable dev tools:
  - pytest, mypy, ruff, flake8 (want latest versions)
  - types-* packages (want latest type definitions)
  - rich, beautifulsoup4 (stable, backwards compatible)

- Keep exact pins for known issues:
  - protobuf (narrow range for compatibility)
  - openlit==1.32.12 (documented issue with 1.33)
  - clusterfuzz==2.6.0 (complex, version-sensitive)

This provides predictable builds with automatic patch updates while
preventing unexpected breaking changes from major/minor version bumps.

* feat: add project metadata for discoverability

Add comprehensive metadata to all components:

Keywords:
- common: cybersecurity, crs, utilities, protobuf, redis, telemetry
- fuzzer: fuzzing, oss-fuzz, libfuzzer, vulnerability-discovery, coverage
- orchestrator: orchestration, task-management, scheduler, api, fastapi
- patcher: patching, vulnerability-repair, llm, ai, code-generation
- program-model: static-analysis, codequery, tree-sitter, semantic-analysis
- seed-gen: test-generation, input-generation, fuzzing, seed-corpus, llm

Classifiers:
- Development Status :: 4 - Beta (all components)
- License :: OSI Approved :: GNU Affero General Public License v3
- Programming Language :: Python :: 3.12
- Topic :: Security (all components)
- Component-specific topics (Testing, AI, Distributed Computing, etc.)
- Operating System :: POSIX :: Linux

URLs:
- Added Documentation URL pointing to README for all components

This improves package discoverability, provides clear metadata for tools,
and gives the project a more professional appearance.

* Standardize tool configurations across all components

- Add pytest.ini_options configuration to all components
- Add coverage configuration with consistent exclude patterns
- Standardize ruff configuration with target-version and lint rules
- Fix missing readme field in fuzzer/pyproject.toml
- Fix python-dotenv spacing inconsistency in seed-gen
- Standardize all dev dependencies to use ~= operator for consistency

* Fix trailing whitespace and line length issues

- Remove trailing whitespace from tree-sitter query strings
- Remove trailing whitespace from test output strings
- Fix line length issues in logger.info() calls by splitting format strings
- Fix line length in datetime formatting by extracting variables
- Split long Pydantic Field descriptions and docstrings
- Leave test data strings unchanged to avoid breaking tests

* Fix dependency resolution issues

- Update argon2-cffi from ~=21.0.0 to ~=21.3.0 (21.0.x doesn't exist on PyPI)
- Update langgraph-checkpoint from ~=2.0.25 to ~=2.1.0 to match langgraph requirements
- Standardize spacing around ~= operators in all dependency specifications
- All components now successfully resolve dependencies with uv

* Apply ruff auto-fixes across project

- Fix import sorting (I001) in fuzzer, orchestrator, and patcher
- Update to PEP 585 type annotations (List->list, Dict->dict, etc.)
- Update to PEP 604 union syntax (Optional[X] -> X | None)
- Remove unnecessary UTF-8 encoding declarations
- Remove redundant file open modes
- Modernize type annotations throughout the codebase

Remaining issues are primarily line length (E501) which require manual review

* Fix line length issues in program-model component

- Break up long Java code strings in test assertions using implicit concatenation
- Split long constructor and method definitions across multiple lines
- Add noqa: E501 comment for 10,977 character struct definition test data
- All program-model line length issues resolved

* fix: revert protobuf enum type annotations to Optional

Protobuf enums (EnumTypeWrapper) don't support the | operator for type unions.
The ruff UP035 rule converted Optional[ProtobufEnum] to ProtobufEnum | None,
but this causes TypeError at runtime. Reverted these specific changes while
keeping the modern type union syntax for regular Python types.

* chore: add ruff protection for protobuf enum type annotations

- Configure ruff to ignore UP045 rule in test_submissions.py
- Add inline noqa comments to document why Optional is needed
- Protobuf enums (EnumTypeWrapper) don't support the | operator
- This prevents future automated fixes from breaking the code

* fix: modernize Python syntax and fix formatting issues

- Convert printf-style formatting to f-strings (UP031)
- Remove trailing whitespace from blank lines (W293)
- Use PEP 695 generic class syntax for Python 3.12+ (UP046)
- Use PEP 695 type alias syntax with 'type' keyword (UP040)

These changes modernize the codebase to use Python 3.12+ features
and fix formatting inconsistencies detected by ruff.

* fix: resolve undefined MsgType reference after PEP 695 conversion

When converting to PEP 695 generic class syntax, the MsgType TypeVar
was removed but was still referenced in overloaded method signatures.
Changed the generic fallback overload to use Message directly.

* fix: resolve line length violations across entire codebase

Applied Black formatter and manual fixes to resolve E501 line length violations:
- Fixed 178 line length issues across common, fuzzer, orchestrator, patcher, and program-model components
- Used Black formatter for automatic reformatting where possible
- Manually split long strings, function calls, and complex expressions
- All files now comply with 120-character line limit

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: revert PEP 695 type alias syntax in node_local.py

The PEP 695 syntax (type X = Y) creates TypeAliasType objects that cannot
be used as constructors at runtime. Since node_local.py uses NodeLocalPath
and RemotePath as constructors (e.g., NodeLocalPath(path)), we must use
the old TypeAlias syntax to maintain runtime functionality.

Added noqa comments to prevent ruff from attempting to modernize these
aliases in the future.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* chore: configure ruff to ignore UP040 for node_local.py

Added per-file configuration to prevent ruff from attempting to convert
TypeAlias annotations to PEP 695 syntax in node_local.py. This protects
the runtime functionality that relies on these type aliases being usable
as constructors.

Also removed redundant inline noqa comments since the ignore is now
configured at the project level.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: correct MsgType reference in static method decorator

The _ensure_group_name static method decorator was incorrectly
referencing MsgType in the wrapper function signature. Since MsgType
is a class-level type parameter and not accessible in static method
scope, changed it to Message which is the appropriate bound type.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* docs: add explanatory comment for Message type in decorator

Added a comment explaining why we must use Message instead of MsgType
in the _ensure_group_name decorator's wrapper function. This prevents
future confusion and protects against accidental "fixes" that would
break the code.

The MsgType parameter is a class-level type variable that's not in
scope within the static method decorator context.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-08-08 19:59:56 -07:00
Brad Swain c7fc0f7c9a remove refs to aixcc resources (#195)
* remove refs to aixcc resources

* remove aixcc.tech

* change aixcc-finals to tob-challenges
2025-07-30 09:34:58 -05:00
Evan Downing 24a83b3ff3 Remove kythe code and update documentation accordingly 2025-07-09 16:31:37 +00:00
Evan Downing b497098585 Re-enable tests (#877)
* Re-enable tests

* Fix tests
2025-06-21 16:07:26 +02:00
Riccardo Schirone 9853d133a2 common|program-model: always use the most accurate container view (#860) 2025-06-17 16:37:07 +02:00
Evan Downing 4279960e2d Update oss-fuzz branch (#788)
Co-authored-by: Michael D Brown <michael.brown@trailofbits.com>
2025-06-04 23:34:40 -04:00
Riccardo Schirone 776732d017 program-model: use module scoped CodeQuery (#665)
* program-model: use module scoped CodeQuery

* fix also sqlite tests

* fix antlr4 test

* wip

* Fix macros test

* appease linting gods

* Fix cleanup routine

---------

Co-authored-by: Henrik Brodin <90325907+hbrodin@users.noreply.github.com>
Co-authored-by: Evan Downing <2077950+evandowning@users.noreply.github.com>
Co-authored-by: Michael D. Brown <michael.brown@trailofbits.com>
2025-05-21 16:40:55 -04:00
Evan Downing 36eee39bc6 Add commons-compress (#647)
Co-authored-by: Michael D Brown <michael.brown@trailofbits.com>
2025-05-20 14:28:49 -04:00
Evan Downing a23452d473 Add freerdp (#643)
* Add freerdp

* linter fix after merge

---------

Co-authored-by: Michael D Brown <michael.brown@trailofbits.com>
2025-05-20 14:17:53 -04:00
Evan Downing aed41348ce Add dropbear (#645) 2025-05-20 10:26:24 -04:00
Evan Downing 0f02c16c2c Add sqlite (#644) 2025-05-20 10:25:53 -04:00
Evan Downing 7c71f0f11e Add additional tests for each existing challenge (#620) 2025-05-19 13:16:57 -04:00
Evan Downing 5c1b2486fc Move challenge fixtures to conftest. (#569) 2025-05-09 16:14:06 -04:00
Evan Downing dd7c9c21e7 Locate functions in java stacktrace (#460)
* Uncomment skipped tests

* Codequery was already working for bouncycastle.

* Add support for copying src after build
2025-05-01 13:12:44 -04:00
Michael D Brown 83fe3eb8b9 Add CRS Action telemetry (#462)
Adds telemetry for all services including required and some extra attributes.
2025-04-29 14:48:37 -04:00
Evan Downing ef323e20e9 Remove logger calls. Redirect subprocess run outputs (#458) 2025-04-25 07:49:57 -04:00
Evan Downing 93f55a274b Add support for inner classes (#443)
* Change cscope call to compress database and improve lookup speed. Update tests.

* Fix log4j2 source paths

* Add private version of cscope
2025-04-25 07:37:39 -04:00
Riccardo Schirone 2475b8e507 Move to use in-container paths (e.g. /src/) (#395)
* disable kythe/graphdb in tests
* fix integration CI
2025-04-16 11:05:44 +02:00
Evan Downing a754c94c82 Add new regression tests for c/java (#378)
* Add zookeeper test file

* Update java function query.

* Add caller/callee queries

* Add example test for C with fixture

* Refactor challenge task creation in tests

---------

Co-authored-by: Boyan MILANOV <boyanmilanov@coder-boyanmilanov-aixcc-boyan.c.production-1-405717.internal>
2025-04-10 10:09:35 +03:00
Evan Downing c47f27debe Speedup graphdb (#193)
* Add bulk loading and update schema

* Remove unnecessary config options for speedup

* Uncomment unit test. Update integration test.

* Fix permission issue with removing temp folder

* Update file paths

* Changing graphdb directory

* Update integration config
2025-03-25 09:38:47 -04:00
Riccardo Schirone b44c21dd6e program-model: fix task-id property and improve tests (#174)
* program-model: fix task-id property and improve tests

* Skipping libpng integration test until kythe is ready

---------

Co-authored-by: Evan Downing <2077950+evandowning@users.noreply.github.com>
2025-03-12 15:20:39 -04:00
Riccardo Schirone e0b898a6b4 program-model: introduce codequery (#178)
* program-model/tree-sitter: refactor classes

* program-model/tree-sitter: change how paths are considered

This allows to parse also code in oss-fuzz (e.g. injected fuzz targets
not in upstream project)

* common: change name format for challenge commit

* program-model/codequery: make it persistent and nice

* program-model: disable janus & co for now

* patcher: make use of codequery instead of just tree-sitter

* program-model: add integration tests for codequery

* program-model/codequery: add support for different languages

* patcher: install codequery in patcher's container

* ci: fix codequery install

* program-model: ensure cqdb storage in /crs_scratch

* program-model: ensure the cqdb storage is available only at the end
2025-03-10 14:11:07 +01:00