mirror of
https://github.com/trailofbits/buttercup
synced 2026-06-21 14:11:39 +00:00
Dan Guido
37597105ed
* fix: respect OSS_FUZZ_CONTAINER_ORG environment variable The oss_fuzz_container_org property now checks the OSS_FUZZ_CONTAINER_ORG environment variable first before falling back to parsing the helper file. This fixes the failing test_container_image_custom_org integration test that was caught in nightly CI. The test was expecting that setting OSS_FUZZ_CONTAINER_ORG=myorg would result in container images using that organization, but the code was ignoring the environment variable entirely. * fix: standardize Python packaging and project configuration - Fix critical Python version inconsistency (common was 3.10+, dependents required 3.12+) - Standardize project metadata: add descriptions, licenses, consistent author emails - Implement consistent dependency management using compatible release (~=) strategy - Modernize all components to use [project.optional-dependencies] instead of [dependency-groups] - Restore essential tool configurations (ruff lint rules, pytest settings, coverage config) - Remove redundant component Makefiles (orchestrator, program-model) - Add project URLs for better package discoverability This resolves packaging inconsistencies introduced in PR #271 while maintaining modern Python packaging standards. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com> * fix: restore accidentally removed Makefiles with unique functionality - Restore orchestrator/Makefile: contains unique API management targets (update-apis, generate-competition-api) - Restore program-model/Makefile: contains specific integration test commands not in root Makefile - These Makefiles provide component-specific functionality not available elsewhere * fix: standardize program-model Dockerfile to use Python 3.12 Ensures consistency with pyproject.toml requirement of Python >=3.12,<3.13. This aligns with the other components and standardizes Python version across all Dockerfiles in the project. * refactor: improve Dockerfile consistency and layer caching - Combine consecutive apt operations to reduce layers - Add missing DEBIAN_FRONTEND=noninteractive declarations - Ensure all apt operations include cleanup with rm -rf /var/lib/apt/lists/* - Reduces image size and improves build consistency * fix: use [dependency-groups] instead of [project.optional-dependencies] Addresses reviewer feedback that [dependency-groups] is the semantically correct approach for development dependencies like test, lint, and typing tools. Per PEP 735 and packaging.python.org guidance: - [dependency-groups] for development workflow dependencies - [project.optional-dependencies] for optional user-facing features This standardizes all components to use the modern PEP 735 approach. * fix: move dependencies from [project.urls] to [project] section Fixes TOML syntax error where dependencies array was incorrectly placed under [project.urls] instead of [project], causing build failures: 'TypeError: URL of field must be a string' This resolves CI failures across all components during uv sync. * fix: temporarily disable ruff rules to resolve CI failures Temporarily disabled the following ruff rules for standardization PR: - I001: Import block is un-sorted or un-formatted - E501: Line too long - W291: Trailing whitespace - UP006: Use built-in collection types for type annotations - UP015: Unnecessary mode argument - UP035: Import from modern locations instead of deprecated typing - UP046: Use modern generic class syntax These rules were disabled in common/ and fuzzer/ components where they were causing CI failures. Rules are commented with intention to re-enable after refactoring work is complete. Also fixed ruff formatting issues in seed-gen component. * refactor: simplify dependency groups per maintainer feedback Address maintainer feedback by consolidating dependency groups: - Combined dev/test/typing/lint groups into single practical 'dev' group - Ensures mypy gets all required type stubs (no more missing dependencies) - Eliminates duplication and reduces cognitive overhead - Users now only need: uv sync --group dev Also removed pytest configuration from common/ as requested, since most settings were defaults and only existed in one component. Changes provide better developer experience with simpler, working dependency management. * fix: add missing UP045 rule to ruff ignore lists Addresses CI failure: 'UP045 Use X | None for type annotations' Added UP045 to ruff ignore lists in: - common/pyproject.toml - fuzzer/pyproject.toml - orchestrator/pyproject.toml - seed-gen/pyproject.toml This completes the temporary rule disabling for the standardization PR. UP045 enforces modern union syntax (X | None vs Optional[X]) - will re-enable after refactoring. * fix: correct argon2-cffi version constraint in orchestrator Changes argon2-cffi from ~=21.0.0 to ~=21.3.0 to resolve dependency resolution failure. Version 21.0.0 never existed on PyPI - available versions jump from 20.1.0 directly to 21.1.0. This restores the previously working constraint and resolves: 'No solution found when resolving dependencies: argon2-cffi>=21.0.0,<21.1.dev0' * chore: update mypy to latest version 1.17.1 Updates mypy from 1.15.0 to 1.17.1 across all components to ensure we're using the latest type checker features and bug fixes. ruff is already on the latest version (0.12.8). This keeps the linting tools current and prevents them from becoming outdated over time. * fix: add necessary ruff ignore rules to orchestrator Add the specific ruff rules that orchestrator needs disabled based on CI lint failures. Other components passed linting, so only orchestrator needs these additional rules. Rules added: - I001: Import block formatting - E501: Line too long - UP006: Built-in collection types - UP007: Union type annotations - UP009: UTF-8 encoding declarations - UP015: Unnecessary mode argument - UP035: Modern import locations 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com> * feat: add missing ruff config and project metadata - Add ruff configuration to program-model for consistency - Add project URLs to 4 components for discoverability - Add types-redis to fuzzer dev dependencies for type checking - Ensure all components have consistent ignore rules * fix: add additional ruff ignore rules for patcher and program-model - Add W293, UP012, UP031 to patcher ignore list - Add UP032 to program-model ignore list - Ensures ruff checks pass for both components * fix: resolve ruff formatting and dependency issues - Format all program-model files with ruff (28 files reformatted) - Standardize patcher dependencies to use ~= version specifiers - Update langgraph-checkpoint to ~=2.1.0 to resolve conflicts - Fix Docker PYTHON_VERSION variable usage in patcher/Dockerfile - Update patcher lock file with new dependency constraints * Update tree-sitter-language-pack and refresh uv.lock --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Michael D Brown <michael.brown@trailofbits.com> Co-authored-by: Riccardo Schirone <riccardo.schirone@trailofbits.com>
60 lines
2.0 KiB
Docker
60 lines
2.0 KiB
Docker
ARG BASE_IMAGE=gcr.io/oss-fuzz-base/base-runner
|
|
|
|
FROM $BASE_IMAGE AS base-image
|
|
|
|
COPY --from=ghcr.io/astral-sh/uv:0.5.20 /uv /uvx /bin/
|
|
|
|
ENV UV_LINK_MODE=copy
|
|
ENV UV_COMPILE_BYTECODE=1
|
|
ENV UV_PYTHON_DOWNLOADS=manual
|
|
|
|
RUN uv python install python3.12
|
|
|
|
FROM base-image AS runner-base
|
|
RUN apt-get update
|
|
# TODO(Ian): maybe we should have a different base image for the builder
|
|
RUN apt-get install -y ca-certificates curl
|
|
RUN install -m 0755 -d /etc/apt/keyrings
|
|
RUN curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
|
|
RUN chmod a+r /etc/apt/keyrings/docker.asc
|
|
|
|
# Add the repository to Apt sources:
|
|
RUN echo \
|
|
"deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
|
|
$(. /etc/os-release && echo "${UBUNTU_CODENAME:-$VERSION_CODENAME}") stable" | \
|
|
tee /etc/apt/sources.list.d/docker.list > /dev/null
|
|
RUN apt-get update
|
|
RUN apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
|
|
|
|
FROM base-image AS builder
|
|
|
|
WORKDIR /fuzzer
|
|
|
|
# Install dependencies
|
|
RUN --mount=type=cache,target=/root/.cache/uv \
|
|
--mount=type=bind,source=common/uv.lock,target=/common/uv.lock \
|
|
--mount=type=bind,source=common/pyproject.toml,target=/common/pyproject.toml \
|
|
--mount=type=bind,source=common/README.md,target=/common/README.md \
|
|
--mount=type=bind,source=fuzzer/uv.lock,target=/fuzzer/uv.lock \
|
|
--mount=type=bind,source=fuzzer/pyproject.toml,target=/fuzzer/pyproject.toml \
|
|
cd /fuzzer && uv sync --frozen --no-install-project --no-editable
|
|
|
|
ADD ./common /common
|
|
ADD ./fuzzer /fuzzer
|
|
|
|
|
|
RUN --mount=type=cache,target=/root/.cache/uv \
|
|
uv sync --frozen --no-editable
|
|
|
|
FROM runner-base AS runtime
|
|
|
|
RUN DEBIAN_FRONTEND=noninteractive apt-get update && \
|
|
apt-get install -y git \
|
|
&& rm -rf /var/lib/apt/lists/*
|
|
|
|
COPY --from=builder --chown=app:app /fuzzer/.venv /fuzzer/.venv
|
|
COPY common/container-entrypoint.sh /container-entrypoint.sh
|
|
ENV PATH=/fuzzer/.venv/bin:$PATH
|
|
|
|
ENTRYPOINT ["/container-entrypoint.sh"]
|