mirror of
https://github.com/trailofbits/buttercup
synced 2026-06-21 14:11:39 +00:00
Riccardo Schirone
e0b898a6b4
* program-model/tree-sitter: refactor classes * program-model/tree-sitter: change how paths are considered This allows to parse also code in oss-fuzz (e.g. injected fuzz targets not in upstream project) * common: change name format for challenge commit * program-model/codequery: make it persistent and nice * program-model: disable janus & co for now * patcher: make use of codequery instead of just tree-sitter * program-model: add integration tests for codequery * program-model/codequery: add support for different languages * patcher: install codequery in patcher's container * ci: fix codequery install * program-model: ensure cqdb storage in /crs_scratch * program-model: ensure the cqdb storage is available only at the end
Program Model
Indexes a program into a graph database.
Requirements
n2-highmem-8 instance.
sudo apt-get install -y codequery
Setup
Set up working directory and tasks directory.
cd afc-crs-trail-of-bits/
cp env.template .env
sudo mkdir /crs_scratch/ && sudo chown `whoami`:`whoami` /crs_scratch && sudo mount --bind ./crs_scratch /crs_scratch
sudo mkdir /tasks_storage && sudo chown `whoami`:`whoami` /tasks_storage && sudo mount --bind ./tasks_storage /tasks_storage
Set up Github token.
gh auth login
GitHub.com
SSH
No
Paste the authentication token
gh auth token | docker login ghcr.io -u USERNAME --password-stdin
Build the Kythe docker image and push to aixcc-finals.
Create your own personal PAT, and give it write:packages permissions -- do not configure SSO for any organization.
Log into ghcr.io like above.
git clone git@github.com:trailofbits/aixcc-kythe.git
cd aixcc-kythe/
# Takes roughly 70 minutes to build and 15 minutes to copy the .tar.gz file.
docker build -t aixcc-kythe -f aixcc.Dockerfile .
docker tag aixcc-kythe ghcr.io/aixcc-finals/buttercup-kythe:main
docker push ghcr.io/aixcc-finals/buttercup-kythe:main
You should see the buttercup-kythe package here.
Usage
Prepare example challenge project.
cd afc-crs-trail-of-bits/
./orchestrator/scripts/task_crs.sh
Start up CRS.
cd afc-crs-trail-of-bits/
docker compose up -d --build --remove-orphans
Send task to Program Model via Redis queue.
docker compose logs -f program-model
cd afc-crs-trail-of-bits/program-model/
uv run mock/trigger_pm.py \
--build_type full \
--package_name libpng \
--sanitizer AddressSanitizer \
--task_dir ../tasks_storage/5cea8f59-a7ab-4c77-97a9-f92fcfeb33d8 \
--task_id libpng
API
See api.md.
Testing on Challenges
See challenges.md.
Development
Sync, reformat, lint, and test before committing changes to this directory:
just all
JanusGraph Indexing References
- https://user3141592.medium.com/single-vs-composite-indexes-in-relational-databases-58d0eb045cbe
- https://docs.janusgraph.org/schema/index-management/index-performance/
- https://docs.janusgraph.org/schema/schema-init-strategies/
- https://docs.janusgraph.org/configs/configuration-reference/#schema
- https://docs.janusgraph.org/v0.3/basics/schema/#:~:text=.util.UUID%20)-,Property%20Key%20Cardinality,all%20elements%20in%20the%20graph.
FAQs
- Why does this use a ubuntu base image?
- Since Kythe uses OSS Fuzz to build and index the challenge source code, we have to use the same base image as ClusterFuzz.
- Why does this use Python 3.10?
- ClusterFuzz uses Python 3.10.