trailofbits-buttercup/guides/SCORED.md

Scored

This document describes how to parse logs from the final round of the AIxCC competition.

NOTE: PoVs were not released to us immediately because DARPA is still involved in responsible disclosure: https://aicyberchallenge.com/Finals-winners-announcement/.

Data

Download the following files from the competition to buttercup/data/

• litellm_backup.sql
• redis-backup.rdb
• task-storage.tar

Get Task IDs for Specific Projects

For example, find the Task IDs for libpng.

cd data/
../scripts/results_unpack.sh
../scripts/results_find.sh libpng

Parse

Query the CRS database for submissions. Documentation can be found at https://redis.io/learn/explore/import#restore-an-rdb-file.

Let's assume a task ID 0197b76a-b429-78fa-8d0e-ad84994c8f44 was outputted from above.

1. Start redis-server in data/ directory

   cd data/
   redis-server
   

2. Print directory to confirm

   redis-cli config get dir
   1) "dir"
   2) "<path/to/data>"
   

3. Stop redis-server and check if dump.rdb exists in data/ directory.

4. Copy .rdb file

   cp redis-backup.rdb dump.rdb
   

5. Start redis-server and query to make sure it has data.

   redis-server
   redis-cli keys \*
   

6. Parse database for task id

   cd common/
   uv sync --all-extras
   uv run buttercup-util read_submissions --task_id 0197b76a-b429-78fa-8d0e-ad84994c8f44 --verbose True > ../data/submissions.txt
   

7. Read through submissions.txt for information like crash harnesses, stacktraces, and patches.