Commit Graph

22 Commits

Author SHA1 Message Date
Riccardo Schirone e0b898a6b4 program-model: introduce codequery (#178)
* program-model/tree-sitter: refactor classes

* program-model/tree-sitter: change how paths are considered

This allows to parse also code in oss-fuzz (e.g. injected fuzz targets
not in upstream project)

* common: change name format for challenge commit

* program-model/codequery: make it persistent and nice

* program-model: disable janus & co for now

* patcher: make use of codequery instead of just tree-sitter

* program-model: add integration tests for codequery

* program-model/codequery: add support for different languages

* patcher: install codequery in patcher's container

* ci: fix codequery install

* program-model: ensure cqdb storage in /crs_scratch

* program-model: ensure the cqdb storage is available only at the end
2025-03-10 14:11:07 +01:00
Henrik Brodin 0a7bac450e Henrik/indexing integration (#130)
Integration of the indexer service
2025-03-05 13:56:31 +01:00
2over12 a36fa80288 Switch to getting kzips using compdb fragments (#120)
* swtich to frags

* or true

* format

* add task id
2025-03-04 15:43:06 -05:00
Evan Downing d3f3cd3765 Add graphdb indexing (#114)
* Add indexes for fields

* Add indexing schema
2025-03-04 10:51:53 -05:00
Evan Downing e42b60e820 Add task id to graph (#94)
* Add task id field to nodes and edges

* Fix properties key
2025-03-04 10:51:35 -05:00
Evan Downing 0bfdfa2d7d Add get_function_body unit test (#96)
* Add get_function_body unit test

* Test passes

* Add janusgraph service

* Update tests with healthcheck
2025-03-04 08:09:25 -05:00
Evan Downing 2e5191b513 Build kythe into its own container (#104)
* Remove environment variables

* Add kythe release from base image

* Copy file from remote container

* Change kythe to remote

* x

* Fix dockerfile
2025-03-04 07:45:23 -05:00
Henrik Brodin 65dd397115 Store task-id in task-meta file (#123)
Store task-id in task-meta file
2025-03-04 13:00:13 +01:00
Evan Downing de724332ef Add try catches to program_model.py (#113) 2025-03-03 11:35:06 -05:00
Evan Downing f4446543ae Get function body by name (#73)
* Add skeleton for api call

* Modify encoding and decoding nodes/edges

* Add change encoding from base64 to hex

* reformat

* Target libpng function in example

* remove old project name

* cli fixes

* Add get_function via byte offsets

* refactor

---------

Co-authored-by: Ian Smith <ian.smith@trailofbits.com>
2025-02-25 10:21:08 -05:00
Riccardo Schirone 2d852a0881 program-model: add tree-sitter parsing for fallbacks (#90) 2025-02-24 09:58:21 +01:00
Evan Downing f6067005e6 Download Kythe automatically (#84)
* Add passing gh token

* Change docker to automatically download kythe release

* Update workflow
2025-02-21 13:33:38 -05:00
Riccardo Schirone 5b1ce401c2 Serve loop (#78)
* patcher: do not use package_name anymore
* patcher: fix tests
2025-02-20 14:26:42 -05:00
Henrik Brodin fe07435b77 Create the TaskMeta structure in the task directory (#79)
* Create the TaskMeta structure in the task directory

it contains information about project name and focus.

* lint

* Address review

* Restore name property
2025-02-20 03:48:23 +01:00
Evan Downing c6892939dc Updates program model directory (#66)
* Modify protobuf

* Update directory parameters for program model

* Update directory calls and build image call

* Always apply diff

* Always apply diff
2025-02-19 15:59:40 -05:00
Evan Downing 90d40ba4d1 Make graphdb storage persistent (#58)
* Add graphdb storage volume

* correct logging

* update volume
2025-02-18 18:02:00 -05:00
2over12 d4c9f74294 Enforce python >=3.10 and use ignore_cleanup_errors in TemporaryDirectory (#2)
* Enforce python >=3.10 and use ignore_cleanup_errors in TemporaryDirectory

* have an upper bound on requires-python

* update uv.locks

---------

Co-authored-by: Riccardo Schirone <riccardo.schirone@trailofbits.com>
2025-02-18 17:56:28 -05:00
Evan Downing 7b15a06214 change root of docker (#59) 2025-02-18 17:06:39 -05:00
Evan Downing af0ceb9e46 Program Model API (#25)
* Add doc skeletons

* Add API skeleton example

* Move graphdb url in settings
2025-02-18 17:06:15 -05:00
2over12 ea2fd093d5 Program model integrate (#13)
* Add documentation

* Add janusgraph custom configuration. Simply graph creation.

* Add properties to edges. Index on uri property.

* reformat

* Create libpng graph in a more reasonable amount of time.

* Update README.md

* Wrap indexing into function

* updating...

* update proto

* fix command

* Update settings for serve and processing indexing

* Add dockerfile to docker container

* Add trigger script

* Fix error with loading graphml file

* reformat

* Remove resource constraints for program model

* Change permissions of temp dir

* Update readme

* Update dockerignore

---------

Co-authored-by: Evan Downing <2077950+evandowning@users.noreply.github.com>
2025-02-18 11:59:16 -05:00
Riccardo Schirone 9885795582 program-model: add empty test to make pytest happy 2025-02-05 10:23:56 +01:00
2over12 c26a3e223d Add program model demo (#17)
* add program model uv

* add tool to common

* remove oss fuzz tool

* add docker

* implement build base image

* fix ret check

* add builder for subimage

* remove old tool

* hack to avoid dupped kzip files

* fixes... need to work out LD hack

* fix find

* justfile for building kythe

* update dockerfile

* add upload tool

* update protobuf

* move into compose

* fix merge invocation

* fix up invoc

* bump kythe

* reformat

* format

* format

* remove unused

* add lint and change version

* fixes

* exclude

* fix formatting:

* dev deps

* add dep

* exclude gen

* fix
2025-01-26 13:09:37 -05:00