This PR introduces a NodeLocal storage concept. Data is always processed on the local node, and only replicated to the azure file storage as needed.
A DaemonSet is introduced that syncs any corpus on the local node with the global corpus directory.
The following path structure is introduced:
/node_data/crs_scratch
/node_data/tasks_storage
/node_data/scratch
The first two directories is the equivalent of the previous usage of
/crs_scratch
/tasks_storage
The scratch is for basing temporary files and directories on the same filesystem as node-local version to be able to do atomic renames.
Further, ChallengeTask data is used normally on the /node_data/... paths, but is stored as /tasks_storage/{uuid}.tgz on the "remote" storage. This is to speed up copying.
Corpus and CrashSets are synced as is.
* Copy signoz deployment from example-crs-architecture
* Make ports local and remove unused clickhouse config
* Upload seed-gen LLM traces to SigNoz with OTel
* Add telemetry to patcher and seed-gen with common function
* Update README and lockfiles
* Implements configurable authentication
This removes the hardcoded auth and introduces a tool for generating the
required keys/tokens/tokenhash values.
buttercup-auth-tool --env
this will output all information needed to configure the server and
client side.
* lint
* Use new auth in ci
* patcher: use pydantic basemodel
* patcher: restructure to use Command
* patcher: remove mock mode because not needed anymore
* remove also env var in docker compose for mock mode
* patcher: remove filtering for now
* patcher: remove | None to fields with default
Henrik Brodin