12 Commits

Author SHA1 Message Date
Giacomo Vercesi dfb9a36bb2 pypeline: implement compression
Implement compression of objects before they are saved into the storage
provider. Each container type can specify which algorithm to use
(currently `none` or `zstd`) and the compression level.
2026-06-17 10:06:35 +02:00
Giacomo Vercesi f0144ac0c9 pypeline: make artifact_categories kebab-case 2026-06-17 09:43:07 +02:00
Giacomo Vercesi bbff447181 pypeline: split ScheduledTask
Due to the drifting differences, split `ScheduledTask` into
`PipeScheduledTask` and `SavepointScheduledTask`, which allows each to
specialize on the task it has to accomplish.
2026-05-22 09:04:35 +02:00
Giacomo Vercesi b7f914d53f StorageProvider: move invalidate into set_model
Make `set_model` perform both the invalidation and the changing of the
model bytes. This makes persisting the result of an analysis to storage
atomic.
2026-05-22 09:04:35 +02:00
Giacomo Vercesi dbc734f23a CMake: force use of Python3
Switch from using `find_package(Python ...)` to `find_package(Python3
...)` as it excludes that CMake picks a Python 2 version by accident.
2026-05-14 11:26:41 +02:00
Giacomo Vercesi 18e3501731 pypeline: implement proper config parsing
Overhaul the pipeline configuration logic by collapsing all dynamic
configuration options, both for analyses and pipes into a single
dictionary. Change all the interfaces so that there is no longer
distinction between the configuration of an analysis and of pipes.
Expose these options to the command line via `--{name}-configuration`
options for each pipe/analysis that is applicable to the command-line
invocation.
2026-04-10 11:45:12 +02:00
Giacomo Vercesi 4c6436f7b2 pypeline: introduce artifact category
Add categories in the pipeline, allow an artifact to have a category
specified which allows it to be shown or hidden by default. Adapt the
CLI tools to hide artifacts of the category that don't
`show_by_default=True`.
2026-03-31 17:00:48 +02:00
Giacomo Vercesi 39f1a9c3a9 pypeline: drop underscores from pipeline yaml
Remove the only instance of underscores in the pipeline yaml from the
use of `analysis_lists`. Substitute it with `analysis-list`.
2026-03-31 17:00:48 +02:00
Giacomo Vercesi f4d0573017 pypeline: StorageProvider works with Model
Change the `get_model` and `set_model` interface of `StorageProvider` so
that it is responsibility of the `StorageProvider` to
serialize/deserialize the model before returning to the caller.
This is in preparation to the model migration being implemented, since
it's now a responsibility of the storage provider to deserialize it it
can trivially re-save it to disk if it is migrated.
2026-03-05 14:23:06 +01:00
Giacomo Vercesi a6a3e15e6b pypeline: add env prefixes
Add the `PYPELINE_` prefix to all the pypeline environment variables.
2026-03-05 14:23:06 +01:00
Giacomo Vercesi e14d1cee6f Add native runner tests 2026-02-16 10:34:39 +01:00
Giacomo Vercesi df4f8b76ac pypeline: re-organize tests
Split the tests into two subdirectories: those that work in pure python
and those that use the C++ bindings.
2026-02-16 10:34:39 +01:00