7 Commits

Author SHA1 Message Date
Giacomo Vercesi dfb9a36bb2 pypeline: implement compression
Implement compression of objects before they are saved into the storage
provider. Each container type can specify which algorithm to use
(currently `none` or `zstd`) and the compression level.
2026-06-17 10:06:35 +02:00
Giacomo Vercesi 2fabd9c400 pypeline: split Pipe.invalidate in two
Split the `invalidate` method in two phases: in `invalidateCheck` it
just checks if the model diff warrants the execution of the actual
`invalidate` method, which requires fetching the custom invalidation
data from storage. The actual `invalidation` method remains the same.
This should make invalidation faster for storage providers that store
the invalidation remotely.
2026-05-22 09:04:35 +02:00
Giacomo Vercesi 14d95a924d pypeline: rework fields in pipeline metadata
Rework which information is transmitted in the pipeline metadata,
avoiding redundancy and moving some information there instead of
returning it every time a request is made.
2026-03-31 17:00:48 +02:00
Giacomo Vercesi c53e615fa0 Cleanup pypeline code
Remove some cruft and apply trivial changes to the existing pypeline
code, especially on the daemon side.
2026-03-31 17:00:48 +02:00
Giacomo Vercesi ce60d8a1e5 pypeline: integrate model migration infrastructure
When calling `Model.deserialize` the model will be automatically be
migrated if it needs to.
2026-03-05 14:23:06 +01:00
Giacomo Vercesi f4d0573017 pypeline: StorageProvider works with Model
Change the `get_model` and `set_model` interface of `StorageProvider` so
that it is responsibility of the `StorageProvider` to
serialize/deserialize the model before returning to the caller.
This is in preparation to the model migration being implemented, since
it's now a responsibility of the storage provider to deserialize it it
can trivially re-save it to disk if it is migrated.
2026-03-05 14:23:06 +01:00
Giacomo Vercesi df4f8b76ac pypeline: re-organize tests
Split the tests into two subdirectories: those that work in pure python
and those that use the C++ bindings.
2026-02-16 10:34:39 +01:00