17 Commits

Author SHA1 Message Date
Giacomo Vercesi dfb9a36bb2 pypeline: implement compression
Implement compression of objects before they are saved into the storage
provider. Each container type can specify which algorithm to use
(currently `none` or `zstd`) and the compression level.
2026-06-17 10:06:35 +02:00
Giacomo Vercesi 20059b5f3a pypeline: make Container.serialize selective
Change the `Container.serialize` interface so that it is possible to
supply a list of objects that will be serialized instead of all the ones
in the container.
2026-03-31 17:00:48 +02:00
Giacomo Vercesi 14d95a924d pypeline: rework fields in pipeline metadata
Rework which information is transmitted in the pipeline metadata,
avoiding redundancy and moving some information there instead of
returning it every time a request is made.
2026-03-31 17:00:48 +02:00
Giacomo Vercesi 7f7f9571a8 pypeline: implement container discarding
Add infrastructure to pypeline that allows containers to be notified
when they are being used last, this allows two things:
* `Pipe`s eagerly clearing those containers once they are done reading
  their contents
* `ScheduledTask`s clearing those out at the end of their execution in
  case the pipe did not do it
This overall should improve memory usage as container no longer take up
memory if they are no longer used as part of a `Schedule`.
2025-12-19 10:29:28 +01:00
Giacomo Vercesi cd3cd5c4d1 pypeline: fix Request.check
The function `Request.check` did not work properly due to
`Container.contains_all` performing the check with the operands swapped.
Fix the call and improve the body of `ObjectSet.issubset`.
Also improve the error message, highlighting the missing objects.
2025-12-10 15:05:53 +01:00
Giacomo Vercesi 4a1edec018 get_registry: use name for selected classes
Use the `name` class attribute to build the registry when discovering
subclasses of `Pipe`, `Analysis` and `Container`.
2025-11-26 14:59:43 +01:00
Tommaso Fontana 91299b0c18 Pypeline: change daemon pipeline format
Now the dameon's pipeline endpoint doesn't return just the parsed yaml
but it returns a format that's easier to use for the UI.
2025-11-24 16:41:15 +01:00
Tommaso Fontana 4d25ba057d Pypeline: add yaml and tar format for containers
Now containers can be dump and loaded in both yaml and tar. Also the
container can automatically try to figure out the format of the
container. Moreover, now containers have to_bytes and from_bytes.
2025-11-24 16:41:15 +01:00
Giacomo Vercesi c0e35d080e ContainerDeclaration: enable order
`ContainerDeclaration` needs to be orderable (e.g. have `__lt__`) to
allow `deterministic_sort_key` to work.
2025-11-17 10:04:12 +01:00
Tommaso Fontana 0ad8e8c849 Pypeline: add pypeline.daemon
Add implementation of the generic pype daemon.
It's implementation is split into `daemon.py` where we implement an
http framework agnostic interface, and `app.py` that uses it to serve
them using starlette.
2025-11-11 17:15:28 +01:00
Tommaso Fontana 71fad8b361 Pypeline: replace load_container with from_file 2025-11-11 17:15:28 +01:00
Tommaso Fontana 639593164a Pypeline: add autocomplete and refactor CLI
Now pype has an `autocomplete` command usable to enable autocompletion
and revng2 now is based on `pype`, modifying its defalults and
injecting new commands.
2025-11-11 17:15:27 +01:00
Tommaso Fontana 2daa9f938e Pypeline: add mime_type and model_name to Model
Now the model has a mime_type that we expect to return to the UI, and
model_name so the name of the model on disk can be impl-specific.
Moreover, now Model and Container use the same logic to detect if
a mimetype is textual.
2025-11-11 17:15:27 +01:00
Tommaso Fontana 652ffbb0f7 Pypeline: remove abbreviations, normalize quotes 2025-11-11 17:15:27 +01:00
Giacomo Vercesi 0f7dc7047d pypeline.Container: fix signature of (de)serialize
Change the signature of `Container.serialize` and
`Container.deserialize` to use `Buffer` instead of `bytes`, as this
makes the functions compatible with the `revng::pypeline::Buffer`
classes.
2025-10-16 16:08:05 +02:00
Giacomo Vercesi 688b9fe111 Introduce compatibility with revng-pypeline
Add the necessary machinery to allow `Pipe`s, `Analysis`es,
`Container`s, `Model` and `ObjectID` to be implemented in C++ and used
by Python.
2025-09-17 14:02:40 +02:00
Tommaso Fontana 80eb02c207 Pypeline: merge pypeline 2025-09-10 12:05:15 +02:00