Improve the `schedule` in the `Pipeline` class by introducing two
optimizations:
* Prune all the `ContainerDeclarations` that are not actually used.
* Skip all tasks that have no outgoing elements that would have been
written by them.
Allow the `configuration` field of a pipe in a pipeline yaml file to be
of any type. When constructing the pipe object, the object will be
re-serialized via `yaml.dump` before being passed to the constructor.
Move the epoch inside the storage_provider, this allows us to make the
storage_provider_factory to return an async context manager.
This way we can simplify the locking for the daemon, but forces the
CLI project commands to deal with async code.
Now the pype command has a `--verbose` argument that to enable debug
logging in the pypelien code.
Adapted the codebase to use this new logging format but replacing
`logging` with `revng.pypeline.utils.logger`, this is done in
preparation of debug-log.
Add implementation of the generic pype daemon.
It's implementation is split into `daemon.py` where we implement an
http framework agnostic interface, and `app.py` that uses it to serve
them using starlette.
Now pype has an `autocomplete` command usable to enable autocompletion
and revng2 now is based on `pype`, modifying its defalults and
injecting new commands.
Now the model has a mime_type that we expect to return to the UI, and
model_name so the name of the model on disk can be impl-specific.
Moreover, now Model and Container use the same logic to detect if
a mimetype is textual.
Introduce the `ImportFiles` pipe which leverages the newly-introduced
`FileStorage` to fetch the required files and store them in the
`BinariesContainer`.
Change the signature of `Container.serialize` and
`Container.deserialize` to use `Buffer` instead of `bytes`, as this
makes the functions compatible with the `revng::pypeline::Buffer`
classes.
The `serialize` function of `revng::pypeline::Model` erroneously
returned a buffer instead of `bytes`, fix the function so that `bytes`
are returned instead.
When click calls the command handler functions, dashes are converted to
underscores to work with the variable name limitations of python. When
accessing kwargs, convert the argument name to the underscored version.
When calling `StorageProvider.invalidate`, return a dictionary that
conveys to the caller which objects were deleted from storage as a
consequence of the paths supplied to the `invalidate` call.
Fix the invalidation logic in both `InMemoryStorageProvider` and
`Sqlite3StorageProvider`, as it did not account for propagating the
invalidation to both children objects and parent objects.
The column stored read model paths was erroneously called
`model_path_hash`, while the entire model was stored instead of a hash.
Rename the column to just `model_path` to avoid any confusion.
Add the needed infrastructures that allow ephemeral classes (`PipeRun`s)
to be wrapped as a regular pypeline `Pipe` class. These automatically
have their dependencies tracked and propagated to the pypeline.
Fix the `_load_model` method of `CLIProject`. Before this commit it
erroneously did not call `_set_model` on all codepaths, resulting in
cases where `self.model` was incorrectly missing the `_project` field.
Handle the case in `revng daemon` where the `PipelineManager` fails to
init. In this scenario the server responds to all requests with a 500
(with the exception of `/` and `/status`, which return 200).
Giacomo Vercesi