Before this commit the pipeline description suffered from some
non-determinism that led to some values changing between runs of
`revng`. Add extra sorting so to eliminate the non-determinism.
Consolidate the saving of object dependencies and saving the actual
objects' data into a single function called `add_objects`. The previous
methods `add_dependencies`, `add_custom_invalidation_data` and `put`
have been removed.
Due to the drifting differences, split `ScheduledTask` into
`PipeScheduledTask` and `SavepointScheduledTask`, which allows each to
specialize on the task it has to accomplish.
Make the issuing of notification completely out of band with respect to
the `Pipeline`. Now the `StorageProvider` is responsible for providing
notifications to clients. Since some providers are local-only, there is
a `LOCAL_QUEUE` which allows sensing notifications through a local
queue, re-using the local revng daemon.
Split the `invalidate` method in two phases: in `invalidateCheck` it
just checks if the model diff warrants the execution of the actual
`invalidate` method, which requires fetching the custom invalidation
data from storage. The actual `invalidation` method remains the same.
This should make invalidation faster for storage providers that store
the invalidation remotely.
Overhaul the pipeline configuration logic by collapsing all dynamic
configuration options, both for analyses and pipes into a single
dictionary. Change all the interfaces so that there is no longer
distinction between the configuration of an analysis and of pipes.
Expose these options to the command line via `--{name}-configuration`
options for each pipe/analysis that is applicable to the command-line
invocation.
Add an additional piece of metadata that states which pipes are used to
compute a specific artifact. This can be used to derive which
configuration options influence the creation of an artifact.
Add two additional fields to `Artifact`: `defined_locations` and
`preferred_artifacts`. These are pieces of metadata that allow
navigation between multiple PTML-enabled documents.
Add categories in the pipeline, allow an artifact to have a category
specified which allows it to be shown or hidden by default. Adapt the
CLI tools to hide artifacts of the category that don't
`show_by_default=True`.
Rework which information is transmitted in the pipeline metadata,
avoiding redundancy and moving some information there instead of
returning it every time a request is made.
Change the `get_model` and `set_model` interface of `StorageProvider` so
that it is responsibility of the `StorageProvider` to
serialize/deserialize the model before returning to the caller.
This is in preparation to the model migration being implemented, since
it's now a responsibility of the storage provider to deserialize it it
can trivially re-save it to disk if it is migrated.
Add the functionality the the pipeline infrastructure and CLI to run
individual pipe and analyses as subcommands instead of in-process. This
allow better debuggability of individual pipes.
Convert the `RuntimeError`s emitted by the pipebox (e.g. when checking
preconditions or running analyses) into `PypelineError`s and have the
pypeline cli infrastructure handle them gracefully, without emitting
stacktraces.
Overload the `get` function of `Requests` so that, in addition to the
usual `Mapping` behavior, will return an empty `ObjectSet` if no default
is specified.
Improve the `schedule` in the `Pipeline` class by introducing two
optimizations:
* Prune all the `ContainerDeclarations` that are not actually used.
* Skip all tasks that have no outgoing elements that would have been
written by them.
Now the pype command has a `--verbose` argument that to enable debug
logging in the pypelien code.
Adapted the codebase to use this new logging format but replacing
`logging` with `revng.pypeline.utils.logger`, this is done in
preparation of debug-log.
Add implementation of the generic pype daemon.
It's implementation is split into `daemon.py` where we implement an
http framework agnostic interface, and `app.py` that uses it to serve
them using starlette.
Now pype has an `autocomplete` command usable to enable autocompletion
and revng2 now is based on `pype`, modifying its defalults and
injecting new commands.
Giacomo Vercesi