Overhaul the pipeline configuration logic by collapsing all dynamic
configuration options, both for analyses and pipes into a single
dictionary. Change all the interfaces so that there is no longer
distinction between the configuration of an analysis and of pipes.
Expose these options to the command line via `--{name}-configuration`
options for each pipe/analysis that is applicable to the command-line
invocation.
Rework the artifact endpoint, making it similar to the CLI invocation.
It now works on a single artifact and can be made to return both `json`
and `tar` as formats.
Add categories in the pipeline, allow an artifact to have a category
specified which allows it to be shown or hidden by default. Adapt the
CLI tools to hide artifacts of the category that don't
`show_by_default=True`.
Rework which information is transmitted in the pipeline metadata,
avoiding redundancy and moving some information there instead of
returning it every time a request is made.
Change the `get_model` and `set_model` interface of `StorageProvider` so
that it is responsibility of the `StorageProvider` to
serialize/deserialize the model before returning to the caller.
This is in preparation to the model migration being implemented, since
it's now a responsibility of the storage provider to deserialize it it
can trivially re-save it to disk if it is migrated.
Instead of executing `chdir` when using the `-C` option, which is
fragile when other paths are involved, store the option in the click
context and propagate it to the required code paths that require knowing
what the base directory is.
Add the functionality the the pipeline infrastructure and CLI to run
individual pipe and analyses as subcommands instead of in-process. This
allow better debuggability of individual pipes.
Improve the handling of signals by leveraging the `Py_AtExit`
functionality to trigger cleanup when the interpreter exits.
When receiving SIGINT handle it specially because exceptions in python
are only thrown when the interpreter is running and not when C code is.
Add infrastructure to pypeline that allows containers to be notified
when they are being used last, this allows two things:
* `Pipe`s eagerly clearing those containers once they are done reading
their contents
* `ScheduledTask`s clearing those out at the end of their execution in
case the pipe did not do it
This overall should improve memory usage as container no longer take up
memory if they are no longer used as part of a `Schedule`.
Inline the body of `PipelineNode.run` into `ScheduledTask.run`, making
`PipelineNode` a pure data structure. This also allows eliding all the
arguments since all of them as properties of `ScheduledTask`.
In equivalence with `LLVMFunctionMixin`, add a mixin that allows
writing a pipe that instead of defining the `runOnFunction` method it
defines the `runOnCliftFunction` which receives both the
`module::Function` and the `FunctionOp`.
Add an enum to explicitly specify the `TaskArgumentAccess` of a given
`Pipe`. This is needed because in some cases it's required to manually
specify the `READ` or `WRITE` access.
Move the epoch inside the storage_provider, this allows us to make the
storage_provider_factory to return an async context manager.
This way we can simplify the locking for the daemon, but forces the
CLI project commands to deal with async code.
Now pype has an `autocomplete` command usable to enable autocompletion
and revng2 now is based on `pype`, modifying its defalults and
injecting new commands.
Now the model has a mime_type that we expect to return to the UI, and
model_name so the name of the model on disk can be impl-specific.
Moreover, now Model and Container use the same logic to detect if
a mimetype is textual.
Change the signature of `Container.serialize` and
`Container.deserialize` to use `Buffer` instead of `bytes`, as this
makes the functions compatible with the `revng::pypeline::Buffer`
classes.
Add a test that checks that the newly implemented invalidation logic
actually works and the objects stored as children or parent objectids
are also dropped when invalidating.
Giacomo Vercesi