Switch out from using uvicorn to hypercorn for the pypeline daemon, make
the facilities create hypercorn commands in stand-alone functions, as
they will be re-used for other servers later on.
Expand the `ProjectMetadata` dataclass to include additional timestamps
and the hash of the pipeline description. Change all the
`StorageProvider`s so that all the new data is saved and returned.
The `PypeGroup` click group did not provide any real benefit:
* It added extra help about pipebox arguments, but text that was not
correct from where the help was coming from (the group does not parse
the arguments). The subcommands still reports it though.
* It forbid using non-PypeGroup and non-PypeCommand commands in the
command tree, but in the long run this is desirable so it should be
dropped
For these reasons it has been dropped.
The `--format` option relied on `click.Context.get_parameter_source` to
determine if the default value for the `--format` parameter was passed.
A bug in click caused this to behave "correctly", click 8.4.0 fixed the
bug. Change the logic by using `None` as the default value so that it
can be checked without using the parameter source.
Overhaul the pipeline configuration logic by collapsing all dynamic
configuration options, both for analyses and pipes into a single
dictionary. Change all the interfaces so that there is no longer
distinction between the configuration of an analysis and of pipes.
Expose these options to the command line via `--{name}-configuration`
options for each pipe/analysis that is applicable to the command-line
invocation.
Change the `Container.serialize` interface so that it is possible to
supply a list of objects that will be serialized instead of all the ones
in the container.
Add categories in the pipeline, allow an artifact to have a category
specified which allows it to be shown or hidden by default. Adapt the
CLI tools to hide artifacts of the category that don't
`show_by_default=True`.
Change the `get_model` and `set_model` interface of `StorageProvider` so
that it is responsibility of the `StorageProvider` to
serialize/deserialize the model before returning to the caller.
This is in preparation to the model migration being implemented, since
it's now a responsibility of the storage provider to deserialize it it
can trivially re-save it to disk if it is migrated.
Overhaul the logic and method names involved in enabling and disabling
reference caching in `TupleTree<T>`. `TupleTreeReference<T, U>` now
lazily caches the target and will traverse the path only when needed.
Also expose and use these functions in the new pipeline, which should
provide some speedup when executing a `Schedule`.
The functions that take care of running an analysis/analysis list have a
lot of duplication, merge the two into a single function that takes care
of running both.
Fix the code that handles setting the `--format`, `--tar` and `--yaml`
command line options, which incorrectly wrote to the `format` kwargs
instead of the `container_format` one.
Instead of executing `chdir` when using the `-C` option, which is
fragile when other paths are involved, store the option in the click
context and propagate it to the required code paths that require knowing
what the base directory is.
Add the functionality the the pipeline infrastructure and CLI to run
individual pipe and analyses as subcommands instead of in-process. This
allow better debuggability of individual pipes.
When setting the variable names for the analysis' configuration,
sanitize the analysis' name, as it might contain dashes and other
undesirable characters.
Now commands 'pype project artifact' and 'pype pipeline run_pipe' now
accepts --format to specify in which format to serialize the resulting
container, and have --tar and --yaml shortcuts. Moreover, these commands
and 'pipe pipeline run_analysis' can automatically figure out the format
of containers so the user won't have to specify them most of the time.
Move the epoch inside the storage_provider, this allows us to make the
storage_provider_factory to return an async context manager.
This way we can simplify the locking for the daemon, but forces the
CLI project commands to deal with async code.
Now the pype command has a `--verbose` argument that to enable debug
logging in the pypelien code.
Adapted the codebase to use this new logging format but replacing
`logging` with `revng.pypeline.utils.logger`, this is done in
preparation of debug-log.
Add implementation of the generic pype daemon.
It's implementation is split into `daemon.py` where we implement an
http framework agnostic interface, and `app.py` that uses it to serve
them using starlette.
Now pype has an `autocomplete` command usable to enable autocompletion
and revng2 now is based on `pype`, modifying its defalults and
injecting new commands.
When click calls the command handler functions, dashes are converted to
underscores to work with the variable name limitations of python. When
accessing kwargs, convert the argument name to the underscored version.
Giacomo Vercesi