This commit introduces the concept of `Component` to a pipeline step.
This, in turn, can be used by clients to figure out which artifacts are
produced by which revng component.
Additionally, this commit overhauls the `revng daemon-self-test`
command, renaming it to `revng graphql` and adding extra flexibility.
Change the behavior of `revng daemon` to avoid a crash at shutdown
caused by `rp_shutdown` not being called due to the pointer of
`rp_manager` not being freed.
Add support for tracing onto the PipelineC. This is done by:
1. Creating wrapper functions for each PipelineC function with the
script in `scripts/PipelineC_add_tracing.py`. These will call a
special function called `wrap` which will ultimately call a method
with a `_` prepended to the name
2. Conversion of all PipelineC methods in `PipelineC.cpp` to `static`
and their rename with a `_` in front, in order for them to work with
the wrapper function in (1)
3. Generation of 2 additional include files, one for types and one for
functions, to be used by users of tracing files in order to have
introspection.
These steps allow the creation of a trace file with the use of the
`REVNG_C_API_TRACE_PATH` environment variable. The traces can then be
used in conjunction with the `revng trace run` and `revng trace
inspect` commands.
Switch from specifying `libraries` and `pipelines` in `rp_initialize`
and `rp_manager_create` to the use of command-line options that are to
be passed via `argc` and `argv` in `rp_initialize`.
* Let GraphQL accept zero or more parameters for analyses
* Handle the general case of analyses using zero or more containers with
a variable ammount of targets
This commit fixes file output `revng.api` in two ways:
* The buffer's size is properly reported in `PipelineC.cpp`
* The function `deserialize_container` in `revng.api.manager` no longer
appends a `\0` to `content`, while avoiding a redundant copy
This change fixes the pipeline behavior when producing targets:
* produce and extract now return rp_buffer, which has a start pointer
and size
* the python api `revng.api` will return on produce/extract either str
or bytes depending on the mime type of the container
* the GraphQL api will encode bytes in base64
Handle crashes via signal handlers. Switch python's revng.api from
a normal python function to faulthandler, which works also in the
case of harsher interruptions (e.g. SIGABRT).
rp_shutdown must be called after all the owning pointers given by
PipelineC are freed, in order to guarantee this easily, the python
ApiWrapper will use an atomic counter, which when shutdown has been
signaled and counter reaches zero will automatically call rp_shutdown
Rework both classes in schema_generator.py, main changes:
* Drop manager.pipeline_artifact_structure as it was used exclusively
by SchemaGenerator and is no longer needed
* Rename SchemaGen and BindableGen to SchemaGenerator and
DynamicBindableGenerator to better express their role, add
docstrings that explain what they do
* Decouple Artifacts and Analyses discovery, since a step can have
analyses and no artifacts (and vice-versa)
A step's artifacts now include singleTargetFilename, which gives a
suggested filename to use when a single element is extracted from the
underlying container.
When requesting a produce for multiple targets the result will be a
dictionary mapping "<target>:<result>". This also changes the GraphQL
API where a json-serialized string is returned.
`recalculateAllPossibleTarget` is now a private member of
PipelineManager This was done since all calls that can trigger a change
in the target list have been isolated and the call is done implicitly.
This removes rp_manager_recompute_all_available_targets from PipelineC,
since it was added as a stopgap until the above was implemented.
Add the possibility of passing a list of FIFOs to `revng daemon`. These
can be used to notify an external program when a non-reproducible change
(binary, context) has occurred.
This commit introduces some changes to how the revng pipeline handles
serializing to disk. Specificaly:
* Pipeline globals (specifically model.yml) are better handled if they
are in a subdirectory. They are now saved in the "context"
subdirectory.
* In python:revng.api the pipeline is serialized whenever there is a
non-reproducible change to the state (e.g. binary upload or model
change).
In the case of analyses this is done conservatively by checking that
the diff produced is not empty.
* The logic for computing a step's subdirectory has been moved to the
pipeline runner, consequently if a step is asked to serialize it
will not create any subdirectories.
* Functionality for saving a single step/context has been exposed in
Pipeline C.
* Finally, all path concatenations are now handled by
llvm::sys::path::append, for extra os-agnosticism.
`revng.api` now autodetects files (libraries, pipeline yamls) for
initialization via the same mechanism used by `revng.cli`. This removes
the need to pass them via environment variables from `revng daemon`
and allows easy interaction with python's C API.
Giacomo Vercesi