Commit Graph

190 Commits

Author SHA1 Message Date
Giacomo Vercesi c482951946 Add components to revng cli driver
This commit introduces the concept of `Component` to a pipeline step.
This, in turn, can be used by clients to figure out which artifacts are
produced by which revng component.
Additionally, this commit overhauls the `revng daemon-self-test`
command, renaming it to `revng graphql` and adding extra flexibility.
2023-08-23 16:47:30 +02:00
Giacomo Vercesi 1416acc8f5 revng.cli: centralize collect_pipelines
Move the logic that collects all the pipeline files to a cental
implementation.
2023-08-23 16:47:30 +02:00
Giacomo Vercesi 57eae2bc96 Drop hard-coded analyses list
Remove any use of hard-coded analyses lists, relying on user interaction
(e.g. command-line arguments) to have it instead.
2023-08-23 16:47:30 +02:00
Giacomo Vercesi 7e96bb2189 revng.api: avoid NULL dereferencing in extract
The call to `rp_container_extract_one` can return a null pointer, add a
check that allows to avoid dereferencing it if this happens.
2023-07-31 16:05:51 +02:00
Giacomo Vercesi 3a006422d3 typescript/python model wrappers: fix structure
Change the way the python and typescript model wrappers handle
attributes in struct fields. Now all fields are populated and none of
the use `None`.
2023-07-21 09:34:00 +02:00
Giacomo Vercesi 0ec2fdd175 revng.daemon: prevent shutdown crash
Change the behavior of `revng daemon` to avoid a crash at shutdown
caused by `rp_shutdown` not being called due to the pointer of
`rp_manager` not being freed.
2023-07-21 09:34:00 +02:00
Alessandro Di Federico 42abf64c62 revng.cli: ensure proper exit code propagation 2023-06-23 17:55:41 +02:00
Ivan Krysak 378f6e76f7 daemon: fix uploading binaries from html
The html view daemon sends when connected directly was not correctly
migrated to support analysis lists.
2023-05-25 06:25:07 +00:00
Giacomo Vercesi 7f73e2bfde revng.daemon: drop REVNG_PROJECT_ID
Remove the logic for computing a manager's workdir using the
REVNG_PROJECT_ID environment variable, as it was unused. The
REVNG_DATA_DIR remains to allow persistence across restarts.
2023-05-24 14:59:16 +02:00
Giacomo Vercesi 5d582a2eec fetch-debuginfo: improve cache folder handling
Rework how the cache folder for debug information is handled. This
commit introduces an additional environment variable, `REVNG_CACHE_DIR`
that allows overriding where semi-disposable files will be stored.
2023-05-24 11:09:04 +02:00
Giacomo Vercesi 2b181e627c fetch_debuginfo/common: improve download_file
Rework the code of `download_file` to leverage requests auto-decoding
option to auto-decompress the response body if needed.
2023-05-24 11:09:04 +02:00
Giacomo Vercesi 31cd2b53f9 rcc: add codespell
Add codespell to the list of checks performed by
revng-check-conventions. This should reduce the amount of typos present
in the revng codebase.
2023-05-11 10:05:32 +02:00
Giacomo Vercesi a13c3e5ac2 revng: fix executable/shebang in scripts
Some files had discordant executable bit presence and shebang presence.
This commit fixes these occurrences by adding removing the required
feautures where needed.
2023-05-11 10:02:46 +02:00
Giacomo Vercesi 9d5b6d1f34 revng cli: migrate legacy scripts to _commands
Move the remaining commands in the `scripts` to be used by the `revng`
command line driver.
2023-05-11 10:02:46 +02:00
Giacomo Vercesi 471b9c8396 rcc: fix issues with mypy>=1.0
Add additional config options to the `revng-check-conventions` config to
make mypy work with the revng codebase (mainly related to untyped
functions now causing a warning in mypy).
2023-05-11 10:02:46 +02:00
Giacomo Vercesi 472710c100 daemon-self-test: Improve usability
Allow debugging commands (e.g. `--gdb`) to be passed through to the
invoked daemon when running `revng daemon-self-test`.
2023-05-09 17:27:00 +02:00
Giacomo Vercesi 41662b0a43 revng.support: Rework process invocation
Refactor the `run` and `try_run` functions in 4 separate functions:
* `try_run`: Run the process and return the exit code
* `run`: Run the process and assert the exit code was 0
* `exec_run`: exec the process, stopping execution of the python
              interpreter
* `popen`: Wrapper around `subprocess.Popen`, allows the most
           flexibility
2023-05-09 17:27:00 +02:00
Giacomo Vercesi 6ab8099531 revng daemon: disable CORS without --production
Set the `REVNG_ALLOWED_ORIGINS` to `*` when not in production mode. This
should avoid cors issues while developing.
2023-05-09 17:27:00 +02:00
Giacomo Vercesi a2b7d0e0bf PipelineC: add tracing
Add support for tracing onto the PipelineC. This is done by:
1. Creating wrapper functions for each PipelineC function with the
   script in `scripts/PipelineC_add_tracing.py`. These will call a
   special function called `wrap` which will ultimately call a method
   with a `_` prepended to the name
2. Conversion of all PipelineC methods in `PipelineC.cpp` to `static`
   and their rename with a `_` in front, in order for them to work with
   the wrapper function in (1)
3. Generation of 2 additional include files, one for types and one for
   functions, to be used by users of tracing files in order to have
   introspection.

These steps allow the creation of a trace file with the use of the
`REVNG_C_API_TRACE_PATH` environment variable. The traces can then be
used in conjunction with the `revng trace run` and `revng trace
inspect` commands.
2023-04-20 14:43:13 +02:00
Giacomo Vercesi 8d89c6a318 PipelineC: rp_initialize: only use argc and argv
Switch from specifying `libraries` and `pipelines` in `rp_initialize`
and `rp_manager_create` to the use of command-line options that are to
be passed via `argc` and `argv` in `rp_initialize`.
2023-04-20 14:43:12 +02:00
Giacomo Vercesi 3bfdb95d92 revng.api: Improve Manager.run_analyses_list
Fix the case where the return of `rp_manager_run_analyses_list` returns
nullptr.
2023-04-20 14:43:12 +02:00
Giacomo Vercesi 07be2c1cae Allow the use of valgrind with python
This commit does 2 main things:
* Adds PYTHONMALLOC=1 environment variable when `--valgrind` is used
  to force python to use normal malloc instead of the custom allocator
  python normally uses that's not friendly with valgrind
* Add a suppression file to remove a couple of false-positives that
  happen consistently and generate useless noise
2023-04-20 14:43:12 +02:00
Giacomo Vercesi 4400924312 revng.cli: allow non-ELFs to be run with gdb
If needed, wrap the execution of scripts in `exec` to allow running
them with gdb. This is especially useful for running python scripts
that use cffi and being able to inspect the C/C++ side.
2023-04-20 14:43:11 +02:00
Giacomo Vercesi 59175e9925 daemon-self-test: do not fail() twice
There was a codepath in `revng-check-conventions` that would cause
`fail(e)` to be called twice. This would cause an exception on the
second invocation due to the undelying process being already terminated.
2023-04-20 14:43:11 +02:00
Giacomo Vercesi 87543f835a revng.cli._commands: drop shebangs
Remove leftover shebangs from python files in `revng.cli._commands` as
those files are no longer executable on their own
2023-04-20 14:43:11 +02:00
Alessandro Di Federico 416c59c43b support.py: rework get_command
We introduce two paths where we can hard code search prefixes
(`/additional-search-prefixes`) and additional binaries paths
(`/share/revng/additional-bin-paths`).

This enables us to add to `additional-bin-paths` the configure-time
specified path of the preferred LLVM installation, instead of relying on
`PATH` to look up, e.g., `opt` at run-time.
2023-04-13 18:11:48 +02:00
Alessandro Di Federico 1fa34fff9e revng opt: force legacy pass manager 2023-04-08 08:42:24 +02:00
Alessandro Di Federico 79bbd8b260 commands_registry.py: drop dict comprehension 2023-04-08 08:42:23 +02:00
Kacper Kołodziej d281d1fa05 Add --binary argument to llvm_pipeline.py 2023-04-04 09:30:08 +02:00
Giacomo Vercesi 2e96e09f63 Drop analyze-all
Drop all the uses of analyzeAll and swap its use with the invocation of
the appropriate analysesList
2023-03-22 17:50:02 +01:00
Giacomo Vercesi af237074f3 Implement AnalysesList in Python API & GraphQL
Implement the new functionality of AnalysesList on the python's side of
rev.ng and the GraphQL schema both in static and autogenerated form
2023-03-22 17:31:46 +01:00
Giacomo Vercesi 23286dc58f GraphQL: expose analysis options
Add an additional parameter that allows specifying options for analyses
as a serialized json string
2023-03-22 17:31:46 +01:00
Giacomo Vercesi 1f097bb46a revng.api: Fix leftover snake_case in as_dict
This commit fixes the name of the fields returned by the `as_dict`
function that were left in snake_case when the field in the schema was
camelCase
2023-03-22 17:31:46 +01:00
Ivan Krysak 3687869795 Model: make AddPrimitiveTypes a pure analysis 2023-03-15 10:19:01 +01:00
Giacomo Vercesi 15da250710 api/daemon: fix single analysis handling
* Let GraphQL accept zero or more parameters for analyses
* Handle the general case of analyses using zero or more containers with
  a variable ammount of targets
2023-03-15 10:19:01 +01:00
Giacomo Vercesi 7751143aad revng daemon: add timeout-keep-alive option
Add the `--timeout-keep-alive` option for all invocations of `revng
daemon`. This avoids some edge cases where connections are abruply
closed.
2023-03-14 20:50:28 +01:00
Giacomo Vercesi 2b4cfafe55 revng.api: Fix file upload/buffer reading
This commit fixes file output `revng.api` in two ways:
* The buffer's size is properly reported in `PipelineC.cpp`
* The function `deserialize_container` in `revng.api.manager` no longer
  appends a `\0` to `content`, while avoiding a redundant copy
2023-02-16 15:11:00 +01:00
Giacomo Vercesi 6f3dc255cf revng.api: adopt rp_error
Change Python's API to handle the new `rp_error` and its two variations:
`DocumentError` and `SimpleError`.
2023-02-09 13:43:33 +01:00
Giacomo Vercesi 275e3943f7 daemon: rework starlette startup code
Rework the startup code used to boot starlette, this allows removing the
use of all globals, moreover it allow dropping the ManagerMiddleware
class as it is no longer needed.
2023-02-07 16:54:35 +01:00
Giacomo Vercesi 8a780a58aa GraphQL: return binaries in b64
This change fixes the pipeline behavior when producing targets:
* produce and extract now return rp_buffer, which has a start pointer
  and size
* the python api `revng.api` will return on produce/extract either str
  or  bytes depending on the mime type of the container
* the GraphQL api will encode bytes in base64
2023-02-07 16:54:35 +01:00
Giacomo Vercesi 82c3962a3c Add GZip Middleware
Add the GZip Middleware to rev.ng daemon, allowing it to handle the
`Content-Encoding: gzip` header
2023-02-07 16:54:35 +01:00
Giacomo Vercesi 23b8ed8dbe revng.daemon: allow extra middlewares
Introduce the environment variable `STARLETTE_MIDDLEWARES` that
allows adding additional middlewares at startup to the startlette
server.
2023-02-07 16:54:35 +01:00
Giacomo Vercesi 88e972c57c revng fifo events: move handling to daemon
Move all the event handling logic to the `revng.daemon` logic, since
now fifo notifications also include the JWT token (if present) of the
request.
2023-02-07 16:54:35 +01:00
Giacomo Vercesi fc93198e25 revng daemon: Fix resolve_step_artifacts
Fixes the GraphQL handler for step's artifacts as it was creating the
wrong object for the 'container' property
2023-02-07 16:54:35 +01:00
Giacomo Vercesi 52ff853d00 revng daemon: Fix CORS allowed headers
Add an option to CORSMiddleware to allow any header to be provided when
checking for CORS
2023-02-07 16:54:35 +01:00
Giacomo Vercesi 6f475d1368 revng.api: Fix signature of Target.kind 2023-02-07 16:54:35 +01:00
Giacomo Vercesi aa344d4122 revng daemon: add -b none
Add the `-b none` option to `revng daemon` that allows disabling any
automatic bind
2023-02-07 16:54:35 +01:00
Giacomo Vercesi 23cd7f871a revng daemon: fix project-id checking
Fixes the regex that's used to check if the project ID is valid
2023-02-07 16:54:35 +01:00
Antonio Frighetto ea0a06a876 revng.daemon: accept no arguments for analysis
Let GraphQL accept zero or more parameters for analyses.
2023-02-06 09:37:23 +01:00
Giacomo Vercesi b4c1cd94e1 revng-daemon: preserve more signals
Add more signals to be handled from python rather than LLVM.
2023-02-01 17:46:19 +01:00