Commit Graph

209 Commits

Author SHA1 Message Date
Giacomo Vercesi 046612f11c revng.graphql: enforce index consistency
Force clients to provide the current index to run `produce` or
`runAnalysis`. Requests that do not provide the correct index return an
IndexError object.
2023-11-03 14:51:31 +01:00
Giacomo Vercesi c83932003b revng.daemon: also send index in invalidations
When sending invalidations via websocket, also send the new context
global index.
2023-11-03 12:13:58 +01:00
Giacomo Vercesi 94aa2b51d2 Add Context Global Index 2023-11-03 12:13:58 +01:00
Giacomo Vercesi 04698bb550 Return invalidations when deserializing container
When deserializing a container in the pipeline, return the invalidations
that have been caused by the container changing content.
2023-11-03 12:13:58 +01:00
Giacomo Vercesi 56af9c1f9d Propagate errors from Analysis/Pipes to daemon
Leverage the `rp_error` mechanism to propagate errors from analyses and
pipes to the GraphQL schema.
2023-11-03 12:13:58 +01:00
Giacomo Vercesi 3b0d708a2f Drop custom globals mutator from PipelineC
Remove globals mutator from PipelineC api, as there are analyses present
that allow changing them.
2023-11-03 12:13:58 +01:00
Giacomo Vercesi 8dd0b64030 revng.daemon: add missing emit_event
The `runAnalysesList` resolve did not emit a `CONTEXT` event when run.
2023-11-03 12:13:57 +01:00
Giacomo Vercesi f467671173 revng.api: drop the use of islice
The `itertools.islice` function does not support negative index slices,
rework the code of `collection.abc.Sequence` classes to use `range`
instead.
2023-11-03 12:13:57 +01:00
Giacomo Vercesi 58cef84e3a cli.graphql: improve log messages
Explicitly state the analyses list being run rather than the generic
"Autoanalysis complete".
2023-11-03 12:13:57 +01:00
Giacomo Vercesi e2504dd6f2 Implement storage credentials update
Add the functionality to rev.ng's infrastructure to allow to dynamically
update the credentials of the underlying storage provider at runtime.
2023-09-14 17:32:08 +02:00
Giacomo Vercesi f710af9f2f Cleanup revng.daemon graphql API
Drop autogenerated queries from the GraphQL API and switch `revng.api`
to use `PipelineDescriptor`.
2023-09-14 15:44:33 +02:00
Giacomo Vercesi 2a239ddf47 Introduce PipelineDescription
Add the PipelineDescription object that describes the structure of a
pipeline. This will be used by all the downstream users instead of
pipeline accessor methods via PipelineC.
2023-09-14 15:44:33 +02:00
Giacomo Vercesi 90ea42b695 model/_util.py: move into tupletree.py
Move the contents of `model/_util.py` into `tupletree.py` as it's
model-agnostic and used by all generated Python tuple-tree wrappers.
2023-09-14 15:44:32 +02:00
Giacomo Vercesi e43e720a35 revng daemon: allow setting exposed CORS headers
Add the enviroment variable `REVNG_EXPOSE_HEADERS` for `revng daemon` to
allow customizing the headers exposed via CORS.
2023-09-06 16:12:43 +02:00
Giacomo Vercesi ca82e47f79 FunctionStringMap: switch to using GzipTarFile
Change the serialization and deserialzation functions of
`FunctionStringMap` to use the GzipTarFile. Also change che filename
of the underlying containers accordingly.
2023-09-06 15:23:43 +02:00
Giacomo Vercesi 93d3a6238d revng: introduce S3 support
Add the capability for all revng tooling to run with an S3-backed
workdir.
2023-09-06 15:23:43 +02:00
Giacomo Vercesi a84ebd0e51 revng.daemon: trim Eventmanager's functionality
Drop most of the functionality provided by EventManager. Add the
functionality that allows saving the working directory after a period of
inactivity.
2023-09-06 15:23:43 +02:00
Giacomo Vercesi e719b3b7e5 Add components to revng cli driver
This commit introduces the concept of `Component` to a pipeline step.
This, in turn, can be used by clients to figure out which artifacts are
produced by which revng component.
Additionally, this commit overhauls the `revng daemon-self-test`
command, renaming it to `revng graphql` and adding extra flexibility.
2023-08-23 17:45:36 +02:00
Giacomo Vercesi 9fc9ca0d1e revng.cli: centralize collect_pipelines
Move the logic that collects all the pipeline files to a cental
implementation.
2023-08-23 17:45:36 +02:00
Giacomo Vercesi 13ba286a9a Drop hard-coded analyses list
Remove any use of hard-coded analyses lists, relying on user interaction
(e.g. command-line arguments) to have it instead.
2023-08-23 17:45:36 +02:00
Alessandro Di Federico 247879f7fc model::Type::ID: switch to progressive IDs
This commit switches `model::Type::ID` from being a GUID to be a
progressive number, in order to make things easier for humans.

On top of this, this commit introduces the following changes:

* TypeCopier: import all the necessary PrimitiveTypes and improve
  handling of CustomName.
* Move Kind as the last field of the key of each TupleTree type used in
  an `UpcastablePointer`.
* Update the ground truth of tests to ignore the `CustomName` in favor
  of focusing on `OriginalName`.
* Increase adoption of `model::Binary::makeType`, equivalent to
  `Binary.recordNewType(makeType<model::*Type>())`.
2023-08-23 16:14:04 +02:00
Alessandro Di Federico fe18317bb7 model compare: introduce $ for exact size match
This commit introduces a change in `revng model compare` so that, for
dictionary entries whose keys start with `$`, it is ensured that the
length of the associated value (be it a list or a dictionary) is the
same.
2023-08-23 16:14:04 +02:00
Giacomo Vercesi 7e96bb2189 revng.api: avoid NULL dereferencing in extract
The call to `rp_container_extract_one` can return a null pointer, add a
check that allows to avoid dereferencing it if this happens.
2023-07-31 16:05:51 +02:00
Giacomo Vercesi 3a006422d3 typescript/python model wrappers: fix structure
Change the way the python and typescript model wrappers handle
attributes in struct fields. Now all fields are populated and none of
the use `None`.
2023-07-21 09:34:00 +02:00
Giacomo Vercesi 0ec2fdd175 revng.daemon: prevent shutdown crash
Change the behavior of `revng daemon` to avoid a crash at shutdown
caused by `rp_shutdown` not being called due to the pointer of
`rp_manager` not being freed.
2023-07-21 09:34:00 +02:00
Alessandro Di Federico 42abf64c62 revng.cli: ensure proper exit code propagation 2023-06-23 17:55:41 +02:00
Ivan Krysak 378f6e76f7 daemon: fix uploading binaries from html
The html view daemon sends when connected directly was not correctly
migrated to support analysis lists.
2023-05-25 06:25:07 +00:00
Giacomo Vercesi 7f73e2bfde revng.daemon: drop REVNG_PROJECT_ID
Remove the logic for computing a manager's workdir using the
REVNG_PROJECT_ID environment variable, as it was unused. The
REVNG_DATA_DIR remains to allow persistence across restarts.
2023-05-24 14:59:16 +02:00
Giacomo Vercesi 5d582a2eec fetch-debuginfo: improve cache folder handling
Rework how the cache folder for debug information is handled. This
commit introduces an additional environment variable, `REVNG_CACHE_DIR`
that allows overriding where semi-disposable files will be stored.
2023-05-24 11:09:04 +02:00
Giacomo Vercesi 2b181e627c fetch_debuginfo/common: improve download_file
Rework the code of `download_file` to leverage requests auto-decoding
option to auto-decompress the response body if needed.
2023-05-24 11:09:04 +02:00
Giacomo Vercesi 31cd2b53f9 rcc: add codespell
Add codespell to the list of checks performed by
revng-check-conventions. This should reduce the amount of typos present
in the revng codebase.
2023-05-11 10:05:32 +02:00
Giacomo Vercesi a13c3e5ac2 revng: fix executable/shebang in scripts
Some files had discordant executable bit presence and shebang presence.
This commit fixes these occurrences by adding removing the required
feautures where needed.
2023-05-11 10:02:46 +02:00
Giacomo Vercesi 9d5b6d1f34 revng cli: migrate legacy scripts to _commands
Move the remaining commands in the `scripts` to be used by the `revng`
command line driver.
2023-05-11 10:02:46 +02:00
Giacomo Vercesi 471b9c8396 rcc: fix issues with mypy>=1.0
Add additional config options to the `revng-check-conventions` config to
make mypy work with the revng codebase (mainly related to untyped
functions now causing a warning in mypy).
2023-05-11 10:02:46 +02:00
Giacomo Vercesi 472710c100 daemon-self-test: Improve usability
Allow debugging commands (e.g. `--gdb`) to be passed through to the
invoked daemon when running `revng daemon-self-test`.
2023-05-09 17:27:00 +02:00
Giacomo Vercesi 41662b0a43 revng.support: Rework process invocation
Refactor the `run` and `try_run` functions in 4 separate functions:
* `try_run`: Run the process and return the exit code
* `run`: Run the process and assert the exit code was 0
* `exec_run`: exec the process, stopping execution of the python
              interpreter
* `popen`: Wrapper around `subprocess.Popen`, allows the most
           flexibility
2023-05-09 17:27:00 +02:00
Giacomo Vercesi 6ab8099531 revng daemon: disable CORS without --production
Set the `REVNG_ALLOWED_ORIGINS` to `*` when not in production mode. This
should avoid cors issues while developing.
2023-05-09 17:27:00 +02:00
Giacomo Vercesi a2b7d0e0bf PipelineC: add tracing
Add support for tracing onto the PipelineC. This is done by:
1. Creating wrapper functions for each PipelineC function with the
   script in `scripts/PipelineC_add_tracing.py`. These will call a
   special function called `wrap` which will ultimately call a method
   with a `_` prepended to the name
2. Conversion of all PipelineC methods in `PipelineC.cpp` to `static`
   and their rename with a `_` in front, in order for them to work with
   the wrapper function in (1)
3. Generation of 2 additional include files, one for types and one for
   functions, to be used by users of tracing files in order to have
   introspection.

These steps allow the creation of a trace file with the use of the
`REVNG_C_API_TRACE_PATH` environment variable. The traces can then be
used in conjunction with the `revng trace run` and `revng trace
inspect` commands.
2023-04-20 14:43:13 +02:00
Giacomo Vercesi 8d89c6a318 PipelineC: rp_initialize: only use argc and argv
Switch from specifying `libraries` and `pipelines` in `rp_initialize`
and `rp_manager_create` to the use of command-line options that are to
be passed via `argc` and `argv` in `rp_initialize`.
2023-04-20 14:43:12 +02:00
Giacomo Vercesi 3bfdb95d92 revng.api: Improve Manager.run_analyses_list
Fix the case where the return of `rp_manager_run_analyses_list` returns
nullptr.
2023-04-20 14:43:12 +02:00
Giacomo Vercesi 07be2c1cae Allow the use of valgrind with python
This commit does 2 main things:
* Adds PYTHONMALLOC=1 environment variable when `--valgrind` is used
  to force python to use normal malloc instead of the custom allocator
  python normally uses that's not friendly with valgrind
* Add a suppression file to remove a couple of false-positives that
  happen consistently and generate useless noise
2023-04-20 14:43:12 +02:00
Giacomo Vercesi 4400924312 revng.cli: allow non-ELFs to be run with gdb
If needed, wrap the execution of scripts in `exec` to allow running
them with gdb. This is especially useful for running python scripts
that use cffi and being able to inspect the C/C++ side.
2023-04-20 14:43:11 +02:00
Giacomo Vercesi 59175e9925 daemon-self-test: do not fail() twice
There was a codepath in `revng-check-conventions` that would cause
`fail(e)` to be called twice. This would cause an exception on the
second invocation due to the undelying process being already terminated.
2023-04-20 14:43:11 +02:00
Giacomo Vercesi 87543f835a revng.cli._commands: drop shebangs
Remove leftover shebangs from python files in `revng.cli._commands` as
those files are no longer executable on their own
2023-04-20 14:43:11 +02:00
Alessandro Di Federico 416c59c43b support.py: rework get_command
We introduce two paths where we can hard code search prefixes
(`/additional-search-prefixes`) and additional binaries paths
(`/share/revng/additional-bin-paths`).

This enables us to add to `additional-bin-paths` the configure-time
specified path of the preferred LLVM installation, instead of relying on
`PATH` to look up, e.g., `opt` at run-time.
2023-04-13 18:11:48 +02:00
Alessandro Di Federico 1fa34fff9e revng opt: force legacy pass manager 2023-04-08 08:42:24 +02:00
Alessandro Di Federico 79bbd8b260 commands_registry.py: drop dict comprehension 2023-04-08 08:42:23 +02:00
Kacper Kołodziej d281d1fa05 Add --binary argument to llvm_pipeline.py 2023-04-04 09:30:08 +02:00
Giacomo Vercesi 2e96e09f63 Drop analyze-all
Drop all the uses of analyzeAll and swap its use with the invocation of
the appropriate analysesList
2023-03-22 17:50:02 +01:00
Giacomo Vercesi af237074f3 Implement AnalysesList in Python API & GraphQL
Implement the new functionality of AnalysesList on the python's side of
rev.ng and the GraphQL schema both in static and autogenerated form
2023-03-22 17:31:46 +01:00