Commit Graph

115 Commits

Author SHA1 Message Date
Giacomo Vercesi f710af9f2f Cleanup revng.daemon graphql API
Drop autogenerated queries from the GraphQL API and switch `revng.api`
to use `PipelineDescriptor`.
2023-09-14 15:44:33 +02:00
Giacomo Vercesi e43e720a35 revng daemon: allow setting exposed CORS headers
Add the enviroment variable `REVNG_EXPOSE_HEADERS` for `revng daemon` to
allow customizing the headers exposed via CORS.
2023-09-06 16:12:43 +02:00
Giacomo Vercesi ca82e47f79 FunctionStringMap: switch to using GzipTarFile
Change the serialization and deserialzation functions of
`FunctionStringMap` to use the GzipTarFile. Also change che filename
of the underlying containers accordingly.
2023-09-06 15:23:43 +02:00
Giacomo Vercesi e719b3b7e5 Add components to revng cli driver
This commit introduces the concept of `Component` to a pipeline step.
This, in turn, can be used by clients to figure out which artifacts are
produced by which revng component.
Additionally, this commit overhauls the `revng daemon-self-test`
command, renaming it to `revng graphql` and adding extra flexibility.
2023-08-23 17:45:36 +02:00
Giacomo Vercesi 9fc9ca0d1e revng.cli: centralize collect_pipelines
Move the logic that collects all the pipeline files to a cental
implementation.
2023-08-23 17:45:36 +02:00
Giacomo Vercesi 13ba286a9a Drop hard-coded analyses list
Remove any use of hard-coded analyses lists, relying on user interaction
(e.g. command-line arguments) to have it instead.
2023-08-23 17:45:36 +02:00
Alessandro Di Federico 247879f7fc model::Type::ID: switch to progressive IDs
This commit switches `model::Type::ID` from being a GUID to be a
progressive number, in order to make things easier for humans.

On top of this, this commit introduces the following changes:

* TypeCopier: import all the necessary PrimitiveTypes and improve
  handling of CustomName.
* Move Kind as the last field of the key of each TupleTree type used in
  an `UpcastablePointer`.
* Update the ground truth of tests to ignore the `CustomName` in favor
  of focusing on `OriginalName`.
* Increase adoption of `model::Binary::makeType`, equivalent to
  `Binary.recordNewType(makeType<model::*Type>())`.
2023-08-23 16:14:04 +02:00
Alessandro Di Federico fe18317bb7 model compare: introduce $ for exact size match
This commit introduces a change in `revng model compare` so that, for
dictionary entries whose keys start with `$`, it is ensured that the
length of the associated value (be it a list or a dictionary) is the
same.
2023-08-23 16:14:04 +02:00
Alessandro Di Federico 42abf64c62 revng.cli: ensure proper exit code propagation 2023-06-23 17:55:41 +02:00
Giacomo Vercesi 7f73e2bfde revng.daemon: drop REVNG_PROJECT_ID
Remove the logic for computing a manager's workdir using the
REVNG_PROJECT_ID environment variable, as it was unused. The
REVNG_DATA_DIR remains to allow persistence across restarts.
2023-05-24 14:59:16 +02:00
Giacomo Vercesi 5d582a2eec fetch-debuginfo: improve cache folder handling
Rework how the cache folder for debug information is handled. This
commit introduces an additional environment variable, `REVNG_CACHE_DIR`
that allows overriding where semi-disposable files will be stored.
2023-05-24 11:09:04 +02:00
Giacomo Vercesi 2b181e627c fetch_debuginfo/common: improve download_file
Rework the code of `download_file` to leverage requests auto-decoding
option to auto-decompress the response body if needed.
2023-05-24 11:09:04 +02:00
Giacomo Vercesi 31cd2b53f9 rcc: add codespell
Add codespell to the list of checks performed by
revng-check-conventions. This should reduce the amount of typos present
in the revng codebase.
2023-05-11 10:05:32 +02:00
Giacomo Vercesi a13c3e5ac2 revng: fix executable/shebang in scripts
Some files had discordant executable bit presence and shebang presence.
This commit fixes these occurrences by adding removing the required
feautures where needed.
2023-05-11 10:02:46 +02:00
Giacomo Vercesi 9d5b6d1f34 revng cli: migrate legacy scripts to _commands
Move the remaining commands in the `scripts` to be used by the `revng`
command line driver.
2023-05-11 10:02:46 +02:00
Giacomo Vercesi 471b9c8396 rcc: fix issues with mypy>=1.0
Add additional config options to the `revng-check-conventions` config to
make mypy work with the revng codebase (mainly related to untyped
functions now causing a warning in mypy).
2023-05-11 10:02:46 +02:00
Giacomo Vercesi 472710c100 daemon-self-test: Improve usability
Allow debugging commands (e.g. `--gdb`) to be passed through to the
invoked daemon when running `revng daemon-self-test`.
2023-05-09 17:27:00 +02:00
Giacomo Vercesi 41662b0a43 revng.support: Rework process invocation
Refactor the `run` and `try_run` functions in 4 separate functions:
* `try_run`: Run the process and return the exit code
* `run`: Run the process and assert the exit code was 0
* `exec_run`: exec the process, stopping execution of the python
              interpreter
* `popen`: Wrapper around `subprocess.Popen`, allows the most
           flexibility
2023-05-09 17:27:00 +02:00
Giacomo Vercesi 6ab8099531 revng daemon: disable CORS without --production
Set the `REVNG_ALLOWED_ORIGINS` to `*` when not in production mode. This
should avoid cors issues while developing.
2023-05-09 17:27:00 +02:00
Giacomo Vercesi a2b7d0e0bf PipelineC: add tracing
Add support for tracing onto the PipelineC. This is done by:
1. Creating wrapper functions for each PipelineC function with the
   script in `scripts/PipelineC_add_tracing.py`. These will call a
   special function called `wrap` which will ultimately call a method
   with a `_` prepended to the name
2. Conversion of all PipelineC methods in `PipelineC.cpp` to `static`
   and their rename with a `_` in front, in order for them to work with
   the wrapper function in (1)
3. Generation of 2 additional include files, one for types and one for
   functions, to be used by users of tracing files in order to have
   introspection.

These steps allow the creation of a trace file with the use of the
`REVNG_C_API_TRACE_PATH` environment variable. The traces can then be
used in conjunction with the `revng trace run` and `revng trace
inspect` commands.
2023-04-20 14:43:13 +02:00
Giacomo Vercesi 07be2c1cae Allow the use of valgrind with python
This commit does 2 main things:
* Adds PYTHONMALLOC=1 environment variable when `--valgrind` is used
  to force python to use normal malloc instead of the custom allocator
  python normally uses that's not friendly with valgrind
* Add a suppression file to remove a couple of false-positives that
  happen consistently and generate useless noise
2023-04-20 14:43:12 +02:00
Giacomo Vercesi 4400924312 revng.cli: allow non-ELFs to be run with gdb
If needed, wrap the execution of scripts in `exec` to allow running
them with gdb. This is especially useful for running python scripts
that use cffi and being able to inspect the C/C++ side.
2023-04-20 14:43:11 +02:00
Giacomo Vercesi 59175e9925 daemon-self-test: do not fail() twice
There was a codepath in `revng-check-conventions` that would cause
`fail(e)` to be called twice. This would cause an exception on the
second invocation due to the undelying process being already terminated.
2023-04-20 14:43:11 +02:00
Giacomo Vercesi 87543f835a revng.cli._commands: drop shebangs
Remove leftover shebangs from python files in `revng.cli._commands` as
those files are no longer executable on their own
2023-04-20 14:43:11 +02:00
Alessandro Di Federico 416c59c43b support.py: rework get_command
We introduce two paths where we can hard code search prefixes
(`/additional-search-prefixes`) and additional binaries paths
(`/share/revng/additional-bin-paths`).

This enables us to add to `additional-bin-paths` the configure-time
specified path of the preferred LLVM installation, instead of relying on
`PATH` to look up, e.g., `opt` at run-time.
2023-04-13 18:11:48 +02:00
Alessandro Di Federico 1fa34fff9e revng opt: force legacy pass manager 2023-04-08 08:42:24 +02:00
Alessandro Di Federico 79bbd8b260 commands_registry.py: drop dict comprehension 2023-04-08 08:42:23 +02:00
Kacper Kołodziej d281d1fa05 Add --binary argument to llvm_pipeline.py 2023-04-04 09:30:08 +02:00
Giacomo Vercesi 2e96e09f63 Drop analyze-all
Drop all the uses of analyzeAll and swap its use with the invocation of
the appropriate analysesList
2023-03-22 17:50:02 +01:00
Ivan Krysak 3687869795 Model: make AddPrimitiveTypes a pure analysis 2023-03-15 10:19:01 +01:00
Giacomo Vercesi 7751143aad revng daemon: add timeout-keep-alive option
Add the `--timeout-keep-alive` option for all invocations of `revng
daemon`. This avoids some edge cases where connections are abruply
closed.
2023-03-14 20:50:28 +01:00
Giacomo Vercesi 23b8ed8dbe revng.daemon: allow extra middlewares
Introduce the environment variable `STARLETTE_MIDDLEWARES` that
allows adding additional middlewares at startup to the startlette
server.
2023-02-07 16:54:35 +01:00
Giacomo Vercesi aa344d4122 revng daemon: add -b none
Add the `-b none` option to `revng daemon` that allows disabling any
automatic bind
2023-02-07 16:54:35 +01:00
Djordje Todorovic 22ba1800c5 idb: fix generation of empty structs and unions
There was a log message that we are creating typedef void* for empty
structs, but we have missed that piece of code. This patch adds that
logic.

In the case of unions, we completely ignored empty ones, and also
there was not handling if the type was generated by ordinal (as
in the case of structs). This patch also fixes it.
2023-01-27 17:49:11 +01:00
Alessandro Di Federico 7913bf9e58 Disable LeakSanitizer 2023-01-21 16:46:35 +01:00
Alessandro Di Federico 47fd084c88 revng --verbose: shorten paths only if beneficial 2023-01-21 16:46:35 +01:00
Djordje Todorovic 0d6ffa7504 Introduce revng fetch-debuginfo 2022-12-13 11:25:36 +01:00
Antonio Frighetto b2d24875e6 override-by-name.py: compare to bytes 2022-11-30 15:06:54 +01:00
Alessandro Di Federico 838d9a5992 revng model override-by-name: fix args doc 2022-11-30 15:06:54 +01:00
Massimo Fioravanti 044071435e Fix handling of binaries with no functions
Fix various edge cases when a binary with no functions was analyzed.
2022-11-22 16:19:28 +01:00
Giacomo Vercesi 48cd3aad42 revng.cli: init commands_registry once
Only init commands_registry once, this is to avoid commands being
re-registered and allow quicker execution of nested commands.
2022-11-02 11:50:40 +01:00
Giacomo Vercesi 4af389c703 daemon-self-test: fix daemon output
Fix the `fail` function, to properly capture the output of the daemon
process in case of failure.
2022-11-02 11:50:40 +01:00
Giacomo Vercesi a6a7a09606 daemon-self-test: fix dumping of daemon output
Fix the `fail` function, to properly capture the output of the daemon
process in case of failure.
2022-10-30 09:12:33 +01:00
Giacomo Vercesi e4adbe8134 revng.support: add get_root
Add the get_root function which will return the absolute path of the
root, relatively to the file location in support
2022-10-24 09:58:55 +02:00
Alessandro Di Federico 5a99154196 Put Python packages into lib/python*/site-packages 2022-10-11 19:08:10 +02:00
Giacomo Vercesi ff48b9de75 Introduce revng daemon-self-test
This new command will, given an executable, try to produce all
artifacts. By default it will run `revng daemon`, however an external
address can be provided to test remote daemons, e.g. running inside a
container.
2022-10-06 15:12:36 +02:00
Giacomo Vercesi a935cc3073 revng daemon: switch from hypercorn to uvicorn
Hypercorn is experiencing weird behaviour when running tests and
receiving SIGPIPEs when a couple of F5s are issued from the browser.
The switch to uvicorn fixed these issues.
2022-10-04 14:13:55 +02:00
Giacomo Vercesi df68b2c7c0 revng ptml: merge cat and strip
Merge the cat and strip subcommands within `revng ptml`
Refactor codebase so that both color and plaintext output are handled
in a unified manner.
2022-09-29 15:43:17 +02:00
Giacomo Vercesi bf4122ed0a Introduce v2 of revng-check-conventions 2022-09-29 15:21:59 +02:00
Alessandro Di Federico f6d5cd24dd Give pretty names to temporary files 2022-09-26 18:09:01 +02:00