Commit Graph

225 Commits

Author SHA1 Message Date
Alessandro Di Federico 757f7e21d7 Import docs 2024-02-09 10:04:24 +01:00
Alessandro Di Federico d9367954a4 Convert all public strings to kebab-case 2024-02-09 10:04:24 +01:00
Alessandro Di Federico 2bdca8ab03 Rename steps and analyses 2024-02-09 10:04:22 +01:00
Alessandro Di Federico 6f60d9b492 Introduce revng --rr 2024-02-09 10:04:19 +01:00
Giacomo Vercesi d4532ab512 revng daemon: print debug log to stderr
When printing the current workdir, use stderr instead of stdout.
2024-01-26 19:13:58 +01:00
Giacomo Vercesi 73cd5a5655 revng daemon: allow manuanlly saving via SIGUSR2
This commit adds the functionality, in debug mode, for `revng daemon` to
save when the SIGUSR2 signal is sent.
2024-01-26 19:13:58 +01:00
Giacomo Vercesi ca66b4ddfc InitRevng: fix command line parsing
This commit fixes command line parsing of revng through the
REVNG_OPTIONS environment variable. Previously the env was prepended to
the actual command-line variables, now it's appended, allowing to
specify command line arguments that depend on `-load`s.
2024-01-26 17:46:21 +01:00
Alessandro Di Federico 028908a592 revng graphql: print analyses' errors 2024-01-02 15:08:26 +01:00
Massimo Fioravanti e5347b2910 Remove llvm pipeline 2024-01-02 11:14:55 +01:00
Giacomo Vercesi 942cf50735 python: create python wheels
Package revng's python code in two wheels: `revng` and `revng_internal`.
The revng wheel contains the
`revng.{pipeline_description,model,tupletree}` modules, while the
`revng_internal` one everything under `revng.internal`.
2023-12-12 14:52:22 +01:00
Giacomo Vercesi 7deb85b863 revng daemon: implement plugin interface
Deprecate the `STARLETTE_MIDDLEWARES_*` environment variables in favor
of a plugin infrastucture, which allows plugins to register middlewares
and save hooks via a `setup` function.
2023-12-12 14:52:22 +01:00
Alessandro Di Federico ca7041741f fetch_debuginfo: fix PDB URL computation
The pefile library changed its API.
On top updating the code accordingly, we now also properly pad hex
numbers.
2023-12-12 12:04:33 +01:00
Alessandro Di Federico 35f31ad469 fetch_debuginfo: do not catch Exception 2023-12-12 12:04:32 +01:00
Alessandro Di Federico 8302717da7 fetch_debuginfo: use HTTPS for msdl.microsoft.com 2023-12-12 12:04:32 +01:00
Alessandro Di Federico 10dcb7be11 Fix semantics of model::Types purging passes
One of the two passes to purge unused types from the model had a
ill-defined semantics.

This commit simplifies them in two passes: one that purges all the types
unreachable from outside `Binary::Types` and one that does the same but
also preserves types with a `CustomName` or an `OriginalName`.
2023-12-05 16:19:39 +01:00
Giacomo Vercesi d737a13117 manager.py: temporarily remove auto-saves
Remove the auto-save code that gets triggered when running analyses.
This improves performance since it doesn't force the manager to save to
storage. This is due to the manager eagerly serializing the pipeline
(both in terms of amount of data and which container should be
serialized), this is especially expensive when the resume directory is
backed by S3.
2023-11-03 16:30:40 +01:00
Giacomo Vercesi 046612f11c revng.graphql: enforce index consistency
Force clients to provide the current index to run `produce` or
`runAnalysis`. Requests that do not provide the correct index return an
IndexError object.
2023-11-03 14:51:31 +01:00
Giacomo Vercesi c83932003b revng.daemon: also send index in invalidations
When sending invalidations via websocket, also send the new context
global index.
2023-11-03 12:13:58 +01:00
Giacomo Vercesi 94aa2b51d2 Add Context Global Index 2023-11-03 12:13:58 +01:00
Giacomo Vercesi 04698bb550 Return invalidations when deserializing container
When deserializing a container in the pipeline, return the invalidations
that have been caused by the container changing content.
2023-11-03 12:13:58 +01:00
Giacomo Vercesi 56af9c1f9d Propagate errors from Analysis/Pipes to daemon
Leverage the `rp_error` mechanism to propagate errors from analyses and
pipes to the GraphQL schema.
2023-11-03 12:13:58 +01:00
Giacomo Vercesi 3b0d708a2f Drop custom globals mutator from PipelineC
Remove globals mutator from PipelineC api, as there are analyses present
that allow changing them.
2023-11-03 12:13:58 +01:00
Giacomo Vercesi 8dd0b64030 revng.daemon: add missing emit_event
The `runAnalysesList` resolve did not emit a `CONTEXT` event when run.
2023-11-03 12:13:57 +01:00
Giacomo Vercesi f467671173 revng.api: drop the use of islice
The `itertools.islice` function does not support negative index slices,
rework the code of `collection.abc.Sequence` classes to use `range`
instead.
2023-11-03 12:13:57 +01:00
Giacomo Vercesi 58cef84e3a cli.graphql: improve log messages
Explicitly state the analyses list being run rather than the generic
"Autoanalysis complete".
2023-11-03 12:13:57 +01:00
Giacomo Vercesi e2504dd6f2 Implement storage credentials update
Add the functionality to rev.ng's infrastructure to allow to dynamically
update the credentials of the underlying storage provider at runtime.
2023-09-14 17:32:08 +02:00
Giacomo Vercesi f710af9f2f Cleanup revng.daemon graphql API
Drop autogenerated queries from the GraphQL API and switch `revng.api`
to use `PipelineDescriptor`.
2023-09-14 15:44:33 +02:00
Giacomo Vercesi 2a239ddf47 Introduce PipelineDescription
Add the PipelineDescription object that describes the structure of a
pipeline. This will be used by all the downstream users instead of
pipeline accessor methods via PipelineC.
2023-09-14 15:44:33 +02:00
Giacomo Vercesi 90ea42b695 model/_util.py: move into tupletree.py
Move the contents of `model/_util.py` into `tupletree.py` as it's
model-agnostic and used by all generated Python tuple-tree wrappers.
2023-09-14 15:44:32 +02:00
Giacomo Vercesi e43e720a35 revng daemon: allow setting exposed CORS headers
Add the enviroment variable `REVNG_EXPOSE_HEADERS` for `revng daemon` to
allow customizing the headers exposed via CORS.
2023-09-06 16:12:43 +02:00
Giacomo Vercesi ca82e47f79 FunctionStringMap: switch to using GzipTarFile
Change the serialization and deserialzation functions of
`FunctionStringMap` to use the GzipTarFile. Also change che filename
of the underlying containers accordingly.
2023-09-06 15:23:43 +02:00
Giacomo Vercesi 93d3a6238d revng: introduce S3 support
Add the capability for all revng tooling to run with an S3-backed
workdir.
2023-09-06 15:23:43 +02:00
Giacomo Vercesi a84ebd0e51 revng.daemon: trim Eventmanager's functionality
Drop most of the functionality provided by EventManager. Add the
functionality that allows saving the working directory after a period of
inactivity.
2023-09-06 15:23:43 +02:00
Giacomo Vercesi e719b3b7e5 Add components to revng cli driver
This commit introduces the concept of `Component` to a pipeline step.
This, in turn, can be used by clients to figure out which artifacts are
produced by which revng component.
Additionally, this commit overhauls the `revng daemon-self-test`
command, renaming it to `revng graphql` and adding extra flexibility.
2023-08-23 17:45:36 +02:00
Giacomo Vercesi 9fc9ca0d1e revng.cli: centralize collect_pipelines
Move the logic that collects all the pipeline files to a cental
implementation.
2023-08-23 17:45:36 +02:00
Giacomo Vercesi 13ba286a9a Drop hard-coded analyses list
Remove any use of hard-coded analyses lists, relying on user interaction
(e.g. command-line arguments) to have it instead.
2023-08-23 17:45:36 +02:00
Alessandro Di Federico 247879f7fc model::Type::ID: switch to progressive IDs
This commit switches `model::Type::ID` from being a GUID to be a
progressive number, in order to make things easier for humans.

On top of this, this commit introduces the following changes:

* TypeCopier: import all the necessary PrimitiveTypes and improve
  handling of CustomName.
* Move Kind as the last field of the key of each TupleTree type used in
  an `UpcastablePointer`.
* Update the ground truth of tests to ignore the `CustomName` in favor
  of focusing on `OriginalName`.
* Increase adoption of `model::Binary::makeType`, equivalent to
  `Binary.recordNewType(makeType<model::*Type>())`.
2023-08-23 16:14:04 +02:00
Alessandro Di Federico fe18317bb7 model compare: introduce $ for exact size match
This commit introduces a change in `revng model compare` so that, for
dictionary entries whose keys start with `$`, it is ensured that the
length of the associated value (be it a list or a dictionary) is the
same.
2023-08-23 16:14:04 +02:00
Giacomo Vercesi 7e96bb2189 revng.api: avoid NULL dereferencing in extract
The call to `rp_container_extract_one` can return a null pointer, add a
check that allows to avoid dereferencing it if this happens.
2023-07-31 16:05:51 +02:00
Giacomo Vercesi 3a006422d3 typescript/python model wrappers: fix structure
Change the way the python and typescript model wrappers handle
attributes in struct fields. Now all fields are populated and none of
the use `None`.
2023-07-21 09:34:00 +02:00
Giacomo Vercesi 0ec2fdd175 revng.daemon: prevent shutdown crash
Change the behavior of `revng daemon` to avoid a crash at shutdown
caused by `rp_shutdown` not being called due to the pointer of
`rp_manager` not being freed.
2023-07-21 09:34:00 +02:00
Alessandro Di Federico 42abf64c62 revng.cli: ensure proper exit code propagation 2023-06-23 17:55:41 +02:00
Ivan Krysak 378f6e76f7 daemon: fix uploading binaries from html
The html view daemon sends when connected directly was not correctly
migrated to support analysis lists.
2023-05-25 06:25:07 +00:00
Giacomo Vercesi 7f73e2bfde revng.daemon: drop REVNG_PROJECT_ID
Remove the logic for computing a manager's workdir using the
REVNG_PROJECT_ID environment variable, as it was unused. The
REVNG_DATA_DIR remains to allow persistence across restarts.
2023-05-24 14:59:16 +02:00
Giacomo Vercesi 5d582a2eec fetch-debuginfo: improve cache folder handling
Rework how the cache folder for debug information is handled. This
commit introduces an additional environment variable, `REVNG_CACHE_DIR`
that allows overriding where semi-disposable files will be stored.
2023-05-24 11:09:04 +02:00
Giacomo Vercesi 2b181e627c fetch_debuginfo/common: improve download_file
Rework the code of `download_file` to leverage requests auto-decoding
option to auto-decompress the response body if needed.
2023-05-24 11:09:04 +02:00
Giacomo Vercesi 31cd2b53f9 rcc: add codespell
Add codespell to the list of checks performed by
revng-check-conventions. This should reduce the amount of typos present
in the revng codebase.
2023-05-11 10:05:32 +02:00
Giacomo Vercesi a13c3e5ac2 revng: fix executable/shebang in scripts
Some files had discordant executable bit presence and shebang presence.
This commit fixes these occurrences by adding removing the required
feautures where needed.
2023-05-11 10:02:46 +02:00
Giacomo Vercesi 9d5b6d1f34 revng cli: migrate legacy scripts to _commands
Move the remaining commands in the `scripts` to be used by the `revng`
command line driver.
2023-05-11 10:02:46 +02:00
Giacomo Vercesi 471b9c8396 rcc: fix issues with mypy>=1.0
Add additional config options to the `revng-check-conventions` config to
make mypy work with the revng codebase (mainly related to untyped
functions now causing a warning in mypy).
2023-05-11 10:02:46 +02:00