Commit Graph

32 Commits

Author SHA1 Message Date
Khaled Ismaeel a52f1f17c2 Add automatic migrations in PipelineManager 2025-04-30 15:08:45 +02:00
Giacomo Vercesi f740badd68 PipelineManager: drop storeContext
The `storeContext` function was a misguided attempt to performance
optimization. Without storing all the containers and the invalidation
metadata the resume directory would be corrupt by just storing the
context.
2025-04-03 15:59:18 +02:00
Giacomo Vercesi 05041a302a PipelineManager: save after running analyses
Save the context after an analysis (or analyses-list) has run. This
guarantees that at least the model is preserved in case of crash.
2024-12-20 12:24:28 +01:00
Alessandro Di Federico c8d4381297 s/Context &Ctx/Context &Context/g 2024-09-27 12:07:17 +02:00
Massimo Fioravanti 7ee8c8000d Introduce revng pipe 2024-06-20 09:57:36 +02:00
Massimo Fioravanti c276a439b5 Add pipeline invalidation
Replace the stub implementation of invalidation with the proper
implementation. A ReadPathCache is added to each global so that it can
keep tracks of what target are associated to which read paths.
2024-01-02 11:14:56 +01:00
Giacomo Vercesi 04698bb550 Return invalidations when deserializing container
When deserializing a container in the pipeline, return the invalidations
that have been caused by the container changing content.
2023-11-03 12:13:58 +01:00
Giacomo Vercesi e2504dd6f2 Implement storage credentials update
Add the functionality to rev.ng's infrastructure to allow to dynamically
update the credentials of the underlying storage provider at runtime.
2023-09-14 17:32:08 +02:00
Giacomo Vercesi f710af9f2f Cleanup revng.daemon graphql API
Drop autogenerated queries from the GraphQL API and switch `revng.api`
to use `PipelineDescriptor`.
2023-09-14 15:44:33 +02:00
Giacomo Vercesi 2a239ddf47 Introduce PipelineDescription
Add the PipelineDescription object that describes the structure of a
pipeline. This will be used by all the downstream users instead of
pipeline accessor methods via PipelineC.
2023-09-14 15:44:33 +02:00
Giacomo Vercesi 0b2754c824 pipeline: drop Disk from storage methods
This commit changes the following method names across the codebase:
* `storeToDisk` -> `store`
* `loadFromDisk` -> `load`
This has been done since the storage is no longer bound to the local
storage.
2023-09-06 15:23:43 +02:00
Giacomo Vercesi 93d3a6238d revng: introduce S3 support
Add the capability for all revng tooling to run with an S3-backed
workdir.
2023-09-06 15:23:43 +02:00
Giacomo Vercesi db7ad24175 PipelineManager: add produceTargets with checks
Move much of the logic of `rp_manager_produce_targets` to a method
inside `PipelineManager`, add additional checks for the existence of the
targets that are requested to be produced.
2023-07-31 16:05:51 +02:00
Giacomo Vercesi 9a972009d1 PipelineC: size and const-correctness
Fix the return type (often mismatched from uint64_t to int) and the
const-correctness of many functions in PipelineC
2023-04-20 14:43:12 +02:00
Giacomo Vercesi 2e96e09f63 Drop analyze-all
Drop all the uses of analyzeAll and swap its use with the invocation of
the appropriate analysesList
2023-03-22 17:50:02 +01:00
Massimo Fioravanti 312e2e8d53 Add analyses lists
Introduces the possibility of specifiying lists of analyses as way to
give them coherent names.
2023-03-22 17:31:46 +01:00
Giacomo Vercesi 81d25f9805 PipelineC: expose invalidation logic 2022-10-30 09:13:14 +01:00
Alessandro Di Federico cf62e6dce7 Minor changes 2022-09-22 17:43:55 +02:00
Massimo Fioravanti 6b099c57e8 Pipeline: introduce options for analyses 2022-08-29 14:43:25 +02:00
Massimo Fioravanti d250d43ee4 revng-pipeline: factor out common CLI options
Common CLI options that were used by multiple tools and have now been
isolated into a single include file to be used before declaring main.
2022-08-10 09:49:24 +02:00
Massimo Fioravanti b49062b2d6 revng-pipeline: introduce --produce-all-single 2022-08-10 09:49:24 +02:00
Massimo Fioravanti 4cd5abe2b0 revng-pipeline: introduce --produce and --analyze 2022-08-10 09:49:24 +02:00
Massimo Fioravanti 9c13f8dbdd Introduce revng-artifact 2022-08-10 09:49:24 +02:00
Massimo Fioravanti 96bd5835d8 revng-pipeline: rework file-to-container syntax 2022-08-10 09:49:24 +02:00
Massimo Fioravanti 85a9c9637f revng-pipeline: rework logging 2022-08-10 09:49:24 +02:00
Giacomo Vercesi 2df691535d Pipeline: Make recalculateAllPossibleTargets
`recalculateAllPossibleTarget` is now a private member of
PipelineManager This was done since all calls that can trigger a change
in the target list have been isolated and the call is done implicitly.
This removes rp_manager_recompute_all_available_targets from PipelineC,
since it was added as a stopgap until the above was implemented.
2022-06-29 14:50:58 +02:00
Giacomo Vercesi 487af0f4b0 Do not pass file paths to revng from PipelineC
Allow deserializing containers from a byte string rather than pointing
to a file, this allows revng to have an opaque workdir.
2022-06-29 14:50:58 +02:00
Giacomo Vercesi 05dc27715d Improve pipeline saving capabilities
This commit introduces some changes to how the revng pipeline handles
serializing to disk. Specificaly:

* Pipeline globals (specifically model.yml) are better handled if they
  are in a subdirectory. They are now saved in the "context"
  subdirectory.
* In python:revng.api the pipeline is serialized whenever there is a
  non-reproducible change to the state (e.g. binary upload or model
  change).
  In the case of analyses this is done conservatively by checking that
  the diff produced is not empty.
* The logic for computing a step's subdirectory has been moved to the
  pipeline runner, consequently if a step is asked to serialize it
  will not create any subdirectories.
* Functionality for saving a single step/context has been exposed in
  Pipeline C.
* Finally, all path concatenations are now handled by
  llvm::sys::path::append, for extra os-agnosticism.
2022-06-29 14:50:58 +02:00
Massimo Fioravanti f0a5ffe3b3 revng-pipeline: globals, extractOne and analyses 2022-05-24 08:58:48 +02:00
Massimo Fioravanti 3d752f6c7f Introduce revng pipeline --invalidate-all 2022-04-15 15:18:28 +02:00
Massimo Fioravanti 602f1e0701 Introduce non-const PipelineManager::context 2022-03-28 12:17:05 +02:00
Massimo Fioravanti a8a8da3bae Introduce revng pipes
This commit imports all the revng pipes (and other support utilities) to
be used with `revng-pipeline`. In particular, the pipes necessary for
binary translations have been introduced.
2022-02-08 00:05:03 +01:00