Commit Graph

46 Commits

Author SHA1 Message Date
Lauri Vasama 90e5dfd4c9 Pass MLIRContext by address
Passing by address is more conventional. This is what MLIR does most of
the time, and it avoids dereferences and addressofs everywhere.
2026-04-30 15:09:16 +03:00
Lauri Vasama a338194eb7 Move namespace mlir::clift out of namespace mlir 2026-04-27 12:48:24 +03:00
Giacomo Vercesi 63a11d0c63 pypeline: add Analysis.is_available
Add a method to the `Analysis` interface which allows an analysis to
report if it is available for execution.
2026-03-31 17:00:48 +02:00
Ivan Krysak 58facb0de2 pipebox: introduce two more clift containers
`CliftSingleTypeContainer` for a type-level entity and
`CliftModuleContainer` for a binary-level one (like a header).
2026-03-25 17:26:24 +00:00
Ivan Krysak b76a09d483 pipebox: rename CliftContainer.h
The new name is `CliftContainers.h`.
2026-03-25 17:26:24 +00:00
Giacomo Vercesi 6ec9f9a952 TupleTree: overhaul reference caching
Overhaul the logic and method names involved in enabling and disabling
reference caching in `TupleTree<T>`. `TupleTreeReference<T, U>` now
lazily caches the target and will traverse the path only when needed.
Also expose and use these functions in the new pipeline, which should
provide some speedup when executing a `Schedule`.
2026-03-04 14:58:02 +01:00
Giacomo Vercesi 704c5e9bbb TupleTree: fix broken operator==
The `operator==` for keyed `StructDefinitions` was the comparison
between keys, which is misleading.
2026-03-04 14:58:02 +01:00
Giacomo Vercesi f1878f1194 pypeline: drop TraceRunner
Drop the `pypeline-trace-run` executable, as its purposed has been
superseded by the combination of `--debug` and native pipe/analysis
runners.
2026-02-16 10:34:39 +01:00
Giacomo Vercesi 25ba3115c4 pypeline: implement native run-{pipe,analysis}
Implement C++-only command-line executables that allow running a single
pipe/analysis without the involvement of python.
2026-02-16 10:34:38 +01:00
Giacomo Vercesi 7f7f9571a8 pypeline: implement container discarding
Add infrastructure to pypeline that allows containers to be notified
when they are being used last, this allows two things:
* `Pipe`s eagerly clearing those containers once they are done reading
  their contents
* `ScheduledTask`s clearing those out at the end of their execution in
  case the pipe did not do it
This overall should improve memory usage as container no longer take up
memory if they are no longer used as part of a `Schedule`.
2025-12-19 10:29:28 +01:00
Giacomo Vercesi e8f293271b BinariesContainer: page files to disk
Store the contents of the files deserialized by `BinariesContainer` to
disk via `TemporaryFile`. This also allows some pipes to avoid shuffling
data around since they needed a file on disk anyways.
2025-12-19 10:28:29 +01:00
Giacomo Vercesi 3cd011c146 Introduce CliftFunctionMixin
In equivalence with `LLVMFunctionMixin`, add a mixin that allows
writing a pipe that instead of defining the `runOnFunction` method it
defines the `runOnCliftFunction` which receives both the
`module::Function` and the `FunctionOp`.
2025-12-10 16:16:14 +01:00
Giacomo Vercesi 16e89416c5 Add CliftFunctionContainer
Add a container that owns one `mlir::ModuleOp` per function, to be used
in conjunction with the new clift backend.
2025-12-10 16:16:14 +01:00
Giacomo Vercesi 9e8e4cdf85 Add AnalyzeDataLayout analysis to pypeline 2025-12-10 16:16:14 +01:00
Giacomo Vercesi 19f67c8223 Introduce LLVMFunctionMixin
Add a mixin class for piperuns that work on an LLVM module and need
both the model function and the `llvm::Function`. Pipe that inherit this
mixin will receive both the model and LLVM function in the
`runOnLLVMFunction` method.
2025-12-10 15:05:53 +01:00
Giacomo Vercesi db94859401 Registrars: add Run suffix for piperuns
Change the `Register.*Pipe` functions to end in `Run` if the template
argument should be a pipe run. Also change the concepts `Is.*Pipe` to
also have the `Run` suffix if applicable.
2025-12-10 15:05:53 +01:00
Giacomo Vercesi 296eb9a76f Make SingleOutputPipe uniform to other piperuns
Make the interface required by `SingleOutputPipe` similar to other
piperuns by requiring a constructor, the use of `PipeRunArgument`s and
the implementation of a non-static `run` function.
2025-12-10 15:05:53 +01:00
Giacomo Vercesi 532a4a19f6 Drop LLVMFunctionPassPipe
Drop the `LLVMFunctionPassPipe` as it will not be used in the pipeline
by any pipe and its infrastructure is not needed.
2025-12-10 15:05:53 +01:00
Giacomo Vercesi 1da6e49b36 ObjectDependenciesTracker: also stop tracking
Call `revng::Tracking::stop` in the destructor of
`ObjectDependenciesTracker` to avoid tracking happening after the object
is destroyed.
2025-12-10 15:05:53 +01:00
Giacomo Vercesi b1b5778822 Add infrastructure for custom invalidation
Add infrastructure to pypeline to allow for pipes to provide custom
invalidation, by implementing an `invalidate` function.
2025-11-26 16:13:28 +01:00
Giacomo Vercesi 8cff10e13b Introduce analyses that manipulate the model
Add 4 analyses to the pipebox that allow setting and verifying the model
and its diffs.
2025-11-26 16:13:28 +01:00
Giacomo Vercesi 4a1edec018 get_registry: use name for selected classes
Use the `name` class attribute to build the registry when discovering
subclasses of `Pipe`, `Analysis` and `Container`.
2025-11-26 14:59:43 +01:00
Giacomo Vercesi de75d94597 {Pipe,Analysis}Run: release Python GIL
Move out of the `runPipe`/`runAnalysis` functions the logic that
extracts the container from the vector type. This logic is now
delegated to `containerVectorToTuple` which converts the vector type to
a tuple of references. This allows python's `run{Pipe,Analysis}` to
safely call `nanobind::gil_scoped_release` which releases the GIL for
the entire duration of the `runPipe`/`runAnalysis` execution.
2025-11-26 14:59:43 +01:00
Giacomo Vercesi ca3c5113d9 Add recompile-isolated to pypeline 2025-11-18 17:47:54 +01:00
Giacomo Vercesi af4b447f89 Add CompileRootModule to pypeline 2025-11-18 17:47:54 +01:00
Giacomo Vercesi 97b471c888 Add Lift to pypeline 2025-11-18 17:47:54 +01:00
Giacomo Vercesi 1b9fb62a33 SingleOutputPipe: relax assertion
Relax the assertion at the beginning of `SingleOutputPipe` that
triggered an assertion when 0 objects where requested for output.
2025-11-18 17:47:54 +01:00
Giacomo Vercesi 5297a354ad PipeRunPipes: automatically track progress
Automatically create a `Task` object when a pipe with multiple steps
(e.g. `FunctionPipe`) is run.
2025-11-18 17:47:54 +01:00
Giacomo Vercesi b4f9147646 Introduce LLVMFunctionsContainer
Add another container for LLVM modules. This one stores one
`llvm::Module` per function (all modules share the same
`llvm::LLVMContext` for performance reasons).
2025-11-18 17:47:54 +01:00
Giacomo Vercesi 57d9c3deb8 PipeboxCommon: add Access enum
Add an enum to explicitly specify the `TaskArgumentAccess` of a given
`Pipe`. This is needed because in some cases it's required to manually
specify the `READ` or `WRITE` access.
2025-11-17 10:04:13 +01:00
Giacomo Vercesi 1c07656448 pipebox.py: add initialize function
Add the `initialize` function to `pipebox.py`. This allows passing
command-line arguments to the pipebox.
2025-11-17 10:04:13 +01:00
Giacomo Vercesi 4f93c9e7ad Container: add mime type
Add the `MimeType` field to the C++ classes. Forward it to python via
nanobind.
2025-11-17 10:04:13 +01:00
Giacomo Vercesi a8aa034acf Registrars.h: fix out-of-bounds read
The `name` parameter of the `nanobind::class_` constructor needs to be
NULL-terminated. Copy the `StringRef` to a temporary string to add the
missing `\0`.
2025-11-17 10:04:13 +01:00
Giacomo Vercesi 000d75dbb7 pypeline: add check_precondition to Pipe
Add the `check_precondition` function to the `Pipe` class. This will be
run before the actual schedule creates containers and runs itself
proper.
2025-11-17 10:04:13 +01:00
Alessandro Di Federico f8bd4c3bac Move around some files in preparation for libtcg
* Make the following private headers public:
  * Lift/CPUStateAccessAnalysisPass.h
  * Lift/CSVOffsets.h
  * Lift/PTCDump.h
  * Lift/VariableManager.h
* Move from revngSupport to revngLift:
  * IRAnnotators.{h,cpp}
  * SelfReferencingDbgAnnotationWriter.{h,cpp}
* Move from revngSupport to revngModel:
  * FunctionTags.{h,cpp}
  * ProgramCounterHandler.{h,cpp}
* Move from revngSupport to revngRecompile:
  * OriginalAssemblyAnnotationWriter.{h,cpp}
2025-10-24 15:34:11 +02:00
Giacomo Vercesi 6abba05d7e Introduce ImportFiles pipe
Introduce the `ImportFiles` pipe which leverages the newly-introduced
`FileStorage` to fetch the required files and store them in the
`BinariesContainer`.
2025-10-16 17:48:45 +02:00
Giacomo Vercesi b622fe8f19 Implement FileProvider interface
Implement the `FileProvider` interface, this allows pipes to request
files given a `FileRequest` (hash and optional size/name).
2025-10-16 17:48:45 +02:00
Giacomo Vercesi ca0f312b1e Pipebox: remove const from ArrayRef
Remove the `const` from `ArrayRef<const char>` as the `ArrayRef` already
implies the underlying type will be accessed only in `const`.
2025-10-16 16:07:09 +02:00
Giacomo Vercesi 18f0bd88f9 pypeline::Model: fix serialize
The `serialize` function of `revng::pypeline::Model` erroneously
returned a buffer instead of `bytes`, fix the function so that `bytes`
are returned instead.
2025-10-16 12:51:52 +02:00
Giacomo Vercesi f165f99392 pypeline.Model: make deserialize to static
Change the `Model.deserialize` to a static function, which will return
a new instance of the Model.
2025-10-16 10:30:01 +02:00
Giacomo Vercesi d92f4cdca3 ObjectID: introduce byte (de)serialization
Add the `to_bytes` and `from_bytes` to `ObjectID`. This allows
serializing an `ObjectID` instance to raw bytes with a specific binary
format.
2025-10-15 11:58:41 +02:00
Giacomo Vercesi 10a396a3b5 PipeboxCommon: rework {Analysis,Pipe}Runner
Rework the `AnalysisRunner` and `PipeRunner` structs into single
functions, these do not need to have a template parameter specifying how
to extract containers as that logic has been delegated to a newly
introduced trait called `ExtractContainerFromList`.
2025-10-15 11:58:41 +02:00
Giacomo Vercesi bb0709effc Introduce compile_time::callWithIndexSequence
Add a helper function that given a `size_t` or a tuple-like type, allows
calling a lambda with the expanded sequence pack. This avoids having to
manually create the index sequence and defining the lambda accepting it.
2025-10-15 11:58:41 +02:00
Giacomo Vercesi 6d1fbdca97 Implement infrastructure for PipeRuns
Add the needed infrastructures that allow ephemeral classes (`PipeRun`s)
to be wrapped as a regular pypeline `Pipe` class. These automatically
have their dependencies tracked and propagated to the pypeline.
2025-10-03 17:40:45 +02:00
Giacomo Vercesi 448f70e7dc Add trace runner
Add the tool `pypeline-trace-runner` which allows reading a serialized
pypeline schedule and execute it purely in C++.
2025-09-17 14:09:16 +02:00
Giacomo Vercesi 688b9fe111 Introduce compatibility with revng-pypeline
Add the necessary machinery to allow `Pipe`s, `Analysis`es,
`Container`s, `Model` and `ObjectID` to be implemented in C++ and used
by Python.
2025-09-17 14:02:40 +02:00