Implement compression of objects before they are saved into the storage
provider. Each container type can specify which algorithm to use
(currently `none` or `zstd`) and the compression level.
Change byte containers from being just a `using` of the relative class
(e.g. `BytesContainer`) to a full fledged subclass. This has the
following benefits:
* The template arguments are now laid out in the container class body,
making it more readable.
* The creation of a new class creates a new symbol, improving
debuggability as now the container classes have their own name rather
than the expanded base class template.
Split the `invalidate` method in two phases: in `invalidateCheck` it
just checks if the model diff warrants the execution of the actual
`invalidate` method, which requires fetching the custom invalidation
data from storage. The actual `invalidation` method remains the same.
This should make invalidation faster for storage providers that store
the invalidation remotely.
Overhaul the logic and method names involved in enabling and disabling
reference caching in `TupleTree<T>`. `TupleTreeReference<T, U>` now
lazily caches the target and will traverse the path only when needed.
Also expose and use these functions in the new pipeline, which should
provide some speedup when executing a `Schedule`.
Add infrastructure to pypeline that allows containers to be notified
when they are being used last, this allows two things:
* `Pipe`s eagerly clearing those containers once they are done reading
their contents
* `ScheduledTask`s clearing those out at the end of their execution in
case the pipe did not do it
This overall should improve memory usage as container no longer take up
memory if they are no longer used as part of a `Schedule`.
Store the contents of the files deserialized by `BinariesContainer` to
disk via `TemporaryFile`. This also allows some pipes to avoid shuffling
data around since they needed a file on disk anyways.
In equivalence with `LLVMFunctionMixin`, add a mixin that allows
writing a pipe that instead of defining the `runOnFunction` method it
defines the `runOnCliftFunction` which receives both the
`module::Function` and the `FunctionOp`.
Add a mixin class for piperuns that work on an LLVM module and need
both the model function and the `llvm::Function`. Pipe that inherit this
mixin will receive both the model and LLVM function in the
`runOnLLVMFunction` method.
Change the `Register.*Pipe` functions to end in `Run` if the template
argument should be a pipe run. Also change the concepts `Is.*Pipe` to
also have the `Run` suffix if applicable.
Make the interface required by `SingleOutputPipe` similar to other
piperuns by requiring a constructor, the use of `PipeRunArgument`s and
the implementation of a non-static `run` function.
Move out of the `runPipe`/`runAnalysis` functions the logic that
extracts the container from the vector type. This logic is now
delegated to `containerVectorToTuple` which converts the vector type to
a tuple of references. This allows python's `run{Pipe,Analysis}` to
safely call `nanobind::gil_scoped_release` which releases the GIL for
the entire duration of the `runPipe`/`runAnalysis` execution.
Add another container for LLVM modules. This one stores one
`llvm::Module` per function (all modules share the same
`llvm::LLVMContext` for performance reasons).
Add an enum to explicitly specify the `TaskArgumentAccess` of a given
`Pipe`. This is needed because in some cases it's required to manually
specify the `READ` or `WRITE` access.
The `name` parameter of the `nanobind::class_` constructor needs to be
NULL-terminated. Copy the `StringRef` to a temporary string to add the
missing `\0`.
* Make the following private headers public:
* Lift/CPUStateAccessAnalysisPass.h
* Lift/CSVOffsets.h
* Lift/PTCDump.h
* Lift/VariableManager.h
* Move from revngSupport to revngLift:
* IRAnnotators.{h,cpp}
* SelfReferencingDbgAnnotationWriter.{h,cpp}
* Move from revngSupport to revngModel:
* FunctionTags.{h,cpp}
* ProgramCounterHandler.{h,cpp}
* Move from revngSupport to revngRecompile:
* OriginalAssemblyAnnotationWriter.{h,cpp}
Introduce the `ImportFiles` pipe which leverages the newly-introduced
`FileStorage` to fetch the required files and store them in the
`BinariesContainer`.
The `serialize` function of `revng::pypeline::Model` erroneously
returned a buffer instead of `bytes`, fix the function so that `bytes`
are returned instead.
Alessandro Di Federico