In the new pipeline the root module is split off in its individual
isolated modules at the end of `isolate`. Before splitting, there are a
lot of global variables in the root module and only a small part is
going to be needed after splitting for each module. To avoid excessive
memory usage employ `ConservativeModuleCloner` in `Isolate` so that
only the needed global variables are actually cloned when splitting off.
Add infrastructure to pypeline that allows containers to be notified
when they are being used last, this allows two things:
* `Pipe`s eagerly clearing those containers once they are done reading
their contents
* `ScheduledTask`s clearing those out at the end of their execution in
case the pipe did not do it
This overall should improve memory usage as container no longer take up
memory if they are no longer used as part of a `Schedule`.
When linking globals that are later going to be uniqued by their id,
change their name to a temporary unique value and then restore it
post-linking, this avoids situations where function declarations shared
a common name but not metadata and were merged together by the linker.
Switch from using an array of integers with `ptrtoint` to using an array
of opaque pointers to store globals in an array to prevent their
deletion. This avoids spurious `ptrtoint`s when looking at uses of
globals.
Add another container for LLVM modules. This one stores one
`llvm::Module` per function (all modules share the same
`llvm::LLVMContext` for performance reasons).
This commit drops libptc in favor of its new form libtcg.
It brings several improvements, among which:
* The QEMU version we work on has been upgraded.
* CPUStateAccessAnalysis has been reimplemented in a way that makes it
easier to debug and solves some limitations (e.g., tracking leaking
pointers).
* Identification of pieces of the CPU state that are read by each helper
and fixing access to the CPU state is now performed at build-time.
* We no longer mmap the code we need to translate, dropping all the
issues related to code that needed to be mapped where something is
already present.
* We now have two distinct flavors of helper modules: the full one and
the "slim" one. The latter contains the definition only of functions
we intend to inline. It is used in most of the pipeline, a good thing
since we spend less time optimizing code we don't really care about.
The full module is only used on the re-compilation branch of the
pipeline.
* We no longer split the `cpu_loop` function.
* We change MetaAddress to rely on architectures from `model::` as
opposed to the LLVM ones.
* We no longer attach debug info to LLVM IR containing the original
assembly.
* We now verify that the lifted code only contains code we expect.
* Make the following private headers public:
* Lift/CPUStateAccessAnalysisPass.h
* Lift/CSVOffsets.h
* Lift/PTCDump.h
* Lift/VariableManager.h
* Move from revngSupport to revngLift:
* IRAnnotators.{h,cpp}
* SelfReferencingDbgAnnotationWriter.{h,cpp}
* Move from revngSupport to revngModel:
* FunctionTags.{h,cpp}
* ProgramCounterHandler.{h,cpp}
* Move from revngSupport to revngRecompile:
* OriginalAssemblyAnnotationWriter.{h,cpp}
getJumpTargetBlock takes llvm::BasicBlock * as argument, but doesn't
need to modify it, so const it adds const to it and other functions that
are used by getJumpTargetBlock: findJumpTarget, isTranslated, getType,
isJumpTarget.
This commit works around a quirk of
`llvm::ConstantDataArray::getString`, which returns a
`llvm::ConstantAggregateZero` for empty strings.
This quirk caused assertions in the previous implementation when passing
an empty `String` to `getUniqueString`.
Using an empty namespace causes the creation of a named metadata with an
empty string as name.
`llvm::verifyModule` does not check for that, so the `Module` still
verifies, but the on-disk serialization generated by `AsmPrinter.cpp`
prints `"empty name"` instead of the metadata name, and if the
serialized `Module` is then reparsed it parses succesfully but it does
not verify anymore.
`getUniqueString` returns a dedup'd char array `GlobalVariable` given a
string. We used to rely on named metadata to dedup them. However, this
lead to issues with `LLVMContainer` cloning and linking.
The new implementation uses the name of the variable as the
de-decuplication key, circumventing the problem entirely and improving
performance.
This commit introduces `BasicBlockID` as the unique identifier for a
`efa::BasicBlock` into the CFG. A `BasicBlockID` is defined by a
`MetaAddress` plus an incremental integer. This enables us to have
multiple instances of the same block in a single function, which is
particularly useful when inlining multiple times the same function.
Apart from this, the commit also does the following:
* It drops representing `MetaAddress`es a `structs` in the IR. This created
several issues related to ABI. We now represent them as strings.
* It defines more functions in `support.h`, instead of defining prototypes
by hand in `CodeGenerator.cpp` and the like. Specifically, `unknownPC`
and `raise_exception_helper`. We also introduce a C "constructor" for
`PlainMetaAddress`.
* It significantly reduces the API of `GeneratedCodeBasicInfo`, which
was supposed to be put on a diet since a long time. Specifically,
many jump target related methods have been moved to free functions in
`IRHelpers.h`. Also `GCBI::getSuccessors` has been pushed into its
only user, `PruneRetSuccessors`, to prevent further usage of a
deprecated API. In the future, it would be nice to drop it entirely.
* It introduces `efa::BasicBlock::InlinedFrom`.
* Introduce an enum to represent named argument indices for `newpc`.
This enables us to more effectively manipulate its argument list.
* It improves the verification and error reporting for
`efa::FunctionMetadata`.
* Update tests.
This commit is preliminary to another piece of work to improve the
generality of inlining beyond the simple "fake function" scenario, for
which the feature was originally conceived.
This is a big step to split revng-lift in two parts: one that only
writes the model and one that actually lifts to LLVM IR.
* Introduce `revng import binary`
* Split off `BinaryFile.h`
* Drop `revng.h`
* `GeneratedCodeBasicInfo`: use model
* Reduce role of `GeneratedCodeBasicInfo` in favor of
`model::Architecture` and `model::Register` methods
* `CodeGenerator`: adopt `RawBinaryView` and model
* `JumpTargetManager`: adopt `RawBinaryView` and model
* `ExternalJumpsHandler`: adopt model
* `InstructionTranslator`: discard `Architecture` in favor of
`EndianessMismatch`
* Many other changes
Giacomo Vercesi