This commit greatly improves the performance by ensuring that, when
computing the set of nodes we want to consider for AVI, we do not
traverse the dispatcher.
Doing so, means including *a lot* of irrelevant nodes and wasting a lot
of computation, since the CFG usually is not influenced by stuff
happening before an indirect jump.
In at least a situation the speedup is in the order of 20x, however this
depends on the size of the binary, since traversing the dispatcher means
including all the binary in the computations (as opposed to just the set
of blocks involved in the dataflow to compute a certain expression).
* Introduce GCBI::buildDispatcher
* Introduce GCBI::getJumpTarget{,Block} and GCBI::getBlocksGeneratedByPC
to easily map `BasicBlock *` to jump targets and viceversa.
* PCH::buildDispatcher now returns a list of the newly created basic
blocks.
* Other minor changes
Changes include:
- `getFunctionCall` has been moved in IRHelpers.h
- `getFallthrough` and `getFunctionCallCallee`
have been simplified and added in IRHelpers.h (their old versions
have been removed respectively from FCI.h and revng.h)
- `FCI::getCall` and `FCI::isCall` have been removed
due to redundancy with `getFunctionCall`.
Sorting the BasicBlocks on which DisjointRanges works is detrimental for
performance. Using a random order takes less time. This is due to the
fact that the number of BasicBlocks to analyze is significantly smaller
than the whole list of BasicBlocks.
This commit introduces the PruneRetSuccessors pass, whose role is to
identify all the indirect jumps whose devirtualized destinations
correspond to return addresses. In fact, they are most likely to be
return instructions and devirtualizing them is always detrimental.
This commit makes possible to execute the GeneratedCodeBasicInfo (GCBI)
Pass even on LLVM IR where the AnyPC and UnexpectedPC BasicBlocks are
not present.
This makes the GCBI pass more flexible, enabling more other passes to
depend on it and to use it to retrieve informations on the generated
code without architecture-dependent hacks.
As an example, one can now use GCBI to retrieve the CSV representing the
Stack Pointer Register or the Program Counter Register without relying
on the register names (which are architecture-specific), even if some
optimization pass along the decompilation pipeline has removed AnyPC or
UnexpectedPC.
Notice that the contracts of GCBI's methods has not been changed.
Calling anyPC() or unexpectedPC() still asserts that those are not
nullptr.
The contract has just been moved from the execution stage of the pass
(basically runOnModule), to the APIs used to query the results.
Unlike the previous iteration of `MetaAddress`, which tried to stuff all
the parts of `MetaAddress` within the existing `PC` CSV, this
implementation adds a set of new CSVs (or marks some existing ones as) to
represent the four portions of the current PC's `MetaAddress`.
* Introduce `ProgramCounterHandler`: a class responsible to maintain the
PC-related CSVs. This class is also used to manipulate the new
dispatcher.
* `AdvancedValueInfo`: update for new MetaAddress.
* External jump handler: do not clobber registers.
When introducing support for dynamic binaries, we didn't realize that
in x86-64 we were clobbering `r11`. To avoid this, we have to jump to
an address stored in memory. However, due to the new `MetaAddress`,
obtaining a *jumpable* address from the PC-related CSVs might require
some computations (and it does in ARM). Therefore, we introduce a new
global variable, `jumpablepc`, whose only role is to contain the
jumpable version of the program counter and then be the target of the
memory-indirect jump instruction.
* Labels care only about absolute addresses.
* CSAA: mark call site, even if no accesses.
`MetaAddress` replaces all the `uint64_t` used to represent a virtual
address. Its main features are:
* It has a non-zero representation of invalid addresses.
* It supports tags to represent code that has different interpretations but
resides at the same address in memory (namely ARM vs Thumb).
* Arithmetic operations cannot overflow.
* It supports epochs, a way we intend to employ to handle self-modifying code
(i.e., different code at the same address at different times).
* It supports "address spaces", which enable handling architectures with
multiple address spaces.
* It fits in two 64-bit registers.
* Drop unused argument names from function prototypes
* Make `static` some methods
* Disable some copy constructors
* Fix casing of Doxygen `\file` directives
* Add some casts to make the compiler happy
* Initialize `hasRelocationAddend` for AArch64
* Use references in range-for where possible
* Drop default for `switch` statements covering all the entries of an `enum`
* Make some global variables `static`
* Drop dead functions
Handle `ConstantPointerNull` explicitly in Advanced Value Info, instead
of trying to obtain the analysis result for its operand (which is by
design a `nullptr`).
LLVM 9 drops the `TerminatorInst` class. This commit replaces it with
`Instruction` where possible and asserts
`Instruction::isTerminator()`. It also switches from
`TerminatorInst::successors` to `successors(TerminatorInst *)`.
* Introduce `ShrinkInstructionOperandsPass`: a transformation shrinking
operands and the results of instructions if they are
zero/sign-extended immediately before and after the instruction.
* Introduce `ConstantRangeSet`: similar to `ConstantRange` but allows
disjoint ranges.
* Introduce `MaterializedValue`: a class that can represent a constant
value or a symbol plus offset pair.
* Introduce `DropHelperCallsPass`: a transformation removing calls to
helpers and replacing them with a function call reading the CSVs that
the helper reads and writing the CSVs that the helper writes
(according to CSAA).
* Introduce `DropRangeMetadataPass`: a transformation dropping the
`range` metadata, which, in certain situations, lowers the quality of
the results provided by `LazyValueInfo`.
* Introduce `AdvancedValueInfo`: an analysis exploiting results of
`LazyValueInfo` but collecting them as `ConstantRangeSet` with a
monotone framework. It produces `MaterializedValue`.
* Anticipate linking of helpers: `AVI` requires `CSAA`, which requires
helper functions to be linked in.
* Drop `--no-link`.
* Force x86-64 `DataLayout`.
* Reorganize harvesting to either collect simple literals or go with
(incremental) `AVI`.
* Drop `SET`, `OSRA`, the reaching definition analysis, the
`SimplifyComparisonsPass` and all the sumjump-related code: e now
clone `root`, optimize it and analyze it with `AVI`.
* Temporarily drop the `NoReturnAnalysis`.
* Link `libLLVMInstCombine`, `libLLVMCodeGen` and `libLLVMPasses`.
* Introduce tests for `AdvancedValueInfo`,
`ShrinkInstructionOperandsPass` and `ConstantRangeSet`.
* Fix test results.
* Add `llvm.bswap.i64` and `@pc` to the LLVM template module for unit
tests.
* `GeneratedCodeBasicInfo::getCSVUsedByHelperCall` and
`GeneratedCodeBasicInfo::extractCSVs`: make the call argument an
`Instruction`.
* Introduce `blockByName`
* Introduce `getUniqueUser`.
* Fix linking issues.
`empty-newpc` is a simple pass suppose to provide an empty body for
the `newpc` function, allowing further optimization pass to take out
all its calls.
In the stack analysis, we used to consider helper functions as indirect
calls. However, the `CPUStateAccessAnalysis` provides us accurate
information about what an helper function does.
This commit transforms calls to helper functions in a series of ABI IR
instructions reading all the input registers of the helper functions and
a series of instructions writing the output registers.
Note that, while this is a serious improvement over considering them
indirect function calls, it's still suboptimal since
`CPUStateAccessAnalysis` doesn't provide us information as fine-grained
as the ABI analysis.
Previously, to enumerate all the CSVs we had to go through the
`GlobalVariable` of a `Module` and see if the were being used in
rev.ng-generated code.
Now we have a named metadata for that: `revng.csv`.
We used to save the type of a block in the `revng.block.type` metadata
as a number. This commit serializes it as a string.
In order to do this, the `BlockType` enum has been promoted to a
namespace with the usual `getName` and `fromName` functions.
This commit does the following:
* It drops `revamb-dump` and transforms all the passes it featured in
passes that can be used directly from `opt`.
* It rename `revamb` to `revng-lift`.
* It introduces a script called `revng` which acts as a driver for the
whole rev.ng project. It replaces `translate`, `revcc`,
`csv-to-ld-options` and `revamb-dump`, since it offers an `opt`
subcommand which allows to easily invoke all the analysis passes.
* It makes the project a CMake package that can be easily used
externally.
* It allows to easily create libraries of analysis to use through
`revng-opt`.
This commit lets the reaching definitions analysis employ results from
the stack analysis to propagate definitions across functions
calls. Specifically, the stack analysis provides a list of registers
that might be clobbered by the callee: definitions concerning those are
not propagated, all the others are propagated.
This change is key to detect jump tables whose address has been
materialized *before* a function call. A test for such situation has
been introduced.
To make this work, the RDA now works over the CFG provided by the
function identification analysis.
The `FunctionCallIdentification` analysis now provides a custom view on
the CFG where 1) dispatcher-related basic blocks are absent, 2) nodes
performing functions calls have an edge to their return address and 3)
nodes ending with a return instruction have no successor.
This CFG is now employed by the reaching definitions analysis and OSRA.
Additionally, the implementation of the `visitSuccessors` and
`visitPredecessors` method has been reviewed. It now consists in a class
that needs to be inherited and for which two methods should be
implemented, one to perform the visit of a block and another one to
enumerate the successors.
In addition, all the users of `visitSuccessors`/`visitPredecessors` have
been updated, a simple set of tests has been introduced and
`GeneratedCodeBasicInfo::visitPredecessors` has been dropped.
The Reaching Definitions Analysis used to ignore all the memory access
that were not relative to a CSV or an `alloca`. In certain situations,
in particular in x86 which makes heavy use of the stack, this led to
miss certain links that were vital for a correct identification of
certain jump targets.
As an example consider the following example:
cmp DWORD PTR [rbx+0x8],0x5
ja 4bc5c8 <uw_update_context_1+0x148>
mov eax,DWORD PTR [rbx+0x8]
movsxd rax,DWORD PTR [r12+rax*4]
add rax,r12
jmp rax
The constaint on `rbx+0x8` is not propagated to the use after the jump,
due to a missing link.
This commit fixes this by tracking all the memory accesses within the
expressive power of the `MemoryAccess` class, in particular those
expressed as register plus offset.
Additionally, this commit also makes RDA consider as reaching
definitions for a `load` only those definitions that match exactly the
load address, while previously we were considering all the `mayAlias`
definitions. This situation, in combination with the previous change,
led to have paths with multiple reaching definitions, which was
problematic for the path sensitive merge.
Finally, we consider a definition to be clobbered only if there's an
exact store to the esame address, instead of a store on an aliasing
address.
In function isolation, every time we met a jump to an unexpected basic
block (i.e., a basic block that is not part of the current function), we
used to throw an exception. However this is unnecessary since oftentimes
it is sufficient to call the `function_dispatcher` or even perform a
regular function call.
The most obvious example is the case of a direct tail call. In this
situation performing a function call to the corresponding isolated
function is the most appopriate thing to do.
Alessandro Di Federico