RecursiveCoroutines are a facility intended to be used as-drop in
replacement of recursive functions.
They provide the following features.
- They can be written almost as regular recursive functions,
with 4 caveats.
1. A recursive coroutine that returns a type `T`, needs to be declared
to return a `RecursiveCoroutine<T>`.
2. Inside the body of a recursive coroutine, when recursively calling
another recursive coroutine, the recursive call needs to be
prepended by the new keyword `rc_recur`.
3. Inside the body of a recursive coroutine, the `return` statement
needs to be substituted with `rc_return`.
4. When launching a recursive coroutine `A` from a function that is
not a recursive coroutine, `A` needs to be called with the provided
dedicated template wrapper `rc_run`.
The syntax is the following `rc_run(A, arg0, arg1, ...)`.
This is necessary to enable swapping off recursive coroutine and
fall back to regular recursion for debug.
See below for how to do it.
- Unlike regular recursive functions, they don't use the system stack
for recursion. They use a custom heap-allocated stack to manage
recursion. This makes them more robust for implementing recursive
functions that manipulate user-defined input, because they are much
less likely to trigger stack overflow.
- They can be turned off compiling with
`-DDISABLE_RECURSIVE_COROUTINES`, falling back to regular recursion,
for debug purposes.
- They support both direct and indirect recursion, i.e. a recursive
coroutine A can recursively call itself, or it can recursively call
another recursive coroutine B, which in turns recursively calls A.
Sorting the BasicBlocks on which DisjointRanges works is detrimental for
performance. Using a random order takes less time. This is due to the
fact that the number of BasicBlocks to analyze is significantly smaller
than the whole list of BasicBlocks.
This commit drops the `--isolate-no-safety-checks` flags since we
adopted a new approach later on in the decompilation pipeline, i.e.,
dropping calls to `raise_exception_helper` and running dead code
elimination.
We recently bumped the GCC version used for the x86-64 toolchain. The
newer version makes slightly different choices of instruction breaking
the names of basic blocks.
This commit drops them, since they are not really necessary: checking
the order is already more than enough.
These are necessary for this iterator to be a forward_iterator according
to the standard.
Missing these causes compilation errors in some cases when using
functions from the Standard Template Library with these iterators.
This patch adds a two markers for llvm::GraphTraits:
- NodePairFilteredGraph
- EdgeFilteredGraph
Both these markers allow to specify a static predicate that is used to
filter edges.
This predicate is a boolean function such that:
- for NodePairFilteredGraph, it takes a pair of const NodeRef & that
are used to represent an edge, and it evaluates a given property of
that pair;
- for EdgeFilteredGraph, it takes a const EdgeRef & that represents an
edge, and it evaluates a given property on the edge.
The filtered graph contains only the edges for which the predicate
evaluates true.
Notice that the predicate must have static lifetime, meaning that all
the edge properties must be entirely evaluated on the pair of node (for
NodePairFilteredGraph) or on the edge (for EdgeFilteredGraph).
This means that you cannot pass mutable state to the predicate at
runtime.
The new markers are designed to interoperate well with llvm::Inverse and
to allow you to traverse the marked graphs with llvm::depth_first,
llvm::inverse_depth_first, llvm::breadth_first, and to compute dominator
trees and post-dominator trees on filtered graphs.
Some tests have been disabled in the past, but they work.
Specifically, this commit re-enables the `memset` and `fibonacci` tests.
The commit also describes why the test remaining disabled are so.
Unlike the previous iteration of `MetaAddress`, which tried to stuff all
the parts of `MetaAddress` within the existing `PC` CSV, this
implementation adds a set of new CSVs (or marks some existing ones as) to
represent the four portions of the current PC's `MetaAddress`.
* Introduce `ProgramCounterHandler`: a class responsible to maintain the
PC-related CSVs. This class is also used to manipulate the new
dispatcher.
* `AdvancedValueInfo`: update for new MetaAddress.
* External jump handler: do not clobber registers.
When introducing support for dynamic binaries, we didn't realize that
in x86-64 we were clobbering `r11`. To avoid this, we have to jump to
an address stored in memory. However, due to the new `MetaAddress`,
obtaining a *jumpable* address from the PC-related CSVs might require
some computations (and it does in ARM). Therefore, we introduce a new
global variable, `jumpablepc`, whose only role is to contain the
jumpable version of the program counter and then be the target of the
memory-indirect jump instruction.
* Labels care only about absolute addresses.
* CSAA: mark call site, even if no accesses.
* Drop unused argument names from function prototypes
* Make `static` some methods
* Disable some copy constructors
* Fix casing of Doxygen `\file` directives
* Add some casts to make the compiler happy
* Initialize `hasRelocationAddend` for AArch64
* Use references in range-for where possible
* Drop default for `switch` statements covering all the entries of an `enum`
* Make some global variables `static`
* Drop dead functions
This commit performs the changes necessary in order to integrate with
revng-qa, the new project for cross-project quality assurance.
Basically, the source code of all the tests has been moved in revng-qa,
which will take care of producing "artifacts" (i.e., compiled programs),
using the appropriate cross-compilers.
revng will then consume them and produce new artifacts to be consumed by
other tools down the pipeline.
The directory structure of the tests has been reworked to reflect
`revng-qa`. A large amount of boilerplate code has been dropped.
Note that certain actions, that used to be carried out during testing,
are now part of the regular build process. Specifically, lifting the
tests is performed at build time, so that they can be installed.
A `ConstantRange` such as `[5,0)` was not handled correctly in
`ConstantRangeSet` due to a spurious 0 at the end of the range.
This commit also fixes the testing infrastructure that was not checking
the size of the range before making the comparison with the reference
vector.
This commit replicates the structure of the install directory into the
build directory in order to simplify the management of RPATHs.
This commit also drop `QEMU_INSTALL_PATH`.
* Introduce `ShrinkInstructionOperandsPass`: a transformation shrinking
operands and the results of instructions if they are
zero/sign-extended immediately before and after the instruction.
* Introduce `ConstantRangeSet`: similar to `ConstantRange` but allows
disjoint ranges.
* Introduce `MaterializedValue`: a class that can represent a constant
value or a symbol plus offset pair.
* Introduce `DropHelperCallsPass`: a transformation removing calls to
helpers and replacing them with a function call reading the CSVs that
the helper reads and writing the CSVs that the helper writes
(according to CSAA).
* Introduce `DropRangeMetadataPass`: a transformation dropping the
`range` metadata, which, in certain situations, lowers the quality of
the results provided by `LazyValueInfo`.
* Introduce `AdvancedValueInfo`: an analysis exploiting results of
`LazyValueInfo` but collecting them as `ConstantRangeSet` with a
monotone framework. It produces `MaterializedValue`.
* Anticipate linking of helpers: `AVI` requires `CSAA`, which requires
helper functions to be linked in.
* Drop `--no-link`.
* Force x86-64 `DataLayout`.
* Reorganize harvesting to either collect simple literals or go with
(incremental) `AVI`.
* Drop `SET`, `OSRA`, the reaching definition analysis, the
`SimplifyComparisonsPass` and all the sumjump-related code: e now
clone `root`, optimize it and analyze it with `AVI`.
* Temporarily drop the `NoReturnAnalysis`.
* Link `libLLVMInstCombine`, `libLLVMCodeGen` and `libLLVMPasses`.
* Introduce tests for `AdvancedValueInfo`,
`ShrinkInstructionOperandsPass` and `ConstantRangeSet`.
* Fix test results.
* Add `llvm.bswap.i64` and `@pc` to the LLVM template module for unit
tests.
Any stack pointer value greater than or equal to the original one used
to be OK for stack analysis to recognize an instruction as a
return. This commit changes this policy by collecting the value of the
stack pointer on all the return points and, at the end, elect a final
stack pointer value. All non-compliant returns are marked a
`BranchType::BrokenReturn`. The function is now considered fake only if
all the return instructions agree on a specific value of the stack
pointer, and it's lower than the original one.
* Drop `BranchType::IndirectTail`.
* `s/BranchType::FunctionSummary/BranchType::RegularFunction/`.
A series of algorithms working on `llvm::GraphTraits`.
* `nodesBetween` computes the set of nodes on all the paths from a node
A to B.
* Factor out code to iterate over infinite loops into
`exitless_scc_range`.
* Introduce unit tests.
* `GeneratedCodeBasicInfo::getCSVUsedByHelperCall` and
`GeneratedCodeBasicInfo::extractCSVs`: make the call argument an
`Instruction`.
* Introduce `blockByName`
* Introduce `getUniqueUser`.
* Fix linking issues.
We used to list all the architectures supported by QEMU in tests,
however this not optimal. This commit switches to a whitelist for the
list of architectures to tests so that we don't test architectures for
which we have a toolchain but aren't ready for the testsuite.
`ClassSentinel` makes use of an undefined behavior to check if an object
has been destroyed. Disable this check in case of presence of address
sanitizer or even regular optimizations.
This commit introduces the `enforce-abi` pass, which consumes the
information provided by the ABI analysis and enforces them in the
isolated functions adding actual arguments.
This commit also rewrites the logic of `ResultsPool::finalize` and
changes the semantic of `Yes` statements on arguments to `YesOrDead`.
`ZipMapIterator` allows you to iterate in parallel over two
`std::map`-like containers.
In the ABI analysis, this allows us to be much more efficient. In
practice, if we have two maps with M and N elements, we pass from
performing N*log(N) + M*log(M) queries to the size of the union of the
set of keys of the two maps.
The `BFSVisitorBase` can go forward and backward starting from a certain
instruction. When going backward, it used to skip the starting
instruction.
For simmetry purposes, this commit changes that.
`LazySmallBitVector` was lacking a comparison operator, which prevented
it from being used as the key of a `std::map`.
This commit implements such operator, along with the `reserve` method.
All the definitions of `greaterThan` were `!lowerThanOrEqual`, and all
the uses were implicitly assuming this semantic.
However, this was confusing because in a Lattice the ordering is not
total, hence `!lowerThanOrEqual` is not equivalent to `greaterThan`.
This commit drops the `greaterThan` method altogether to avoid
confusion.
This commit drops the old FunctionBoundariesDetectionPass and introduces
a new one based on the results provided by the StackAnalysis. A very
similar pass, the ABIDetectionPass, is now available to offer the
results of the ABI analysis too.
These two new passes are a thin shim depending on the appropriate
version of the StackAnalysis (with or withour ABI anlysis) and simply
call `serializeMetadata`, which decorates the LLVM IR with the requested
information.
In addition to drop the old analysis, this commit also isolates the
function boundaries detection pass from `revamb` making it available as
a library only.
This commit does the following:
* It drops `revamb-dump` and transforms all the passes it featured in
passes that can be used directly from `opt`.
* It rename `revamb` to `revng-lift`.
* It introduces a script called `revng` which acts as a driver for the
whole rev.ng project. It replaces `translate`, `revcc`,
`csv-to-ld-options` and `revamb-dump`, since it offers an `opt`
subcommand which allows to easily invoke all the analysis passes.
* It makes the project a CMake package that can be easily used
externally.
* It allows to easily create libraries of analysis to use through
`revng-opt`.
Pietro Fezzardi