This commit introduces the type system of the model along with several
various other improvements to the model and its users.
* Introduce the type system.
* Introduce possibility to tag certain fields in the model as to be
optional during YAML serialization.
* All the `Name` fields have been replaced in favor of `CustomName` plus
a `name` method that will use `CustomName` if available, or an
automatically generated name otherwise.
* Make TupleTreeReferences behavior more robust: now you either need to
have a valid pointer to `Root` and a `Path` or be default constructed
(`nullptr` for `Root` and an empty `Path`). Any other configuration is
invalid.
* The type system introduces `RawFunctionType`: this superseds the
previous way in which we were specifying arguments and return
values. Users of such information have been updated accordingly.
This commit drops the KeyTraits in favor of a std::any-like solution.
Basically, we type erase any key the user wants to employ, just exposing
a virtual version of the destructor, a comparison operator and a clone
primitive.
Before this commit, the execution logic of RecursiveCoroutine used an
underlying std::stack allocated on the heap to hold all the
coroutine_handles. It also manually managed passing return values from
callees to callers.
This commit drops this unnecessary auxiliary stack.
When a RecursiveCoroutine co_awaits another one, the handle of the
awaiter is injected into the awaitee, so that when the awaitee is done
it can directly execute the remaining part of the awaiter as a
continuation.
This commit takes out of EnforceABI the part taking care of creating
wrappers for calls to helpers and promoting CSV to local variables.
This decoupling, enables to run -promote-csvs multiple times, for
instance after inlining.
This commit greatly improves the performance by ensuring that, when
computing the set of nodes we want to consider for AVI, we do not
traverse the dispatcher.
Doing so, means including *a lot* of irrelevant nodes and wasting a lot
of computation, since the CFG usually is not influenced by stuff
happening before an indirect jump.
In at least a situation the speedup is in the order of 20x, however this
depends on the size of the binary, since traversing the dispatcher means
including all the binary in the computations (as opposed to just the set
of blocks involved in the dataflow to compute a certain expression).
This commit ensures that FunctionIsolation and EnforceABI do only
thing. This means that they no longer modify `root`.
Instead, we have a new pass, `invoke-isolated-functions` that needs to
be run after them and replaces the entry point of the functions with
invokes to the isolated functions, possibly with the appropriate
arguments.
When we compare StackAnalysis test results, the order in which things
appear in the JSON is relevant. However, the output was
non-deterministic due to a `std::map` using a pointer as key.
This commit improves the situation by sorting the elements by name
before dumping them in JSON.
This enables us to handle situations where the comparison against the
switch value is performed on the lower 32 bits, but the address
computations uses the full 64 bits.
cmp edi,0x23
ja ...
movsxd rdx,DWORD PTR [rcx+rdi*4]
Note that this does not solve all these situations yet.
This commit also introduces a reference output for the new
switch-jump-table-32-bit-comparison test case.
This commit drops support for running StackAnalysis without ABI
analysis. This has been broken for quite some time and a source of slow
downs in (badly) crafted optimization pipelines.
This commit:
* Drops `KeyTraits::toString`: if needed, use `getNameFromYAMLScalar`.
* Makes many methods in TupleTree.h return `nullptr` or `std::optional`
in order to gracefully handle failures.
* Provides `KeyTraits` specializations for integral types and tuple-like
composed by types providing `KeyTraits`.
* Introduces `CompositeScalar`, which enables tuple-like objects to be
YAML-serializable scalars by joining the YAML-serialization of its
members through a customziable character.
* Implements `PathMatcher`, a very simple "regular expression" mechanism
for paths on tuple trees.
* Introduce testing for the Model.
Before this commit, recursive coroutines did not work properly if they
had out arguments with reference type.
The reason is that `rc_run` was inferring the type of its arguments from
the arguments themselves, not from the prototype of the recursive
coroutine.
Hence, code snippets like the following did not work properly, because
`rc_run` was taking x by value, not by reference.
```
RecursiveCoroutine<void> accumulate_on_i(int &i) {
// ...
}
int f() {
int x = 0;
rc_run(accumulate_on_i, x);
return x;
}
```
This commit fixes the problem. Now the arguments of `rc_run` are
properly forwarded to the recursive coroutine.
This commit fixes a couple of bugs preventing SortedVector and
MutableSet from being serialized.
Also, it introduces minimal testing for serialization.
These data structures are substitutes for a `std::map<Key, Value>` where
`Key` is embedded in `Value`. Their main goal is to be serializable in a
YAML sequence while preserving the order enforced by the key.
`MutableSet` is implemented using a map.
`SortedVector` is implemented using a sorted vector.
RecursiveCoroutines are a facility intended to be used as-drop in
replacement of recursive functions.
They provide the following features.
- They can be written almost as regular recursive functions,
with 4 caveats.
1. A recursive coroutine that returns a type `T`, needs to be declared
to return a `RecursiveCoroutine<T>`.
2. Inside the body of a recursive coroutine, when recursively calling
another recursive coroutine, the recursive call needs to be
prepended by the new keyword `rc_recur`.
3. Inside the body of a recursive coroutine, the `return` statement
needs to be substituted with `rc_return`.
4. When launching a recursive coroutine `A` from a function that is
not a recursive coroutine, `A` needs to be called with the provided
dedicated template wrapper `rc_run`.
The syntax is the following `rc_run(A, arg0, arg1, ...)`.
This is necessary to enable swapping off recursive coroutine and
fall back to regular recursion for debug.
See below for how to do it.
- Unlike regular recursive functions, they don't use the system stack
for recursion. They use a custom heap-allocated stack to manage
recursion. This makes them more robust for implementing recursive
functions that manipulate user-defined input, because they are much
less likely to trigger stack overflow.
- They can be turned off compiling with
`-DDISABLE_RECURSIVE_COROUTINES`, falling back to regular recursion,
for debug purposes.
- They support both direct and indirect recursion, i.e. a recursive
coroutine A can recursively call itself, or it can recursively call
another recursive coroutine B, which in turns recursively calls A.
Sorting the BasicBlocks on which DisjointRanges works is detrimental for
performance. Using a random order takes less time. This is due to the
fact that the number of BasicBlocks to analyze is significantly smaller
than the whole list of BasicBlocks.
This commit drops the `--isolate-no-safety-checks` flags since we
adopted a new approach later on in the decompilation pipeline, i.e.,
dropping calls to `raise_exception_helper` and running dead code
elimination.
We recently bumped the GCC version used for the x86-64 toolchain. The
newer version makes slightly different choices of instruction breaking
the names of basic blocks.
This commit drops them, since they are not really necessary: checking
the order is already more than enough.
These are necessary for this iterator to be a forward_iterator according
to the standard.
Missing these causes compilation errors in some cases when using
functions from the Standard Template Library with these iterators.
Ivan Krysak