`Logger` is the new infrastructrure for debug messages. They are
supposed to be used by just creating a global variable in a translation
unit and then writing there directly as if they were a stream.
Key features:
* Self-registration: a `Logger` by default register itself automatically
in a register. This means that at run-time, unlike with `DBG`, we have
a list of all the possible `Logger`s. One benefit of this approach is
being able to list all the available `Logger`s in `--help`.
* Indentation: `Logger`s can be indented. In particular the
`LoggerIndent` helper class can automatically indent a certain
`Logger` during the lifetime of its instance.
* Atomic messages: a message is no longer simply delimited by a "\n": to
mark the end of a message, an instance of `LogTerminator` has to be
streamed to the `Logger`. This can also be associated directly to the
lifetime of an object by using the `LogOnReturn` class.
* Custom class formatting: it is possible to decide how a certain object
should be formatted when sent to a `Logger` by implementing the
`writeToLog` function. For example, `llvm::Value`-derived objects are
passed through the `getName` function. If no custom formatter is
specified, the `operator<<` is employed.
Sometimes, dealing with move-semantics in C++ can be challenging. To
mitigate this problem, this commit introduces `ClassSentinel`, a simple
class that can be added as a member of a class and that will allow the
user to easily monitor if an instance of a class is used after being
moved in an unwanted way. To do so, simply call the
`ClassSentinel::check` method on the class member.
`ClassSentinel` performs similar (but less reliable) checks on usage of
destroyed objects.
`ClassSentinel` can also (optionally, though the `SENTINEL_STACKTRACES`
macro) collect stack traces of the points where the object was moved or
destroyed.
This commit also includes some basic testing for the class.
`SmallMap` is a `std::map`-like data structure whose storage is inline
if the number of elements is small, pretty much as `llvm::SmallVector`
and friends. In case the `SmallMap` is in its small form, the search
cost is linear in the number of elements.
This commit also introduces `Iteratall`, an iterator class that can wrap
iterators of different types but with the same `::value_type` and
`::reference` types. This is used to abstract away the fact that
`SmallMap` iterators can either be iterators over a `std::array` or
`std::map`.
Note that if you want to iterate on the elements, and you want to be
sure that they are ordered (as happens with `std::map`), you have to
call the `sort` method before, which might or might not triggered a
sort. Note also that the internal inline storage uses a `std::array`,
which means that the `K` and `V` must be default constructible.
SET was predisposed to be able to track all the addresses from which a
load has been performed, but this has never been fully implemented. This
commit does that.
It is important to know that a load has been performed from a certain
address since, most likely, this means that part of the program is not
code. In fact, this information is used to tag jump targets, so that
other analysis can take into account this fact. Note however that, at
the current stage, the jump target itself is preserved (and, therefore,
translated).
`reinterpret_cast`s can lead to undefined behaviors, since the compiler
can assume that each object will be accessed exclusively through
pointers of a single type.
Enabling strict aliasing and its warnings, even in debug builds, allows
us to catch this kind of problems earlier.
If CMake detects `libhello` correctly and it is present in the system
paths, it passes it to the linker as `-lhello`. However, sometimes, in
particular when linking Boost, the compiler detects a different version.
Forcing the CMake policy `CMP0060` to the new approach solves the
problem, since CMake now always uses the full path of the required
library, even if it's already available in the system paths.
`skipCasts` used to skip over `CastInst`, now it also skips `BitCast`
`ConstantExpr`s.
Additionally, `isCallTo` now uses `skipCasts` and a `isCallToHelper`
function has been introduced to check if an instruction is a call to a
helper function.
This commit introduces support for dynamic programs. The current
implementation translate the main binary and uses native libraries. This
works only if the target architecture is the same as the source
one. Currently we only handle x86-64.
* The `ExternalJumpsHandler` class has been introduced. It basically
takes care of extending the dispatcher handling the case in which the
program counter is an address outside the range of executable
addresses of the input program. In this case, a `setjmp` is perfomed,
the CPU state is serialized to physical registers and jump to the
value of the program counter is performed.
Once the target code will try to return to the translated program, a
segmentation fault will be triggered, a `longjmp` is performed and the
CPU state is deserialized so that the execution can resume (from the
dispatcher).
* `early-linked.c` has been introduced. Its purposes is to provide
declarations of variables and functions defined in `support.c`. In the
past, we had to manually create these definitions, a cumbersome and
error prone we now avoid by letting `clang` compile `early-linked.c`
and then linking it in.
* The old `support.h` is now known as `commonconstants.h`. `support.h`
now contains declarations that have to be consumed by
`early-linked.c`.
* Each architecture now provides additional information:
1. Which registers are part of the ABI and have to be preserved. If
necessary the QEMU name can be provided. For each register it's
also possible to provide their position within the `mcontext_t`
structure, provided by the signal handler.
2. Three assembly snippets, one to write a register, one to read it
and one perform an indirect jump.
Some of this information is also exposed in the output module as
metadata.
* `support.c` now installs a SIGSEGV signal handler. Since pages that
were originally executable are no longer executable, jumping there
(typically, from a library) will trigger a SIGSEGV that we will
handle. This allows us to properly deserialize the CPU state and
resume execution of the translate code.
* Now also a dynamic version of each test program is translated and
tested.
* The `merge-dynamic.py` script has been introduced: it takes case of
rewriting the translated binary so to tell the linker to performe both
the relocations of the translate program and the relocations of the
original program. It does so by rewriting a large portion of the
sections employed by the dynamic linker such as `.dynamic`, `.dynsym`
and so on.
* The `compile-time-constants.py` script has been introduced: it a
user-specified compiler on a source file producing an object
file. This object file is inspected and the value of global read-only
variables is produced in a CSV.
This ensure floating point memory accesses work properly on x86. In the
past this was working only in certain cases depending on the number of
entries in the auxiliary vector, arguments and environment variables.
This commit makes `revamb` produce a new file `.ll.need.csv` containing
a list of all the dynamic libraries required by the input program. This
will be transformed by the 'csv-to-ld-options` (was:
`li-csv-to-ld-options`) into the appropriate linking options.
The `argparse` library treats boolean arguments as
integers. Specifically, each time a boolean argument is meet the
associated variable is incremented. This led to weird behaviors having
`2` being converted to `false`. Using `int` as a type solves this issue.
This commit introduces the Function Isolation Pass. We use the
information provided by the Function Boundaries Detection Pass to
organize the code that `revamb` places inside the `root` function in
different LLVM functions. To do this we obviously need to introduce some
changes and tricks to handle the execution of the translated program.
The main idea is to have two different realms (one where the isolated
functions live, one in which we have basically the old root function).
We start the execution from the realm of the *non isolated* functions,
and we transfer, as soon as possible, the execution to the *isolated
functions* realm. We then have a fallback mechanism to restore the
execution in the right place in the *non isolated* functions realm, and
so on.
The largest change, besides the re-organization of the code in different
functions, is the use of the exception handling mechanism provided by
the LLVM framework in order to be able to manage the switch between the
two realms.
We also introduce the `support.h` header file, which contains a couple
of definitions used by `support.c` and that need to be shared with some
of the components involved in the translation process. We have defined
some helper functions, directly in C, that we use both for handling the
exception mechanism and for giving extra debug informations when an
exception is raised.
The `revamb-dump` utility now supports the `-i` option to specify the
path were to save the new LLVM module.
The `translate` utility now supports the `-i` option that produces a
binary in which the function isolation has been applied.
We also introduced some tests that apply the function isolation pass to
the `Runtime/` tests already present. In this way we can verify that the
translation and the following function isolation preserve the behavior
of the program.
When serializing the new LLVM module we regenerate the metadata used for
debug purposes, and for doing this, since we not longer have only the
`root` function, we have changed some details in the `DebugHelper` class
in order to be able to emit the metadata for all the functions of our
interest in a single shot.
Solved a bug that always allocated an `i32` for representing the return
address PC in the `function_call` helper (third parameter).
Now we allocate an `i64` or an `i32` depending on the input architecture
of the translated binary.
The header of an ELF file contains two bytes representing the ISA.
These two bytes are offset 0x12 in the file, which is offset 18, not
offset 17 like it was in the script.
`GenericFunctor` is substituted with the `std::integral_constant`
template. This also allows us to remove code that requires C++14.
It also removes the now useless cmake tests on the compiler flag
`-Wno-error=noexcept-type` that was introduced to disable fatal warnings
on the type `GenericFunctor`.
Now that this type has been removed the check is not necessary anymore,
because the `std::integral_constant` template used now does not cause
the warning.
So we can go back to enabling the fatal warnings.
Changed the names of the global variables (removed the leading `.`)
representing the segments of the binary, in order to prevent errors with
duplicated names when recompiling a binary with `llc` in debug mode.
We used to check if the value of `/proc/sys/vm/mmap_min_addr` is at
least as high as the minimum segment of the input binary. If this is not
the case the linked program will segfault at run-time without much
explanation.
However, in truth, we need to be able to map also the page before the
lowest page the original binary mapped. The main reason for this is to
have space for the (outer) ELF header.
It turns out that on many distros the default minimum value is
`0x10000`, which happens to be exactly the same address at which ARM
binaries mmap their lowest page. This lead to no warning, but a segfault
at run-time.
The AWK script now checks for the correct value, and also suggests the
correct value.
In the future, we might want to create a new segment for the outer ELF
header and position it elsewhere in the address space.
We now take advantage of a macro to add a series of compilation flags,
macro that also takes care of checking that the flags are supported by
the compiler.
This patch has been developed by Alessandro Di Federico.
The check to see if a compiler supports the `no-pie` flag was done only
for the main C compiler, and not for the cross-compilers used for
creating the executables for the different supported architectures.
This commit introduces the aforementioned missing checks.
In addition instead of hard-coding the flags to check in the CMakeLists
file we have a list that we pass each time we instantiate a project for
the cross-compilers, and we check for the availability of all the flags.
In order to do this we need to apply a sort of serialization and
deserialization to avoid the "unpack" of the list passed as argument to
the external project (that is implemented as a `;` separated string).
Also implemented a fix suggested in the merge request for a line that
mistakenly added the `TEST_CFLAGS` variable to the `NO_PIE` variable.
Many Linux distributions prevent programs from mapping memory pages at
low addresses. This can lead the translated program to segfault without
any additional explanation. This happens in particular with ARM
binaries, which tend to have the first segment allocated at very low
addresses.
This behavior can be configured through
`/proc/sys/vm/mmap_min_addr`. This commit introduces a warning to the
user in the `li-csv-to-ld-options` script in case a segment with an
address lower than `mmap_min_addr` is requested.
A previous commit introduced `-no-pie` to disable PIE in GCC versions
higher than 5.2. However, earlier versions don't support such an option.
This commit introduces the necessary detection mechanism to enable it or
not.
This commit introduces `value_type` in `LazySmallBitVector` to fix
compilation error on Boost versions newer than 1.65.
The bug was triggered in unit tests.
Add this flag to the flags used for Runtime tests and to the flags used
in the translate script.
Recent GCC versions (`gcc-7` and later) enable PIE by default, and
`-fno-pie` apparently is not enough to disable it.
This warning was introduced with gcc-7 to (quoting the documentation)
"Warn if the C++1z feature making noexcept part of a function type
changes the mangled name of a symbol relative to C++14. Enabled by -Wabi
and -Wc++1z-compat.".
It is triggered from the `GenericFunctor` class template. It can be
safely made not fatal, because this class template is not exposed
outside and the whole project is currently compiled with the same C++
standard compiler flags.
This commit adds machinery to `CMakeLists.txt` to make the warning not
fatal, but only if present. Disabling it when not present would trigger
build errors.
When we have an indirect call (or jump) we are sometimes able to
identify one or more possible targets, therefore, as an optimization,
before performing the indirect jump we check if the target is one of the
expected ones.
This optimization however was creating two issues with the handling of
indirect function calls: 1) the call to the `function_call` marker was
no longer positioned right before the terminator and 2) the function
call was no longer identified as an indirect function call but as call
to `anyPC`. This commit fixes these two issues.
These issues have been identified thanks to a report from Andrea
Gussoni.
The stack analysis is the foundation to obtain accurate information
about the body of a function, which registers are callee-saved,
arguments, return values and so on.
It is implemented as a pass to run in revamb-dump.
This commit also introduces analysis tests specific to what we aim to
obtain from the analysis and also some basic unit tests for data
structures related to the stack analysis.
`LazySmallBitVector` is a bit set data structure which holds up to 31/63
bits in place (depending on the pointer size), and if more than that is
needed, the required space is automatically allocated on the heap.
`LazySmallBitVector` also features lazyness, meaning that any bit index
that has never been set will return `false`, while every time a certain
bit needs to be set the data structure is automatically enlarged, if
necessary.
The `NoReturnAnalysis` now tracks, through metadata, not just the fact
that a basic block is a killer basic block, but also the reason why it
is. This let's the user know whether it's a killer because leads to an
actual killer basic block or because it is a killer syscall/endless
loop.
`GeneratedCodeBasicInfo` now exposes the instruction alignment of the
current architecture, the stack pointer register, the size of the PC
register, a reference to `anyPC` and correctly handles the
`DispatcherFail` basic block.
This commit extendes the FunctionCallIdentification pass to identify,
for each function call, where the return address is stored, i.e., the
link register. If the the return address is stored on the top of the
stack then the link register is `nullptr`.
This information is encoded as an extra argument to the marker
`function_call`.
This commit also makes the pass ignore dispatcher-related basic blocks
and ensures that calls to `function_call` are placed *before* calls to
`exitTB` so that they won't get purged.
Most of the times, when we need to get the next instruction, we actually
want to skip over "marker" function calls (e.g., calls to `newpc` and
`function_call`). `nextNonMarker` does exactly this.
`FunctionCallIdentification::isCall` and `JumpTargetManager::setCFGForm`
have also been extended to correctly handle such situations.
Serialization (in the form of the `revamb.jt.reasons` metadata) of the
reason why a certain address is a jump target has been moved from
`JumpTargetManager::finalizeJumpTargets` to a new function
(`JumpTargetManager::createJTReasonMD`) which is invoked after the
function boundaries detection algorithm has been run.
Alessandro Di Federico