This commit introduces support for dynamic objects. We do not support
translating dynamic libraries yet, therefore this commit introduces
support for PIE programs.
At the current stage, QEMU does not provide us explicit information
about an instruction using the program counter, but introduces its value
as an immediate. As a consequence, we cannot support arbitrary
relocation. For this reason, we statically relocate the program to a
fixed address (`0x50000000` by default, but it can be customized through
the `--base` argument). Therefore, all the addresses read from ELF data
structure need to be relocated.
Code compiled with `-fPIC` cannot store in global data the address of a
function, since it will be relocated at run-time. This means that the
global data harvesting won't bring any benefit. On the other hand, going
through dynamic symbols can be hugely beneficial. Same argument for
`*_RELATIVE` relocations.
The `merge-dynamic.py` script has been improved to find the appropriate
spot to put the rewritten program/section and headers and the dynamic
sections (the kernel is peeky on them).
Finally the `setRegister` function has been introduced in the module
produced by `revamb`. This function allows to keep CSVs static and, at
the same time, it allow `support.c` to set them. This is particularly
useful when we want to call the `root` function with specific values in
the registers (e.g., during for fuzzing purposes) or, as it's the case
for PIE, to synchronize the value of the FS register, which is
initialized by the dynamic loader, before execution gets to the `main`
function in `support.c`.
The reaching definition analysis now considers loads and stores from and
to absolute addresses.
This improves the identification of target of indirect jumps in x86-64
PIC code.
This function computes the CSV that may be accessed from a call in root.
Before this commit, it assumed that all the load and store had to be
aligned with the underlying variables. This happens to be too strict an
assumption, and it is relaxed in this commit.
When a load or a store is not aligned with the underlying type, this
commit introduces code to mark all the spanned CSV as accessed.
This is information is then attached to the call site in root as
metadata, as it already happened before.
This commit improves the capabilities of the `CPUStateAccessAnalysis`.
In particular, it is capable of handling GEPs that access arrays at
unknown offsets. This is a common operation in QEMU helpers, because it
is often used to access some elements in the array of GPRs using indexes
that are not known at translate-time. Handling this case allows us to
shrink the size of the generated translated code, because we only
generate accesses at valid offsets in the arrays instead of handling all
the possible wild accesses in the CPU State.
This commit adds a new analysis pass: `CPUStateAccessAnalysisPass`.
This pass currently performs 4 operations.
1. A preliminary analysis of the call graph, to select the functions
that are reachable from the root function through direct calls. All
the other performed operations are executed on this set of reachable
functions.
2. An interprocedural forward taint analysis, starting from the uses of
`env`, the global variable pointing to the QEMU struct continaint the
CPU. This analysis taints all the instructions that use the address
of `env`, until a load or a store is met. If a load or a store uses a
tainted Value as address it means that it is accessing a CSV at a
given offset (which at this point is still unknown).
3. An interprocedural offset analysis, which deduces the possible
offsets used by every tainted load/store to access the CSV. This
analysis initially works backwards, exploring all the Values that
contribute at the computation of the addresses used by tainted
load/stores. Once it finds all the sources, it starts propagating the
values forward, collecting the offsets computed along the way. It
does this until it reaches the tainted load/stores again. At that
point the analysis knows all the possible offsets used by each
tainted load/store to access the CPU state.
4. The results of the previous steps are used to do 3 things:
* marking all the indirect calls with tainted arguments as illegal;
this is necessary because those calls may access the CPU State in
unpredictable ways;
* attaching metadata to all the call sites to QEMU helpers in the root
function; these metadata provide information on which parts of the
CPU State may be accessed from that call site, which is a
potentially useful information for users of libtinycode that we also
plan to use in other parts of revamb;
* substituting loads, stores, and memcpys to and from the CPU state
with accesses to global variables; this operation effectively
replaces what was previously done by the CorrectCPUStateUsagePass,
which is now obsolete and was removed in this commit.
This function takes a `Instruction *` pointing to a `CallInst`, and
returns a `Function *` to the Callee if it's a direct call, skipping all
the bitcasts if any.
If the argument is not a `CallInst`, or it's not a direct call, it
returns `nullptr`.
Check that the calling `unindent()` never reduces the indentation
"below" zero, causing `IndentLevel` to wrap around to high numbers.
If it drops below zero it's a bug anyway, so assert!
This commit does 6 things on `getTypeAtOffset()`:
1. it changes the second argument from `StructType *TheStruct` to a more
generic argument `Type *VarType`, making it capable of working on any
type;
2. it removes recursion, substituting it with a while loop;
3. it removes the now useless `Depth` argument;
4. it changes the return type to `std::pair<IntegerType *, unsigned>`,
because this was the assumption that all the callers did anyways;
5. it guards all the unexpected types with an assertion;
6. it purposely avoids to guard pointer types with assertions, as a
workaround for a specific situation documented in detail in the new
comments.
The function `getTypeSizeInBits()` was wrongly used in many places when
reasoning about memory allocation, memory accesses, and memory offsets,
The result was often divided by 8 (possibly losing spare bits) or even
multiplied by 8, which makes no sense.
These uses were error prone, even if they didn't cause problems yet.
The `getTypeAllocSize()` is better suited for these uses, because it
returns the number of bytes necessary to allocate an object of the given
Type.
This class introduces the `RunningStatistics` class, which allows to
compute the mean and standard deviation of a set of numbers. These
values are computed incrementally and can be associated to a name. The
values computed by `RunningStatistics` can be dumped upon regular
program termination, `SIGABRT` and `SIGINT`. In practice they are
printed at the end of the program execution, even in case of asserts and
`Ctrl + C`. Moreover, `SIGUSR1` is used to trigger printing the
statistics without crashing the program.
`Logger` is the new infrastructrure for debug messages. They are
supposed to be used by just creating a global variable in a translation
unit and then writing there directly as if they were a stream.
Key features:
* Self-registration: a `Logger` by default register itself automatically
in a register. This means that at run-time, unlike with `DBG`, we have
a list of all the possible `Logger`s. One benefit of this approach is
being able to list all the available `Logger`s in `--help`.
* Indentation: `Logger`s can be indented. In particular the
`LoggerIndent` helper class can automatically indent a certain
`Logger` during the lifetime of its instance.
* Atomic messages: a message is no longer simply delimited by a "\n": to
mark the end of a message, an instance of `LogTerminator` has to be
streamed to the `Logger`. This can also be associated directly to the
lifetime of an object by using the `LogOnReturn` class.
* Custom class formatting: it is possible to decide how a certain object
should be formatted when sent to a `Logger` by implementing the
`writeToLog` function. For example, `llvm::Value`-derived objects are
passed through the `getName` function. If no custom formatter is
specified, the `operator<<` is employed.
Sometimes, dealing with move-semantics in C++ can be challenging. To
mitigate this problem, this commit introduces `ClassSentinel`, a simple
class that can be added as a member of a class and that will allow the
user to easily monitor if an instance of a class is used after being
moved in an unwanted way. To do so, simply call the
`ClassSentinel::check` method on the class member.
`ClassSentinel` performs similar (but less reliable) checks on usage of
destroyed objects.
`ClassSentinel` can also (optionally, though the `SENTINEL_STACKTRACES`
macro) collect stack traces of the points where the object was moved or
destroyed.
This commit also includes some basic testing for the class.
`SmallMap` is a `std::map`-like data structure whose storage is inline
if the number of elements is small, pretty much as `llvm::SmallVector`
and friends. In case the `SmallMap` is in its small form, the search
cost is linear in the number of elements.
This commit also introduces `Iteratall`, an iterator class that can wrap
iterators of different types but with the same `::value_type` and
`::reference` types. This is used to abstract away the fact that
`SmallMap` iterators can either be iterators over a `std::array` or
`std::map`.
Note that if you want to iterate on the elements, and you want to be
sure that they are ordered (as happens with `std::map`), you have to
call the `sort` method before, which might or might not triggered a
sort. Note also that the internal inline storage uses a `std::array`,
which means that the `K` and `V` must be default constructible.
SET was predisposed to be able to track all the addresses from which a
load has been performed, but this has never been fully implemented. This
commit does that.
It is important to know that a load has been performed from a certain
address since, most likely, this means that part of the program is not
code. In fact, this information is used to tag jump targets, so that
other analysis can take into account this fact. Note however that, at
the current stage, the jump target itself is preserved (and, therefore,
translated).
`reinterpret_cast`s can lead to undefined behaviors, since the compiler
can assume that each object will be accessed exclusively through
pointers of a single type.
Enabling strict aliasing and its warnings, even in debug builds, allows
us to catch this kind of problems earlier.
If CMake detects `libhello` correctly and it is present in the system
paths, it passes it to the linker as `-lhello`. However, sometimes, in
particular when linking Boost, the compiler detects a different version.
Forcing the CMake policy `CMP0060` to the new approach solves the
problem, since CMake now always uses the full path of the required
library, even if it's already available in the system paths.
`skipCasts` used to skip over `CastInst`, now it also skips `BitCast`
`ConstantExpr`s.
Additionally, `isCallTo` now uses `skipCasts` and a `isCallToHelper`
function has been introduced to check if an instruction is a call to a
helper function.
This commit introduces support for dynamic programs. The current
implementation translate the main binary and uses native libraries. This
works only if the target architecture is the same as the source
one. Currently we only handle x86-64.
* The `ExternalJumpsHandler` class has been introduced. It basically
takes care of extending the dispatcher handling the case in which the
program counter is an address outside the range of executable
addresses of the input program. In this case, a `setjmp` is perfomed,
the CPU state is serialized to physical registers and jump to the
value of the program counter is performed.
Once the target code will try to return to the translated program, a
segmentation fault will be triggered, a `longjmp` is performed and the
CPU state is deserialized so that the execution can resume (from the
dispatcher).
* `early-linked.c` has been introduced. Its purposes is to provide
declarations of variables and functions defined in `support.c`. In the
past, we had to manually create these definitions, a cumbersome and
error prone we now avoid by letting `clang` compile `early-linked.c`
and then linking it in.
* The old `support.h` is now known as `commonconstants.h`. `support.h`
now contains declarations that have to be consumed by
`early-linked.c`.
* Each architecture now provides additional information:
1. Which registers are part of the ABI and have to be preserved. If
necessary the QEMU name can be provided. For each register it's
also possible to provide their position within the `mcontext_t`
structure, provided by the signal handler.
2. Three assembly snippets, one to write a register, one to read it
and one perform an indirect jump.
Some of this information is also exposed in the output module as
metadata.
* `support.c` now installs a SIGSEGV signal handler. Since pages that
were originally executable are no longer executable, jumping there
(typically, from a library) will trigger a SIGSEGV that we will
handle. This allows us to properly deserialize the CPU state and
resume execution of the translate code.
* Now also a dynamic version of each test program is translated and
tested.
* The `merge-dynamic.py` script has been introduced: it takes case of
rewriting the translated binary so to tell the linker to performe both
the relocations of the translate program and the relocations of the
original program. It does so by rewriting a large portion of the
sections employed by the dynamic linker such as `.dynamic`, `.dynsym`
and so on.
* The `compile-time-constants.py` script has been introduced: it a
user-specified compiler on a source file producing an object
file. This object file is inspected and the value of global read-only
variables is produced in a CSV.
This ensure floating point memory accesses work properly on x86. In the
past this was working only in certain cases depending on the number of
entries in the auxiliary vector, arguments and environment variables.
This commit makes `revamb` produce a new file `.ll.need.csv` containing
a list of all the dynamic libraries required by the input program. This
will be transformed by the 'csv-to-ld-options` (was:
`li-csv-to-ld-options`) into the appropriate linking options.
The `argparse` library treats boolean arguments as
integers. Specifically, each time a boolean argument is meet the
associated variable is incremented. This led to weird behaviors having
`2` being converted to `false`. Using `int` as a type solves this issue.
This commit introduces the Function Isolation Pass. We use the
information provided by the Function Boundaries Detection Pass to
organize the code that `revamb` places inside the `root` function in
different LLVM functions. To do this we obviously need to introduce some
changes and tricks to handle the execution of the translated program.
The main idea is to have two different realms (one where the isolated
functions live, one in which we have basically the old root function).
We start the execution from the realm of the *non isolated* functions,
and we transfer, as soon as possible, the execution to the *isolated
functions* realm. We then have a fallback mechanism to restore the
execution in the right place in the *non isolated* functions realm, and
so on.
The largest change, besides the re-organization of the code in different
functions, is the use of the exception handling mechanism provided by
the LLVM framework in order to be able to manage the switch between the
two realms.
We also introduce the `support.h` header file, which contains a couple
of definitions used by `support.c` and that need to be shared with some
of the components involved in the translation process. We have defined
some helper functions, directly in C, that we use both for handling the
exception mechanism and for giving extra debug informations when an
exception is raised.
The `revamb-dump` utility now supports the `-i` option to specify the
path were to save the new LLVM module.
The `translate` utility now supports the `-i` option that produces a
binary in which the function isolation has been applied.
We also introduced some tests that apply the function isolation pass to
the `Runtime/` tests already present. In this way we can verify that the
translation and the following function isolation preserve the behavior
of the program.
When serializing the new LLVM module we regenerate the metadata used for
debug purposes, and for doing this, since we not longer have only the
`root` function, we have changed some details in the `DebugHelper` class
in order to be able to emit the metadata for all the functions of our
interest in a single shot.
Solved a bug that always allocated an `i32` for representing the return
address PC in the `function_call` helper (third parameter).
Now we allocate an `i64` or an `i32` depending on the input architecture
of the translated binary.
The header of an ELF file contains two bytes representing the ISA.
These two bytes are offset 0x12 in the file, which is offset 18, not
offset 17 like it was in the script.
`GenericFunctor` is substituted with the `std::integral_constant`
template. This also allows us to remove code that requires C++14.
It also removes the now useless cmake tests on the compiler flag
`-Wno-error=noexcept-type` that was introduced to disable fatal warnings
on the type `GenericFunctor`.
Now that this type has been removed the check is not necessary anymore,
because the `std::integral_constant` template used now does not cause
the warning.
So we can go back to enabling the fatal warnings.
Changed the names of the global variables (removed the leading `.`)
representing the segments of the binary, in order to prevent errors with
duplicated names when recompiling a binary with `llc` in debug mode.
We used to check if the value of `/proc/sys/vm/mmap_min_addr` is at
least as high as the minimum segment of the input binary. If this is not
the case the linked program will segfault at run-time without much
explanation.
However, in truth, we need to be able to map also the page before the
lowest page the original binary mapped. The main reason for this is to
have space for the (outer) ELF header.
It turns out that on many distros the default minimum value is
`0x10000`, which happens to be exactly the same address at which ARM
binaries mmap their lowest page. This lead to no warning, but a segfault
at run-time.
The AWK script now checks for the correct value, and also suggests the
correct value.
In the future, we might want to create a new segment for the outer ELF
header and position it elsewhere in the address space.
Alessandro Di Federico