admin/revng-revng
1
0
Fork 0
mirror of https://github.com/revng/revng synced 2026-06-21 14:07:57 +00:00
Code Issues Packages Projects Releases Wiki Activity

Initial import of documentation

Browse Source
This commit is contained in:
Alessandro Di Federico
2017-01-11 16:02:19 +01:00
parent 72337caf07
commit 5154cb57f2
9 changed files with 1188 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.rst
+52
View File
@@ -0,0 +1,52 @@
*******
Purpose
*******
`revamb` is a static binary translator. Given a input ELF binary for one of the
supported architectures (currently MIPS, ARM and x86-64) it will analyze it and
emit an equivalent LLVM IR. To do so, `revamb` employs the QEMU intermediate
representation (a series of TCG instructions) and then translates them to LLVM
IR.
************
How to build
************
`revamb` employs CMake as a build system. The build system will try to
automatically detect the QEMU installation and the GCC toolchains require to
build the test binaries.
If everything is in standard locations, you can just run::
mkdir build/
cd build/
cmake ..
make -j$(nproc)
make install
For further build options and more advanced configurations see
docs/BuildSystem.rst (TODO: reference).
To run the test suite simply run::
make test
***********
Example run
***********
The simplest possible example consists in the following::
cd build
cat > hello.c <<EOF
#include <stdio.h>
int main(int argc, char *argv[]) {
printf("Hello, world!\n");
}
EOF
armv7a-hardfloat-linux-uclibceabi-gcc -static hello.c -o hello.arm
./translate hello.arm
# ...
./hello.arm.translated
Hello, world!
docs/BuildSystem.rst
+83
View File
@@ -0,0 +1,83 @@
***********************************
Identifying the required components
***********************************
`revamb` requires three main components: LLVM, QEMU and one or more GCC
toolchains.
LLVM
====
LLVM should be automatically detected if it's a standard location. If you want
to use a custom build in a non-standard location use the ``LLVM_DIR`` option,
making it point to the directory containing ``LLVMConfig.cmake`` in the install
directory (typically ``$INSTALL_PATH/share/llvm/cmake``):
.. code-block:: sh
cmake -DLLVM_DIR=/home/me/llvm-install/share/llvm/cmake ../
QEMU
====
By default the build system will look for the system QEMU installation (i.e., in
``/usr/``). If this is not what you want you can customize the QEMU install path
using the ``QEMU_INSTALL_PATH`` variable:
.. code-block:: sh
cmake -DQEMU_INSTALL_PATH="/home/me/qemu-install/" ../
GCC
===
The `revamb` build system will try to automatically detect toolchains to compile
code for the supported architectures. The toolchains are required to correctly
run the `revamb` test suite.
The autodetction mechanism looks in all the directories in the ``PATH``
environment variable for an executable matching the pattern ``$ARCH*-musl*-gcc``
or ``$ARCH*-uclibc*-gcc`` (glibc is currently unsupported).
It's possible to force the usage of a specific compiler for a certain
architecture using the CMake variable ``C_COMPILER_$ARCH``:
.. code-block:: sh
cmake -DC_COMPILER_mips="/home/me/my-mips-compiler" \
-DC_COMPILER_miparm="/home/me/my-arm-compiler" \
-DC_COMPILER_x86_64="/home/me/my-x86_64-compiler" \
../
The `revamb` build system also provides an option to specify additional flags to
employ when using the above mentioned commpilers for compilation and linking of
test binaries. This can be done using the variables ``TEST_CFLAGS_$ARCH`` (for
compile-time flags) and ``TEST_LINK_LIBRARIES_$ARCH`` (for linking).
This is particularly useful to force x86-64 binaries to compile with software
floating point, which can be achieved with GCC as follows:
.. code-block:: sh
cmake -DTEST_CFLAGS_x86_64="-msoft-float -mfpmath=387 -mlong-double-64" \
-DTEST_LINK_LIBRARIES_x86_64="-lc $LLVM_INSTALL_PATH/lib/linux/libclang_rt.builtins-x86_64.a" \
../
********************
Common CMake options
********************
In the following we provide some useful-to-know CMake variables, that are not
specific to `revamb`. They can be specified using the ``-D`` flag, e.g.:
.. code-block:: sh
cmake -DCMAKE_INSTALL_PREFIX=/tmp ../
:CMAKE_INSTALL_PREFIX: Specify the install path. This is the destination
directory where all the necessary files will be copied
upon ``make install``.
:CMAKE_BUILD_TYPE: Specify the type of build. The two most useful values are
``Debug``, for an unoptimized build with debugging
information, and ``RelWithdebinfo``, for an optimized build
with debugging information.
docs/FromIRToExecutable.rst
+123
View File
@@ -0,0 +1,123 @@
*****************************
From the IR to the executable
*****************************
This document aims to describe how to obtain a working executable from the IR
generated by `revamb`. This for documentation purposes only, standard users can
simply use the `translate` script, which will take care of everything,
From the IR to the object file
==============================
As an optional step, before proceeding in compiling the IR, it's possible to run
the optimization pipeline on the generated IR using the LLVM `opt` tool:
.. code-block:: sh
opt -O2 -S translated.ll -o translated.opt.ll
Depending on the size of the input program this might take some time.
The first step to obtain the final executable consists in compiling the IR. To
do this, we can use `llc`, the LLVM static compiler, which takes an LLVM IR file
and produces an object file:
.. code-block:: sh
llc -O0 translated.ll -filetype=obj -o translated.o
`llc` exposes the compiler backend. The backend can do another series of
optimizations:
.. code-block:: sh
llc -O2 translated.ll -filetype=obj -o translated.o -regalloc=fast
Note: as for `opt`, optimizations can take a considerable amount of time, in
particular if slow register allocation methods are used (i.e., make sure to use
`-regalloc=fast`). Non-trivial register allocation techniques on the `root`
function can be prohibitively costly (see also `GeneratedIRReference.rst`_).
Support functions
=================
The IR produced by `revamb` is mostly self-contained, since all the QEMU helper
functions are statically linked in the output module (unless ``--no-link`` was
specified, in which case the module won't be functional). Howver, besides the
QEMU helper functions, some additional support functions are required to obtain
a working program, in particular for initialization purposes.
`support.c` provides the default implementation for them. In the following we
provides a quick summary of the required functions as they are implemented in
`support.c`:
:main: The entry point of the program, maps in memory the stack and the heap. It
also performs the stack initialization, which is a very delicate
operation. As it can be seen in the `prepare_stack` function the
environment variables, the arguments and the auxiliary vectors are
initialized mimicing the stack initialization performed by the Linux
kernel. `main` also performs a call to `syscall_init` a function defined
in the generated module, taken from QEMU, which carries on some
syscall-related initializations. Finally, the entry point of the module
generated by `revamb` (the `root` function) is called.
:path: Function invoked by the QEMU Linux syscall emulation layer to *filter*
the path being opened by the translated program. For instance, it is
possible to catch attempts to open ``/proc/self/map`` and redirect it
somewhere else transparently.
:unknownPC: Function handling the situation in which the code at an address that
should be executed has not been translated (see
`GeneratedIRReference.rst`_). It is basically an error handling
function that is supposed to never return.
:newpc: If the ``--tracing`` option has been given, the calls to the `newpc`
functions emitted by `revamb` are turned into calls to an external
function, that the user can implement. This function can access the
status of the CPU and perform high performance tracing.
The remaining functions in `support.c` are of little interest and might be
removed in the future. Note also that `support.c` works automatically with all
the supported architectures, but needs to know which was the input
architecture. For this reason, defined a macro ``TARGET_arch`` (e.g.,
``TARGET_arm``) on the compilation command line.
Linking
=======
Once we have the object file generated by `revamb` we can compile `support.c`
and link them together. Be sure to link also `libm.so`, `libz.so` and
`librt.so`.
In addition to this, we also have to tell the linker to force the *segment
variables* (see `GeneratedIRReference.rst`_) at the appropriate location in
memory. In fact, each segment of the original binary should be loaded exactly at
the same address where it was originally supposed to be loaded. `revamb` emits
the required mapping automatically in a file with the same name as the output
file plus a ``.li.csv`` suffix. This file is a CSV file composed by three
columns:
:name: The name of the section in the generated module containing the data of
the considered segment.
:start: Start address where the segment originally was located and, therefore,
where it should be placed by the linker.
:end: Corresponding end address.
The `li-csv-to-ld-options` script converts this CSV file into parameters for the
linker to enforce the location of this sections.
As a result of this operation, the actual translated code might end up in an
unusual location, but the linker should be able to figure this out.
In conclusion, to link the final program:
.. code-block:: sh
gcc $(li-csv-to-ld-options translated.ll.li.csv) \
translated.o \
support.c \
-DTARGET_arm \
-lz -lm -lrt -Wno-pointer-to-int-cast -Wno-int-to-pointer-cast -g \
-o translated.elf
.. _`GeneratedIRReference.rst`: GeneratedIRReference.rst
docs/GeneratedIRReference.rst
+493
View File
@@ -0,0 +1,493 @@
***********************
revamb Output Reference
***********************
The main goal of revamb is to produce an LLVM module reproducing the behavior of
the input program. The module should be compilable and work out of the
box. However, such module contains also a rich set of additional information
recovered during the analysis process that the user can exploit to develop any
kind of analysis he wants.
This document details how to interpret the information present in the generated
module. Please refer to the `LLVM Language Reference Manual`_ for details on the
LLVM language itself.
The various sections of this document will present example to clarify the
presented concepts. All the examples originate from the translation of a simple
program compiled for x86-64:
.. code-block:: c
int myfunction(void) {
return 42;
}
int _start(void) {
int a = 42;
return a + myfunction();
}
The program has been compiled as follows:
.. code-block:: sh
x86_64-gentoo-linux-musl-gcc -static -nostdlib -O0 -fomit-frame-pointer example.c -o example
Producing the following assembly:
.. code-block:: objdump
00000000004000e8 <myfunction>:
4000e8: mov eax,0x2a
4000ed: ret
00000000004000ee <_start>:
4000ee: sub rsp,0x10
4000f2: mov DWORD PTR [rsp+0xc],0x2a
4000fa: call 4000e8 <myfunction>
4000ff: mov edx,eax
400101: mov eax,DWORD PTR [rsp+0xc]
400105: add eax,edx
400107: add rsp,0x10
40010b: ret
And it has been translated as follows:
.. code-block:: sh
./revamb --no-link --functions-boundaries --use-sections --debug-info ll example example.ll
Global variables
================
The CPU State Variables
-----------------------
The CPU State Variables (or CSV) are global variables that represent a part of
the CPU. They vary from architecture to architecture and they are created
on-demand, which means that not all modules will have all of them.
Some CSV variables have a name, in particular registers (e.g., `rsp`), some
others are instead identified by their position within the QEMU data structure
that contains them (e.g., ``state_0x123``). For example:
.. code-block:: llvm
@pc = global i64 0
@rsp = global i64 0
@rax = global i64 0
@rdx = global i64 0
@cc_src = global i64 0
@cc_dst = global i64 0
@cc_op = global i32 0
`@pc` represents the program counter (aka `rip`), `@rsp` the stack pointer
register, while `@rax` and `@rdx` are general purpose registers. The `@cc_*` are
helper variables used to compute the CPU flags.
CSVs are used by the generated code and by the helper functions. This is also
the reason why they cannot be promoted to local variables in the `root`
function`
Note that since they are global variables, the generated code interacts with
them using load and store operations, which might sound unusual for registers.
Segment variables
-----------------
The translated program expects the memory layout to be exactly as the one in the
original binary. This means that all the segments have to be loaded at the
original addresses. In the generated module, they are encoded as global
variables containing all the data of the segments. These variables have a name
similar to ``.o_permissions_address`` (e.g., ``.o_rx_0x10000``), where
*permissions* it's a string representing what type of accesses are allowed to
that segment (read, execute, write), and *address* is the starting address.
These variables are associated to special sections which will be assigned to the
appropriate virtual address at link-time.
In our example we have single segment, readable and writable:
.. code-block:: llvm
@.o_rx_0x400000 = constant [344 x i8] c"\7FELF\02\01\01\0...", section ".o_rx_0x400000", align 1
As you can see it is initalized with a copy of the original segment and its
assigned to the `.o_rx_0x400000` section.
Other global variables
----------------------
Apart from CSVs and segment variables, the output module will contain a number
of other global variables, mainly for loading purposes (see ``support.c``). In
the following we report the most relevant ones.
:.elfheaderhelper: a variable whose only purpose is to create the
`.elfheaderhelper` section, which is employed to force an
appropriate layout at link-time. It isn't of general
interest.
:e_phentsize: size of the ELF program header structure of the input binary.
:e_phnum: number of ELF program headers in the input binary.
:phdr_address: virtual address where the ELF program headers are loaded.
For more information on the ELF program headers, see ``man elf``. In the
example program we have three program headers of 56 bytes, loaded at
`0x400040`:
.. code-block:: llvm
@.elfheaderhelper = constant i8 0, section ".elfheaderhelper", align 1
@e_phentsize = constant i64 56
@e_phnum = constant i64 3
@phdr_address = constant i64 4194368
Input architecture description
==============================
The generated module also contains a *named metadata node*:
`revamb.input.architecture`. Currently it's composed by a metadata tuple with
two values:
:u32 DelaySlotSize: the size, in number of instructions of the delay slot of the
input architecture.
:string PCRegName: the name of the CSV representing the program counter.
Here's how this information appears in our example:
.. code-block:: llvm
!revamb.input.architecture = !{!0}
!0 = !{i32 0, !"pc"}
There is no delay slot on x86-64 and the CSV representing the program counter is
`@pc`.
The `root` function
===================
This section describes how the function collecting all the translated code is
organized. This fuction is knonw as the `root` function:
.. code-block:: llvm
define void @root() {
; ...
}
The dispatcher
--------------
The first set of basic blocks are related to the dispatcher. Every time we have
an indirect branch that we cannot fully handle we jump to the *dispatcher*,
which basically maps (with a huge ``switch`` statement) the starting address of
each basic block A in the input program to the first basic block containing the
code generated due to A.
:``dispatcher.entry``: the body of the dispatcher. Containes the ``switch``
statement. If the requested address has not been
translated, execution is diverted to
``dispatcher.default``.
:``dispatcher.default``: calls the `unknownPC` function, whose definition is
left to the user.
:``anypc``: handles the situation in which we were not able to fully enumerate
all the possible jump targets of an indirect jump. Typically will
just jump to ``dispatcher.entry``.
:``unexpectedpc``: handles the situation in which we though we were able to
enumerate all the possible jump targets, but an unexpected
program counter was requested. This indicates the presence of
a bug. It can either try to proceed with execution going to
``dispatcher.entry`` or simply abort.
The very first basic block is `entrypoint`. It's main purpose is to create all
the required local variables (``alloca`` instructions) and ensure that all the
basic blocks are reachable. In fact, it is terminated by a ``switch``
instruction which make all the previously mentioned basic blocks reachable. This
ensures that we can compute a proper dominator tree and no basic blocks are
collected as dead code.
Here's how it looks like in our example:
.. code-block:: llvm
define void @root() {
entrypoint:
%0 = alloca i64
%1 = bitcast i64* %0 to i8*
switch i8 0, label %bb._start [
i8 1, label %dispatcher.entry
i8 2, label %anypc
i8 3, label %unexpectedpc
]
dispatcher.entry: ; preds = %unexpectedpc, %anypc, %bb.myfunction, %bb._start.0x11, %entrypoint
%2 = load i64, i64* @pc
switch i64 %2, label %dispatcher.default [
i64 4194536, label %bb.myfunction
i64 4194542, label %bb._start
i64 4194559, label %bb._start.0x11
], !revamb.block.type !1
dispatcher.default: ; preds = %dispatcher.entry
call void @unknownPC()
unreachable
anypc: ; preds = %entrypoint
br label %dispatcher.entry, !revamb.block.type !2
unexpectedpc: ; preds = %entrypoint
br label %dispatcher.entry, !revamb.block.type !3
; ...
}
As you can see, we have three jump targets: `myfunction`, `_start` and
`_start+0x11` (the return address after the function call. In this specific
example we decide to divert execution to the dispatcher both in `anypc` and
`unexpectedpc`.
The translated basic blocks
---------------------------
The rest of the function is composed by basic blocks containing the translated
code. If symbols are available in the input binary, each basic block has name in
the form ``bb.closest_symbol.distance`` (e.g., ``bb.main.0x4`` means 4 bytes
after the symbol `main`). Otherwise the name is simply in the form
``bb.absolute_address`` (e.g., ``bb.0x400000``).
In our example we have three basic blocks:
.. code-block:: llvm
define void @root() {
; ...
bb._start: ; preds = %dispatcher.entry, %entrypoint
; ...
bb._start.0x11: ; preds = %dispatcher.entry
; ...
bb.myfunction: ; preds = %dispatcher.entry, %bb._start
; ...
}
Debug metadata
--------------
Each instruction we generate is associated with three types of metadata:
:dbg: LLVM debug metadata, used to be able to step through the generated LLVM IR
(or input assembly or tiny code).
:oi: *original instruction* metadata, contains a string-integer pair. The string
represents the disassembled input instruction that generated the current
instruction. The integer is the program counter associated to that
instruction.
:pi: *portable tiny code instruction* metadata, contains a string representing
the textual representation of the TCG instruction that generated the
current instruction.
Note: some optimizations passes might remove the metadata.
For debugging purposes, the generated LLVM IR contains comments with information
derived from these metadata.
As an example, let's see the first instruction of `myfunction`, ``mov
eax,0x2a``:
.. code-block:: llvm
define void @root() {
; ...
bb.myfunction: ; preds = %dispatcher.entry, %bb._start
; 0x00000000004000e8: mov eax,0x2a
; movi_i64 tmp0,$0x2a
; ext32u_i64 rax,tmp0
store i64 42, i64* @rax, !dbg !135, !oi !133, !pi !136
; ...
}
; ...
!4 = distinct !DISubprogram(name: "root", ...)
!133 = distinct !{!"0x00000000004000e8: mov eax,0x2a\0A", i64 4194536}
!134 = distinct !{!"movi_i64 tmp0,$0x2a\0A"}
!135 = !DILocation(line: 244, scope: !4)
!136 = distinct !{!"ext32u_i64 rax,tmp0,\0A"}
The `!dbg` metadata points to a `DILocation` object, which tells us that we're
at line 244 within the `root` function. This information will allow the debugger
(e.g., `gdb`) to perform step-by-step debugging. `!oi` points to a metadata node
containing the diassembled instruction that lead to generate this instruction
and its address (`4194536`). Finally, `!pi` points to the TCG instruction
leading to the creation of this instruction.
Above the instruction, we also have, for easier reading, the corresponding
original and TCG instructions.
Delimiting generated code
-------------------------
The code generated due to a certain input instruction is delimited by calls to a
marker function `newpc`. This function takes three arguments plus a set of
variadic arguments:
:u64 Address: the address of the instruction leading to the generation of the
code coming after the call of `newpc`.
:u64 InstructionSize: the size of the instruction at `Address`.
:u1 isJT: a boolean flag indicating whether the instruction at `Address` is a
jump target or not.
:u8 \*LocalVariables: a series of pointer to all the local variables used by
this instruction.
The call to `newpc` prevents the optimizer to reorder instructions across its
boundaries and perform other optimizations. This is useful during analysis and
for debugging purposes, but to achieve optimal performances all these function
calls should be removed.
Let's see how this works for the `bb.myfunction` basic block:
.. code-block:: llvm
bb.myfunction: ; preds = %dispatcher.entry, %bb._start
; 0x00000000004000e8: mov eax,0x2a
call void (i64, i64, i32, i8*, ...) @newpc(i64 4194536, i64 5, i32 1, i8* null), !oi !55, !pi !56
; ...
; 0x00000000004000ed: ret
call void (i64, i64, i32, i8*, ...) @newpc(i64 4194541, i64 1, i32 0, i8* null), !oi !58, !pi !59
; ...
As you cane see there are two calls to `newpc`, the first for the ``mov``
instruction at ``0x4000e8`` (5 bytes long) and the second one for the `ret`
instruction at ``0x4000ed`` (1 byte long). Note that the first instruction is a
jump target, in fact `newpc`'s third parameter is set to ``1``, unlike the
second call.
Function calls
--------------
revamb can detect function calls. The terminator of a basic block can be
considered a function call if it's preceeded by a call to a function called
`function_call`. This function take three parameters:
:BlockAddress Callee: reference to the callee basic block. The target of the
function call, most likely a function.
:BlockAddress Return: reference to the return basic block. It's the basic block
associated to the return address.
:u64 ReturnPC: the return address.
In our example we had a function call in the `_start` basic block:
.. code-block:: llvm
bb._start: ; preds = %dispatcher.entry, %entrypoint
; ...
; 0x00000000004000fa: call 0x4000e8
; ...
store i64 4194536, i64* @pc, !dbg !58, !oi !46, !pi !59
call void @function_call(i8* blockaddress(@root, %bb.myfunction), i8* blockaddress(@root, %bb._start.0x11), i32 4194559), !dbg !60
br label %bb.myfunction, !dbg !61, !func.entry !62, !func.member.of !63
As expected, before the branch instruction representing the function call, we
have a call to `@function_call`. The first argument is the callee basic block
(`bb.myfunction`), the second argument is the return basic block (`_start+0x11`)
and the third one is the return address (``0x4000ff``).
Function boundaries
-------------------
revamb can identify function boundaries. This information is also encoded in the
generated module by associating two types of metadata (`func_entry` and `func`)
to the terminator instruction of each basic block.
:func.entry: denotes that the current basic block is the entry block of a
certain function. The associated metadata tuple contains a single
string node representing the name assigned to the function.
:func.member.of: denotes that the current basic block is part of a set of
functions. The associated metadata tuple contains a set of
string nodes representing the name assigned to the
corresponding functions.
In our example we had three basic blocks: `_start`, `_start+0x11` and
`myfunction`. Let's see to what function they belong:
.. code-block:: llvm
define void @root() !dbg !4 {
; ...
bb._start: ; preds = %dispatcher.entry, %entrypoint
; ...
br label %bb.myfunction, !func.entry !62, !func.member.of !63
bb._start.0x11: ; preds = %dispatcher.entry
; ...
br label %dispatcher.entry, !func.member.of !63
bb.myfunction: ; preds = %dispatcher.entry, %bb._start
; ...
br label %dispatcher.entry, !func.entry !151, !func.member.of !152
; ...
}
; ...
!62 = !{!"bb._start"}
!63 = !{!62}
; ...
!151 = !{!"bb.myfunction"}
!152 = !{!151}
As it can be seen, `bb._start` and `bb._start.0x11` belong to a single function,
identified by `bb._start`. `bb._start` is also marked as the entry point of the
function. On the other hand, `bb.myfunction` also belongs to (and it's the entry
point of) a single function, with the same name.
Helper functions
================
Certain features of the input CPU would be to big to be expanded in TCG
instructions by QEMU (and therefore translate them in LLVM IR). For this reason,
call to *helper functions* are emitted. An example of an helper function is the
function handling a syscall or a floating point division. These functions can
take arguments and can read and modify freely all the CSV.
Helper functions are obtained from QEMU in the form of LLVM IR (e.g.,
``libtinycode-helpers-mips.ll``) and are statically linked by revamb before
emitting the module.
The presence of helper functions also import a quite large number of data
structures, which are not directly related to revamb's output.
Note that an helper function might be present multiple times with different
suffixes. This happens every time an helper function takes as an argument a
pointer to a CSV: for each different invocation we specialize that callee
function by fixing that argument. In this way, we can deterministically know
which parts of the CPU state is touched by an helper.
Currently, there is no complete documentation of all the helper functions. The
best way to understand which helper function does what, is to create a simple
assembly snippet using a specific feature (e.g., a performing a syscall) and
translate it using revamb.
.. _LLVM Language Reference Manual: http://llvm.org/docs/LangRef.html
docs/PythonExample.rst
+191
View File
@@ -0,0 +1,191 @@
***************************************************
Using `revamb` with Python: a simple instrumenation
***************************************************
In this document we'll guide the user through using `revamb`'s output from
Python. Amomng the many possibilities that arouse from the LLVM IR provided by
`revamb`, in this document we'll show how it's possible to perform a simple
instrumentation of an existing binary, by injecting some code in the generated
LLVM IR and recompiling it.
To manipulate LLVM IR from Python we'll use `llvmcpy`_, a lightweight wrapper
around the LLVM-C API for Python.
An Hello World for ARM
======================
Let's first of all create a simple program in C called `hello.c`:
.. code-block:: c
#include <stdio.h>
int main(int argc, char *argv[]) {
printf("Hello world!\n");
return 0;
}
We can compile `hello.c` for ARM and link it statically:
.. code-block:: sh
armv7a-hardfloat-linux-uclibceabi-gcc hello.c -o hello -static
Using the `translate`_ tool we can have `revamb` produce the
LLVM IR and recompile it for us. The output should be a working
``hello.translated`` program for x86-64 (our host architecture):
.. code-block:: sh
$ translate hello
$ ./hello.translated
Hello world!
All good!
Tracing all the syscall invocations
===================================
For this example, we'll write a simple Python script (``instrument.py``) which
takes in input the `revamb` generated LLVM IR, identifies all the syscalls and
instrument them injecting the code to print the number of syscall to be
performed.
In the ARM architecture, syscalls are expressed as calls to a function whose
name begins with `helper_exception_with_syndrome_`. After this prefix there is
an index identifying the specialization of the function, as explained in
`GeneratedIRReference.rst`_, but this is not interesting for us in this case.
Once we identified all the calls to the said functions, we will simply load the
value of the `r7` register, which holds the number of the syscall, and print it
to `stderr` using the `dprintf` function.
First of all we need to import `llvmcpy`_,obtain the default `LLVMContext`
object and load the input LLVM IR:
.. code-block:: python
from llvmcpy import llvm
context = llvm.get_global_context()
buffer = llvm.create_memory_buffer_with_contents_of_file(sys.argv[1])
module = context.parse_ir(buffer)
Now that we a reference to the module produced by `revamb` we can collect the
objects required to perform the `dprintf` call, i.e., the function itself, the
CSV representing the register `r7`, a constant integer representing `stderr` and
the format string for `dprintf`:
.. code-block:: python
r7 = module.get_named_global("r7")
dprintf = module.get_named_function("dprintf")
two = context.int32_type().const_int(2, True)
message_str = context.const_string("%d\n", 4, True)
message = module.add_global(message_str.type_of(), "message")
message.set_initializer(message_str)
message_ptr = message.const_bit_cast(context.int8_type().pointer(0))
Note that to build the format string we first have to create a new global
variable, then set its initializer with the constant string and finally cast it
to `int8 *` so that can be passed to `dprintf`, whose prototype is:
.. code-block:: c
int dprintf(int fd, const char *format, ...);
At this point we have to iterate over all the instructions of the function
containing the generated code (`root`):
.. code-block:: python
root_function = module.get_named_function("root")
for basic_block in root_function.iter_basic_blocks():
for instruction in basic_block.iter_instructions():
# ...
However, we are not interested in all instructions, but only in calls to
`helper_exception_with_syndrome_*` functions. Therefore, we check the opcode of
the instruction, and, if it's a call, we consider the last operand (which
represents the called function) and check it's name:
.. code-block:: python
if instruction.instruction_opcode == llvm.Call:
last_operand_index = instruction.get_num_operands() - 1
callee = instruction.get_operand(last_operand_index)
if not callee.name:
assert(callee.get_num_operands() == 1)
callee = callee.get_operand(0)
if callee.name.startswith("helper_exception_with_syndrome_"):
# ...
Note that the called function is often casted to a slightly different function
type, but we are not interested in this cast. The ``if not callee.name:`` block
handles this situation by moving to the first operand of the cast instruction.
Finally, we've found a location where we want to insert our instrumentation. To
do this, we create a *builder* object, position it right before the call
instruction, emit an instruction to load `r7`, prepare the other arguments and,
finally, emit the call to `dprintf`:
.. code-block:: python
builder = context.create_builder()
builder.position_builder_before(instruction)
load_r7 = builder.build_load(r7, "")
builder.build_call(dprintf, [two, message_ptr, load_r7], "")
That's all. The last thing left to do is to serialize the new IR to file:
.. code-block:: python
module.print_module_to_file(sys.argv[2])
Let's now run our script and recompile the code:
.. code-block:: sh
$ mv hello.ll hello.ll.original
$ python instrument.py hello.ll.original hello.ll
$ translate -s hello
$ ./hello.translated
45
45
983045
5
3
6
54
54
4
Hello world!
248
We can compare the result with a QEMU run of the original program:
.. code-block:: sh
$ qemu-arm -strace hello
7346 brk(NULL) = 0x00039000
7346 brk(0x000394b0) = 0x000394b0
7346 open("/dev/urandom",O_RDONLY) = 3
7346 read(3,0xf6ffde84,4) = 4
7346 close(3) = 0
7346 ioctl(0,21505,-151003688,0,221184,0) = 0
7346 ioctl(1,21505,-151003688,1,221184,0) = 0
7346 write(1,0x372a8,13)Hello world!
= 13
7346 exit_group(0)
The complete `instrument.py` script is available in `docs/instrument.py`_.
.. _llvmcpy: https://rev.ng/llvmcpy
.. _translate: TranslateUsage.rst
.. _`GeneratedIRReference.rst`: GeneratedIRReference.rst
.. _`docs/instrument.py`: instrument.py
docs/RevambDumpUsage.rst
+49
View File
@@ -0,0 +1,49 @@
***********
revamb-dump
***********
----------------------------------------
extract information from `revamb` output
----------------------------------------
:Author: Alessandro Di Federico <ale+revng@clearmind.me>
:Date: 2016-12-22
:Copyright: MIT
:Version: 0.1
:Manual section: 1
:Manual group: rev.ng
SYNOPSIS
========
revamb-dump [options] INFILE
DESCRIPTION
===========
`revamb-dump` is a simple tool to extract some high level information from the
IR produced by `revamb`.
OPTIONS
=======
Note that all the options specifying a path support the special value ``-``
which indicates ``stdout``. Note also that `revamb-dump` expresses the *name of
a basic block* as represented by `revamb` in the generated module (typically
``bb.0xaddress`` or ``bb.symbol.0xoffset``.
:``-c``, ``--cfg``: Path where the control-flow graph should be stored. The
output will be a CSV file with two columns: `source` and
`destination`. Both of them contain the name of a basic
block.
:``-n``, ``--noreturn``: Path where the list of the ``noreturn`` basic block
should be stored. The output will be a CSV file with a
single column `noreturn`, containing the name of the
``noreturn`` basic block.
:``-f``, ``--function-boundaries``: Path where the list of *function*<->*basic
block* pairs should be stored. The output
will be a CSV file with two column:
`function`, the name of the entry basic
block of the function, and `basicblock`, the
name of a basic block belonging to
`function`.
docs/RevambUsage.rst
+87
View File
@@ -0,0 +1,87 @@
******
revamb
******
---------------------------------------------------------
lift a binary from any architecture to compilable LLVM IR
---------------------------------------------------------
:Author: Alessandro Di Federico <ale+revng@clearmind.me>
:Date: 2016-12-22
:Copyright: MIT
:Version: 0.1
:Manual section: 1
:Manual group: rev.ng
SYNOPSIS
========
revamb [options] [--] INFILE OUTFILE
OPTIONS
=======
`revamb` options allow to customize its output. In the following the main ones
are described:
:``-e``, ``--entry``: Tell `revamb` to ignore the program entry point (typically
the `_start` function) and start to translated code at the
specified address (as *decimal* number). Note that this
option also disables the harvesting of the global data in
search for code pointers (see
`docs/GeneratedIRReference.rst`). This option is mainly
useful to test how `revamb` acts on a limited portion of
code without translating the whole program. Default: entry
point specified by the ELF header (``ElfN_Ehdr.e_entry``).
:``-i``, ``--linking-info``: Path where the CSV containing instructions for the
linker on how to position the segment variables
(see `docs/FromIRToExecutable.rst`). Default:
``OUTFILE.li.csv``.
:``-g``, ``--debug-info``: Type of debug information to associate to the
generated module, i.e. the type of *source code* to
use in step-by-step debugging of the generated
program (e.g., using `gdb`). Possible values are:
`none` for no debugging information, `asm` to dump
the disassembly of the input code, `ptc` to dump TCG
instructions and `ll` to dump the LLVM IR.
``--debug-path`` specifies the location of the
output. Default locations are ``OUTFILE.S`` for
`asm`, ``OUTFILE.ptc`` for `ptc` and ``OUTFILE``
itself for `ll`.
:``-s``, ``--debug-path``: Path where the *debug source* should be saved. See
``--debug-info`` for additional information and
default value.
:``-c``, ``--coverage-path``: Path where the list of the address ranges that
have been translated should be stored (in CSV
form). This option is deprecated in favor of using
`revamb-dump` and will be removed. Default:
``OUTFILE.coverage.csv``
:``-d``, ``--debug``: Enable verbose debugging info during execution. The
argument is a comma-separated list of keywords identifying
a type of debug information. This debug information are
for development usage and possible values can be
identified directly in the source code in the first
argument of the ``DBG(...)`` macro.
:``-O``, ``--no-osra``: Disable OSR Analysis. Useful for debugging purposes to
evaluate its effectiveness.
:``-L``, ``--no-link``: Do not link in QEMU helper functions. The call will
appear as calls to extern functions. This option makes
the resulting module non-functioncal but it's useful for
analysis-only purposes.
:``-t``, ``--tracing``: Function calls to `newpc` (which delimit the code
generated by an input instruction from another) target
an external function that the use can implement (see
`docs/FromIRToExecutable.rst`) instead of calling a
no-op function. This is useful to execute a certain
action, or inspect the CPU state at each instruction.
:``-S``, ``--use-sections``: If they are available, ELF sections are employed to
identify executable code. This options is useful to
evaluate `revamb` ignoring the code identification
issue.
:``-b``, ``--bb-summary``: Output path for the CSV containing statistics about
the translated basic blocks. This option is
deprecated in favor of using `revamb-dump` and will
be removed. Default: ``OUTFILE.bbsummary.csv``.
:``-f``, ``--function-boundaries``: Enable function boundaries detection. This
process currently can be quite expensive and
it's therefore disabled by default.
docs/TranslateUsage.rst
+42
View File
@@ -0,0 +1,42 @@
*********
translate
*********
----------------------------------------------------------------
translate a program from an architecture to another using revamb
----------------------------------------------------------------
:Author: Alessandro Di Federico <ale+revng@clearmind.me>
:Date: 2016-12-22
:Copyright: MIT
:Version: 0.1
:Manual section: 1
:Manual group: rev.ng
SYNOPSIS
========
revamb-dump [options] [--] INFILE [revamb options]
DESCRIPTION
===========
`translate` is a simple wrapper script which using `revamb` to lift an input
program to LLVM IR and then compiling it to x86-64.
In practice, `translate` first invokes `revamb` then, depending on the options,
some optimizations are performed using `llc` and or `opt`, and finally the
generated object file is linked against the require libraries and `support.c`.
Options after `INFILE` are forwarded as is to `revamb`.
OPTIONS
=======
:``-O0``: Disable optimizations both in the mid-end (`opt`) and the backend
(`llc`). This is default option.
:``-O1``: Enable backend optimizations (`llc`).
:``-O2``: Enable optimizations both in the mid-end (`opt`) and the backend
(`llc`).
:``-s``: Skip invoking `revamb`, assumes a file named `INFILE.ll` already
exists. This is useful for optimizing previously generated code.
docs/instrument.py
+68
View File
@@ -0,0 +1,68 @@
#!/usr/bin/env python
import sys
from llvmcpy import llvm
def main():
context = llvm.get_global_context()
# Load and parse the input file
buffer = llvm.create_memory_buffer_with_contents_of_file(sys.argv[1])
module = context.parse_ir(buffer)
# Identify the CSV (global variable) for the register containing the syscall
# number (r7 for ARM)
r7 = module.get_named_global("r7")
# Prepare requirements to build our call to dprintf
# 1) Identify the dprintf function (prototype)
dprintf = module.get_named_function("dprintf")
# 2) Create an integer constant for stderr
two = context.int32_type().const_int(2, True)
# 3) Create the message to print
message_str = context.const_string("%d\n", 4, True)
message = module.add_global(message_str.type_of(), "message")
message.set_initializer(message_str)
message_ptr = message.const_bit_cast(context.int8_type().pointer(0))
# Identify the function generated by revamb (root)
root_function = module.get_named_function("root")
# Loop over all the generated instructions and add instrumentation code
builder = context.create_builder()
for basic_block in root_function.iter_basic_blocks():
for instruction in basic_block.iter_instructions():
# Check if the current instruction is a call
if instruction.instruction_opcode == llvm.Call:
# The callee is the last operand
last_operand_index = instruction.get_num_operands() - 1
callee = instruction.get_operand(last_operand_index)
# If there's a bitcast, skip it
if not callee.name:
assert(callee.get_num_operands() == 1)
callee = callee.get_operand(0)
# Check if it's performing a syscall
if callee.name.startswith("helper_exception_with_syndrome_"):
# Set the builder's insert point before the call instruction
builder.position_builder_before(instruction)
# Load the value of r7 (which contains the syscall number)
load_r7 = builder.build_load(r7, "")
# Create the list of arguments for dprintf
arguments = [two, message_ptr, load_r7]
# Create the function call to dprintf:
# dprintf(2, "%d\n", r7)
builder.build_call(dprintf, arguments, "")
# Produce the instrumented LLVM IR to file
module.print_module_to_file(sys.argv[2])
if __name__ == "__main__":
main()