revng-revng/docs/FromIRToExecutable.rst

*****************************
From the IR to the executable
*****************************

This document aims to describe how to obtain a working executable from the IR
generated by `revamb`. This for documentation purposes only, standard users can
simply use the `translate` script, which will take care of everything. In the
following we will assume the output of `revamb` is an LLVM IR file named
`translated.ll`.

Support functions
=================

The IR produced by `revamb` is mostly self-contained, since all the QEMU helper
functions are statically linked in the output module (unless ``--no-link`` was
specified, in which case the module won't be functional). However, besides the
QEMU helper functions, some additional support functions are required to obtain
a working program, in particular for initialization purposes.

`support.c` provides the default implementation for them. In the following we
provides a quick summary of the required functions as they are implemented in
`support.c`:

:main: The entry point of the program, maps in memory the stack and the heap. It
       also performs the stack initialization, which is a very delicate
       operation. As it can be seen in the `prepare_stack` function the
       environment variables, the arguments and the auxiliary vectors are
       initialized mimicing the stack initialization performed by the Linux
       kernel. `main` also performs a call to `syscall_init` a function defined
       in the generated module, taken from QEMU, which carries on some
       syscall-related initializations. Finally, the entry point of the module
       generated by `revamb` (the `root` function) is called.

:path: Function invoked by the QEMU Linux syscall emulation layer to *filter*
       the path being opened by the translated program. For instance, it is
       possible to catch attempts to open ``/proc/self/map`` and redirect it
       somewhere else transparently.

:unknownPC: Function handling the situation in which the code at an address that
            should be executed has not been translated (see
            `GeneratedIRReference.rst`_). It is basically an error handling
            function that is supposed to never return.

:newpc: As seen in `GeneratedIRReference.rst`_, each input instruction is
        delimited by a call to a `newpc` function. This function is not just a
        placeholder, but the user can actually provide it. It is particularly
        useful to access the status of the CPU and perform tracing.

The remaining functions in `support.c` are of little interest and might be
removed in the future. Note also that `support.c` works automatically with all
the supported architectures, but needs to know which was the input
architecture. For this reason, defined a macro ``TARGET_arch`` (e.g.,
``TARGET_arm``) on the compilation command line.

The provided `support.c` offers two modes of operations: `normal` and
`trace`. The only difference between the two modes is that the latter activates
the program counter tracing support (which is implemented through `newpc`). This
means that while running the program a list of the execute program counters will
be dumped to the path specified by `REVAMB_TRACE_PATH`, if available. This is
optional at compile-time, since it introduces an overhead even if disabled at
run-time.

`revamb` distribution provide a pre-compiled version of both the flavors in the
form of LLVM IR: `support-x86_64-normal.ll` and `support-x86_64-trace.ll`. They
have to be linked into the module generated by `revamb`:

.. code-block:: sh

    llvm-link -S translated.ll support-x86_64-normal.ll -o translated.linked.ll

From the IR to the object file
==============================

As an optional step, before proceeding in compiling the IR, it's possible to run
the optimization pipeline on the generated IR using the LLVM `opt` tool:

.. code-block:: sh

    opt -O2 -S translated.linked.ll -o translated.opt.ll

Depending on the size of the input program this might take some time.

The first step to obtain the final executable consists in compiling the IR. To
do this, we can use `llc`, the LLVM static compiler, which takes an LLVM IR file
and produces an object file:

.. code-block:: sh

    llc -O0 translated.linked.ll -filetype=obj -o translated.o

`llc` exposes the compiler backend. The backend can do another series of
optimizations:

.. code-block:: sh

    llc -O2 translated.linked.ll -filetype=obj -o translated.o -regalloc=fast -disable-machine-licm

Note: as for `opt`, optimizations can take a considerable amount of time, in
particular if slow register allocation methods are used (i.e., make sure to use
`-regalloc=fast`). Non-trivial register allocation techniques on the `root`
function can be prohibitively costly (see also `GeneratedIRReference.rst`_).

Linking
=======

Once we have the object file we have to link it to link to `libm.so`, `libz.so`
and `librt.so`.

In addition to this, we also have to tell the linker to force the *segment
variables* (see `GeneratedIRReference.rst`_) at the appropriate location in
memory. In fact, each segment of the original binary should be loaded exactly at
the same address where it was originally supposed to be loaded. `revamb` emits
the required mapping automatically in a file with the same name as the output
file plus a ``.li.csv`` suffix. This file is a CSV file composed by three
columns:

:name: The name of the section in the generated module containing the data of
       the considered segment.
:start: Start address where the segment originally was located and, therefore,
        where it should be placed by the linker.
:end: Corresponding end address.

The `csv-to-ld-options` script converts this CSV file into parameters for the
linker to enforce the location of this sections.

As a result of this operation, the actual translated code might end up in an
unusual location, but the linker should be able to figure this out.

In conclusion, to link the final program:

.. code-block:: sh

    gcc $(csv-to-ld-options translated.ll.li.csv) \
        translated.o \
        -lz -lm -lrt \
        -o translated.elf

.. _`GeneratedIRReference.rst`: GeneratedIRReference.rst