cea-sec-miasm/doc/jitter/jitter.ipynb

Miasm Emulation Engine

This short document provides an introduction to the APIs of the Miasm emulation engine. This emulation engine is commonly referred to as jitter throughout the project's code (as it is based on JiT methods).

The first step is to obtain an instance of the jitter. To do this, we will go through an intermediate object, Machine. It will allow us to instantiate a set of elements linked to a given architecture supported by Miasm (x86, arm, mips, ...).

In [1]:

from miasm.analysis.machine import Machine
Machine.available_machine()

Out [1]:

['arml',
 'armb',
 'armtl',
 'armtb',
 'sh4',
 'x86_16',
 'x86_32',
 'x86_64',
 'msp430',
 'mips32b',
 'mips32l',
 'aarch64l',
 'aarch64b',
 'ppc32b',
 'mepl',
 'mepb']

In [2]:

machine = Machine("x86_32")
print(machine.name)

x86_32

The Machine class in Miasm provides various interfaces:

• jitter: an emulation engine
• dis_engine: a disassembly engine
• lifter: a lifting engine, to lift assembly code to the corresponding Miasm internal representation (IR)
• lifter_model_call: as lifter, but function calls are abstracted
• mn: low level object to interact with an architecture (assembly, disassembly of only a few bytes, etc.)
• gdbserver: a GDB remote debugging server (to link with a jitter instance)

The IR related objects are already discussed in this notebook.

To obtain an emulator instance, we first need to:

1. instanciate a symbol table (LocationDB in Miasm, already explained in this notebook) ;

2. (optionnaly) choose a backed for the emulation (more specifically, to JiT the code):

   • "python": a backend entirely in Python, slow
   • "gcc": a backend based on the compiler GCC (Assembly \rightarrow IR Miasm \rightarrow C \rightarrow GCC), fast
   • "llvm": a backend based on the LLVM JiT engine (Assembly \rightarrow IR Miasm \rightarrow IR LLVM \rightarrow LLVM JiT), fast

Note: using a JiT backend as llvm or gcc greatly improve the performance over the python one. As a result, without any argument, instanciating an emulator will automatically try to look for and choose the best suited backend.

In [3]:

from miasm.core.locationdb import LocationDB

loc_db = LocationDB()
jitter = machine.jitter(loc_db)
print(jitter)

<miasm.arch.x86.jit.jitter_x86_32 object at 0x7f195fe07d60>

For now, our emulator is an empty box. It has:

• registers, reachable from the .cpu attribute. These are initialized to 0.
• a virtual memory, reachable from the .vm attribute. It starts empty.

CPU state

Let's manipulate a few registers:

In [12]:

# Read a register
print(jitter.cpu.EAX)

# Write a register
jitter.cpu.EAX = 1
print(jitter.cpu.EAX)

0
1

A few helpers are also available. For instance, one can get every registers using:

In [13]:

# GPReg : General Purpose registers
regs = jitter.cpu.get_gpreg()
print(regs)

{'RAX': 1, 'RBX': 0, 'RCX': 0, 'RDX': 0, 'RSI': 0, 'RDI': 0, 'RSP': 0, 'RBP': 0, 'R8': 0, 'R9': 0, 'R10': 0, 'R11': 0, 'R12': 0, 'R13': 0, 'R14': 0, 'R15': 0, 'RIP': 0, 'zf': 0, 'nf': 0, 'pf': 0, 'of': 0, 'cf': 0, 'af': 0, 'df': 0, 'ES': 0, 'CS': 0, 'SS': 0, 'DS': 0, 'FS': 0, 'GS': 0, 'MM0': 0, 'MM1': 0, 'MM2': 0, 'MM3': 0, 'MM4': 0, 'MM5': 0, 'MM6': 0, 'MM7': 0, 'XMM0': 0, 'XMM1': 0, 'XMM2': 0, 'XMM3': 0, 'XMM4': 0, 'XMM5': 0, 'XMM6': 0, 'XMM7': 0, 'XMM8': 0, 'XMM9': 0, 'XMM10': 0, 'XMM11': 0, 'XMM12': 0, 'XMM13': 0, 'XMM14': 0, 'XMM15': 0, 'tsc': 1234605616436508552}

Note: in Miasm, x86_32 implementation is merged with the x86_64 one. As a result, the register names are the x64 ones. Still, the computation is made accordingly to the 32bits implementation, and registers can be accessed either using the x64 name or the 32bits name (ie .EAX and .RAX).

Virtual memory

Initially, the virtual memory is empty. The repr output offers a summary of the availables pages:

In [14]:

jitter.vm

Out [14]:

Addr               Size               Access Comment

Memory pages can then be added. The memory address is arbitrary (no alignment requirement). The size is also arbitrary (byte precision).

Optionnaly, a comment associated to the page can be provided.

In [16]:

from miasm.jitter.csts import PAGE_READ, PAGE_WRITE, PAGE_EXEC
jitter.vm.add_memory_page(0x1000, PAGE_READ | PAGE_WRITE | PAGE_EXEC, b"\x00" * 0x1000, "test page")
jitter.vm.add_memory_page(0x112233, PAGE_READ | PAGE_WRITE | PAGE_EXEC, b"\x00" * 0x666, "no alignment, byte precision")
jitter.vm

Out [16]:

Addr               Size               Access Comment
0x1000             0x1000             RWX    test page
0x112233           0x666              RWX    no alignment, byte precision

The memory can now be accessed, read and write:

In [17]:

jitter.vm.get_mem(0x1000, 0x10)

Out [17]:

b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'

In [18]:

jitter.vm.set_mem(0x1000, b"toto")
jitter.vm.get_mem(0x1000, 0x10)

Out [18]:

b'toto\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'

Note: having a byte-precision memory has some advantages. For instance:

• when mapping a partially known structure, instead of defaulting unknown fields to 0, we can instead only map in memory the known field. As a result, we obtain a sparse memory layout and if the programm try to read an unknown field, the execution will stop right on the responsible instruction, helping the reversing work
• allocation can be made byte-wise. Thus, it could be easier to detect overflows and underflows

Emulation

We will now run the emulator. First, actual instructions are needed:

In [21]:

jitter.vm.set_mem(0x1000, bytes.fromhex("B844332211C3"))

We will also enable debug logging. By default, the logger will logs executed instruction and register values.

In [22]:

jitter.set_trace_log()

The emulation is initialized with jitter.init_run(address) and resumed with jitter.continue_run(). For convenience, jitter.run(address) is usually used, wrapping these two API.

In [27]:

jitter.vm.add_memory_page(0x88880000, PAGE_READ | PAGE_WRITE | PAGE_EXEC, b"\x00" * 0x1000, "stack")
jitter.vm.set_mem(0x88880000 + 0x1000 - 4, b"\xef\xbe\x37\x13")
jitter.cpu.ESP = 0x88880000 + 0x1000 - 4

jitter.run(0x1000)

# The execution ends with an error, which is expected.
# Indeed, we RET on 0x1337beef, which is not mapped in memory,
# hence the "WARNING: address 0x1337BEEF is not mapped in virtual memory"

[WARNING ]: [Errno cannot get mem ad] 0x1337beef
WARNING: address 0x1337BEEF is not mapped in virtual memory:
[WARNING ]: cannot disasm at 1337BEEF

00001000 MOV        EAX, 0x11223344
EAX 11223344 EBX 00000000 ECX 00000000 EDX 00000000 ESI 00000000 EDI 00000000 ESP 88880FFC EBP 00000000 EIP 00001005 zf 0 nf 0 of 0 cf 0
00001005 RET        
EAX 11223344 EBX 00000000 ECX 00000000 EDX 00000000 ESI 00000000 EDI 00000000 ESP 88881000 EBP 00000000 EIP 1337BEEF zf 0 nf 0 of 0 cf 0

Breakpoints

One can register breakpoints to be raised just before a given address will be executed.

As we control the CPU, the breakpoint implementation is invisible from the emulated environnement and we can use an arbitrary number of them.

In [31]:

# Breakpoints are actually callbacks which receive the emulation instance in argument
def hello_world(jitter):
    print("Hello, world!")
    print("EAX value is %d" % jitter.cpu.EAX)
    # Stop execution right here
    return False

jitter.add_breakpoint(0x1005, hello_world)

If a breakpoint callback returns True, the emulation continues. Otherwise, the emulation will stop and the value returned by the jitter.continue_run() call is the callback return value.

Note: if one need to add more arguments to the callback, there are many Pythonic way to do it. For instance, one can use lambda to capture an argument or object instance (ie. jitter.add_breakpoint(address, obj.callback) with a def callback(self, jitter) inside the obj definition).

The breakpoint mechanism is used to implement several features. For instance:

• it can be used to hook a function, by breakpointing on the function address, emulating a few side effects then returning to the return address. Indeed, modifying the programm counter (usually using jitter.pc = XXX) in a breakpoint will resume the execution on this new address
• in Miasm examples, a breakpoint on a fake address is often used to properly stop the emulation. The callback is usually named code_sentinelle and allow to get back control after the emulation of a code snippet

Exercices

For the interested reader, this section introduce a few exercice to practice the API.

Exercice 1

Starting from a new instance:

1. Add a memory page which will act as a stack page
2. Update the stack pointer accordingly
3. Manually push a fake return address
4. Map a RET instruction (0xC3) in another page
5. Run the emulation and ensure it finish on your fake address

Exercice 1 - Spoiler

A few helpers are actually availables to reduce the burden of these manual actions:

In [30]:

# Init a stack and set stack pointer accordingly (architecture agnostic way of writing)
jitter.init_stack()

# Push 0x1000 on the stack
jitter.push_uint32_t(0x1000)

# Pop 0x1000 from the stack
print(hex(jitter.pop_uint32_t()))

0x1000

Exercice 2

Same a Exercice 1, but instead of ending on an error, we will this time add a breakpoint on the fake return address to properly stop the execution.

Exercice 2 - Spoiler

The example jitter/x86_32.py is exactly doing this.