* Add skeleton for PPC
* Copyright notices
* Fill in some details for the PPC arch
* Start building a (wrong) PPC runtime
* Begin populating state structure
* First pass for EIS state structure
* Map registers to Sleigh register names
* More fixes
* add optional param
* Create handle unsupported and invalid instruction isels
* Correct typo
* Get a basic `remill-lift` invocation running without failure
* Fix capitalisation
* Set vle context reg
* Fix SleighDecoder signatures
* Set VLE context register in the Sleigh engine in addition to our
internal context reg mapping
* Capitalize reg names
* Add the flag registers for XER and CR
* Rename bitflag structures in PPC state
* PPC Sleigh patches (#643)
* Modified sleigh patch script to generate patches for multiple .sinc files
* update README with new examples of sleigh patch script invocation
* add ppc register definition
* add ppc sleigh patches
* fix issue with remill_insn_size definition
* regenerate sleigh patches for PPC
* update CMakeLists.txt to include PPC patches
* Add TEA signal as a register in the PPC state
* Uppercase the stack pointer register name
* Fix PPC instruction sizes
* initial PPC tests
* remove duplicate tests
* fix tests for e_stmvgprw/e_ldmvgprw
* add tests for loading/storing from special registers
* add tests with internal conditionals in pcode
* fix for pc reg and addr width not being the same... I suspect this issue is going to come up elsewhere
* add heuristic for flow from normal intrainstruction flow
* rework tests to allow testing for different sized registers
* add tests for overflow and record add
* fix bug with log printout
* add intrafunction control flow lifting
* handle edge case where there is no pcode op at the zero index
* Fix another inconsistency with mismatching address and PC reg size
* Allocate unique ptrs in the entry block
* Fix `INT_LEFT` and `INT_RIGHT` impl where shift exceeds bit width
* fix supiece lift?
* Add PPC emulate instruction to hyper call
* fix for pc reg and addr width not being the same... I suspect this issue is going to come up elsewhere
* add heuristic for flow from normal intrainstruction flow
* add intrafunction control flow lifting
* handle edge case where there is no pcode op at the zero index
* Fix another inconsistency with mismatching address and PC reg size
* fix supiece lift?
* Allocate unique ptrs in the entry block
* Fix `INT_LEFT` and `INT_RIGHT` impl where shift exceeds bit width
* fix int2float semantics
should use appropriate sized float based on the output size
* add tests for lifting int2float
* fix INT_{LEFT,RIGHT} semantics
should be `ICmpSGE` instead of `ICmpSGT`
* add cr0-7 registers
* fix formatting
* fix conditional branch test
* add test for compare
* re-enable rotate left word immediate and mask test
* genericize TestSpecOutput
* explicit instruction data size
* add test for syscall/callother (disabled)
* add tests for store/load word
* add test to convert from float to int
* specify intrinsic arg type, fixes null deref
* Add PPC emulate instruction to hyper call
* add headers + formatting
* remove old comment
* Map CRALL register
* Add basic LLVM data layout that specifies 32-bit addresses
* Remove unused variables
* convert auto* to auto when possible
* RegisterPrecondition -> RegisterCondition
* fix variable name
* convert any to variant
* use std::move
* bump to c++20, use concepts
* set arch in constructor since class isn't generic anyways
* formatting
* make type aliases
* bump cxx-common
* add comment
* clang format
* throw exception if register not found
* use const ref
* use shorthand for lambda capture values
* Add more detail to data layout to include proper stack alignment
* Compare to the correct size for SUBPIECE impl
* Add Sleigh message to error
* throw exception in else case
* throw runtime error if register value has incorrect type
* use reference instead of value
* get rid of unnecessary type alias
* formatting
* Propagate VLE context reg value into Sleigh
* Remove unnecessary whitespace
* Remove stale TODO and NOTE comments
* add additional parameter to test runner to specify decoding context
* drop llvm 14, bump macos version
* bump cxx-common, fix ci.yml mac build
* add test for unconditional relative negative branch
* add missing space to pcode debug log
* fix bug due to unordered_map, iteration order matters
* add error log in case we aren't able to adjust PC value
* use helper for getting register reference
* Revert "add optional param"
This reverts commit 51ed49f8cf.
* Remove remaining LLVM 14 compatibility code and configuration
* Add padding between CR and XER flags
* Use `enum class`
* Remove void cast
* Remove unnecessary variable
* Use initialiser lists where appropriate
* Remove redundant `else`
* Prefer `CHECK` over `assert`
* Polish PowerPC function initialisation with lambda
* zero out xer_so to fix tests
* log error when we see claim_eq with no usages
* Collapse namespace blocks
* Remove unnecessary `this->`
* Use `auto` where appropriate
* Remove unnecessary `else`
* Use `emplace` over `insert` for `std::map`
Co-authored-by: lkorenc <lukas.korencik@trailofbits.com>
* Use `constexpr` for VLE reg name
* Use module verification util
* Add `VerifyFunction` util and use where applicable
* Extract lambda to improve readability of flow categorisation
* Use type alias for context values
* Introduce type alias for block exit
* Create type alias for optional branch taken
* Refactor `PcodeCFGBuilder`
* Use lambda to avoid conditional mutation
* Extract duplicated bit-shift code generation into helper
* Simplify flow with ternary
* Add `GetBlock` helper
* Rename variables
* Move statement for clarity
* Create helpers for working with Sleigh context register values
* Convert loop to `std::copy`
* Add a comment explaining the use of set to de-duplicate and sort
* Refactor `IntraProcTransferCollector`
* Expose static method to easily use `IntraProcTransferCollector`
* Rename PPC related variables to include address width
* add docs to intrainstructionindex
* remove llvm 14 ifdefs
* don't log error if no claim_eqs were used
* update comments
* Cleanup exit visitors
---------
Co-authored-by: 2over12 <ian.smith@trailofbits.com>
Co-authored-by: William Tan <1284324+Ninja3047@users.noreply.github.com>
Co-authored-by: lkorenc <lukas.korencik@trailofbits.com>
Remill 
Remill is a static binary translator that translates machine code instructions into LLVM bitcode. It translates AArch64 (64-bit ARMv8), SPARC32 (SPARCv8), SPARC64 (SPARCv9), x86 and amd64 machine code (including AVX and AVX512) into LLVM bitcode. AArch32 (32-bit ARMv8 / ARMv7) support is underway.
Remill focuses on accurately lifting instructions. It is meant to be used as a library for other tools, e.g. McSema.
Build Status
Additional Documentation
- How to contribute
- How to implement the semantics of an instruction
- How instructions are lifted
- The design and architecture of Remill
Getting Help
If you are experiencing undocumented problems with Remill then ask for help in the #binary-lifting channel of the Empire Hacking Slack.
Supported Platforms
Remill is supported on Linux platforms and has been tested on Ubuntu 20.04. Remill also works on macOS, and has experimental support for Windows.
Remill's Linux version can also be built via Docker for quicker testing.
Dependencies
Most of Remill's dependencies can be provided by the cxx-common repository. Trail of Bits hosts downloadable, pre-built versions of cxx-common, which makes it substantially easier to get up and running with Remill. Nonetheless, the following table represents most of Remill's dependencies.
| Name | Version |
|---|---|
| Git | Latest |
| CMake | 3.14+ |
| Google Flags | Latest |
| Google Log | Latest |
| Google Test | Latest |
| LLVM | 15 |
| Clang | 15 |
| Intel XED | Latest |
| Python | 2.7 |
| Unzip | Latest |
| ccache | Latest |
Getting and Building the Code
Docker Build
Remill now comes with a Dockerfile for easier testing. This Dockerfile references the cxx-common container to have all pre-requisite libraries available.
The Dockerfile allows for quick builds of multiple supported LLVM, architecture, and Linux configurations.
Quickstart (builds Remill against LLVM 15 on Ubuntu 20.04 for AMD64):
Clone Remill:
#Clone the repository.
git clone https://github.com/lifting-bits/remill.git
cd remill
Build Remill Docker container:
# do the build
docker build . -t remill \
-f Dockerfile \
--build-arg UBUNTU_VERSION=20.04 \
--build-arg ARCH=amd64 \
--build-arg LLVM_VERSION=15
Ensure remill works:
# Decode some AMD64 instructions to LLVM
docker run --rm -it remill \
--arch amd64 --ir_out /dev/stdout --bytes c704ba01000000
# Decode some AArch64 instructions to LLVM
docker run --rm -it remill \
--arch aarch64 --address 0x400544 --ir_out /dev/stdout \
--bytes FD7BBFA90000009000601891FD030091B7FFFF97E0031F2AFD7BC1A8C0035FD6
On Linux
First, update aptitude and get install the baseline dependencies.
sudo dpkg --add-architecture i386
sudo apt-get update
sudo apt-get upgrade
sudo apt-get install \
git \
python3 \
wget \
curl \
build-essential \
lsb-release \
ccache \
libc6-dev:i386 \
'libstdc++-*-dev:i386' \
g++-multilib
Next, clone the repository. This will clone the code into the remill directory.
git clone https://github.com/lifting-bits/remill.git
Next, we build Remill. This script will create another directory, remill-build,
in the current working directory. All remaining dependencies needed
by Remill will be built in the remill-build directory.
./remill/scripts/build.sh
Next, we can install Remill. Remill itself is a library, and so there is no real way to try it. However, you can head on over to the McSema repository, which uses Remill for lifting instructions.
cd ./remill-build
sudo make install
We can also build and run Remill's test suite.
cd ./remill-build
make test_dependencies
make test
Full Source Builds
Sometimes, you want to build everything from source, including the cxx-common libraries remill depends on. To build against a custom cxx-common location, you can use the following cmake invocation:
mkdir build
cd build
cmake \
-DCMAKE_INSTALL_PREFIX="<path where remill will install>" \
-DCMAKE_TOOLCHAIN_FILE="<path to cxx-common directory>/vcpkg/scripts/buildsystems/vcpkg.cmake" \
-G Ninja \
..
cmake --build .
cmake --build . --target install
The output may produce some CMake warnings about policy CMP0003. These warnings are safe to ignore.
Common Build Issues
If you see errors similar to the following:
fatal error: 'bits/c++config.h' file not found
Then you need to install 32-bit libstdc++ headers and libraries. On a Debian/Ubuntu based distribution, You would want to do something like this:
sudo dpkg --add-architecture i386
sudo apt-get update
sudo apt-get install libc6-dev:i386 libstdc++-10-dev:i386 g++-multilib
This error happens because the SPARC32 runtime semantics (the bitcode library which lives in <install directory>/share/remill/<version>/semantics/sparc32.bc) are built as 32-bit code, but 32-bit development libraries are not installed by default.
A similar situation occurs when building remill on arm64 Linux. In that case, you want to follow a similar workflow, except the architecture used in dpkg and apt-get commands would be armhf instead of i386.
Another alternative is to disable SPARC32 runtime semantics. To do that, use the -DREMILL_BUILD_SPARC32_RUNTIME=False option when invoking cmake.
