Files
lifting-bits-remill/remill/BC/Util.cpp
T
Peter Goodman 08f5384a94 Issue 51 third party code (#60)
* Working on factoring out external dependencies

* Making progress

* CMake for remill-lift seems to work :-)

* Improving build process and dependency list. Using a blank ubuntu 14.04 vm for testing.

* Still fighting with various dependencies.

* Making more progress with CMake stuff! Now have the semantics get compiled with cmake, and support installing stuff. I have also attempted to auto-find the semantics files when --bc_in is not provided.

* Adding protobuf compilation back to the steps.

* Improving readme

* Improving readme

* Switch to using the  command.

* Updated travis. Updated README to include new build badge icons, to change where the ldconfig step goes, and to add a CMake notice for ubuntu trusty.

* Added script to generate tests. Added auto-yes in apt-add-repository commands.

* Minor fix

* Enable auto-finding of --bc_in based on --arch_in.

* Adding more test generation and compilation. Unfortunately compiling tests isn't working due to llvm::CloneFunctionInto not quite working anymore.

* llvm::CloneIntoFunction seems to work, which may resolve Issue #55. I'll need to double check with the official release version of LLVM; right now I'm using a custom debug build to try to catch issues. Added code to build and install gtest. Directly referenced which LLVM libraries must be linked. Added backup paths for deducing the semantics bitcode files based on the input arch. Made remill-lift report simple flag errors via stderr instead of using glog to abort the program. Fixed an issue where, when I added the various arch versions, xed was decoding avx-enabled instructions as 64-bit because the comparison condition deciding which decode state to use was too narrow.

* Tests generation and running scripts now work for me. Gotta test in fresh vms now.

* Addresses Issue #55. The problem was that the release version of clang 3.9 isn't guaranteed to preserve c++ variables name in the bitcode. I've added some stuff to re-introduce the variable names based on debug info.

* Build the test runner with rtti so that gtest doesn't complain on some versions of ubuntu.

* Fixing up Travis build stuff

* I am dumb; forgot to add the os-generic install file for travis to pick up.

* Change the trusty build type to generic, otherwise Travis uses a YUGE vm with all the things.

* Disable aptitude upgrading. dhclient is getting in the way.

* Re-order the linux-specific travis install script. Hope that protocol buffers gets installed correctly this time. Remove test generation and running from end of install script -- it doesn't belong there.

* Try to purge dhcp client, then add it back in, then do the normal update + upgrade

* Something is very dumb about Travi CI's generic linux build environment. It seems to be having trouble with the protobuf-compiler package -- that or somehting is going wrong just before that package is installed.

* Use the cpp build language -- I think this defaults to that mega vm though. Print out better OS information at the beginning of the travis install. Individually install needed packages, as opposed to doing it all at once.

* OK, using cpp language got me further, but builds take much longer. I was still missing cstdint, though. This time I'll try building with the generic trusty travis image, and with build-essential and the g++ 4.8 stdlib dev files installed.

* Lets try using the libstdc++ dev files

* Add in the x32 versions of the dev files. Hopefully that'll resolve the missing cstdint for the 32-bit builds of some of the instruction semantics bitcode files.

* Hopefully these are right!

* Install g++ multilib package.

* Simplifications to installer, and also to the README to now use g++-multilib.
2016-10-01 19:10:37 -04:00

220 lines
7.4 KiB
C++

/* Copyright 2015 Peter Goodman (peter@trailofbits.com), all rights reserved. */
#include <glog/logging.h>
#include <sstream>
#include <system_error>
#include <sys/stat.h>
#include <unistd.h>
#include <llvm/Bitcode/ReaderWriter.h>
#include <llvm/IR/BasicBlock.h>
#include <llvm/IR/Function.h>
#include <llvm/IR/Instructions.h>
#include <llvm/IR/IRBuilder.h>
#include <llvm/IR/LLVMContext.h>
#include <llvm/IR/Module.h>
#include <llvm/IR/Verifier.h>
#include <llvm/IRReader/IRReader.h>
#include <llvm/Support/FileSystem.h>
#include <llvm/Support/raw_ostream.h>
#include <llvm/Support/SourceMgr.h>
#include <llvm/Support/ToolOutputFile.h>
#include "remill/BC/ABI.h"
#include "remill/BC/Util.h"
namespace remill {
// Initialize the attributes for a lifted function.
void InitFunctionAttributes(llvm::Function *function) {
// Make sure functions are treated as if they return. LLVM doesn't like
// mixing must-tail-calls with no-return.
function->removeFnAttr(llvm::Attribute::NoReturn);
// Don't use any exception stuff.
function->addFnAttr(llvm::Attribute::NoUnwind);
function->removeFnAttr(llvm::Attribute::UWTable);
// To use must-tail-calls everywhere we need to use the `fast` calling
// convention, where it's up the LLVM to decide how to pass arguments.
//
// TODO(pag): This may end up being finicky down the line when trying to
// integrate lifted code function with normal C/C++-defined
// intrinsics.
function->setCallingConv(llvm::CallingConv::Fast);
// Mark everything for inlining.
function->addFnAttr(llvm::Attribute::AlwaysInline);
function->addFnAttr(llvm::Attribute::InlineHint);
}
// Create a tail-call from one lifted function to another.
void AddTerminatingTailCall(llvm::Function *source_func,
llvm::Value *dest_func) {
if (source_func->isDeclaration()) {
llvm::IRBuilder<> ir(llvm::BasicBlock::Create(
source_func->getContext(), "", source_func));
std::vector<llvm::Value *> args;
for (llvm::Argument &arg : source_func->args()) {
args.push_back(&arg);
}
llvm::CallInst *call_target_instr = ir.CreateCall(dest_func, args);
// Make sure we tail-call from one block method to another.
call_target_instr->setTailCallKind(llvm::CallInst::TCK_MustTail);
call_target_instr->setCallingConv(llvm::CallingConv::Fast);
ir.CreateRetVoid();
} else {
AddTerminatingTailCall(&(source_func->back()), dest_func);
}
}
void AddTerminatingTailCall(llvm::BasicBlock *source_block,
llvm::Value *dest_func) {
CHECK(nullptr != dest_func)
<< "Target function/block does not exist!";
LOG_IF(ERROR, source_block->getTerminator() ||
source_block->getTerminatingMustTailCall())
<< "Block already has a terminator; not adding fall-through call to: "
<< (dest_func ? dest_func->getName().str() : "<unreachable>");
llvm::IRBuilder<> ir(source_block);
// Set up arguments according to our ABI.
std::vector<llvm::Value *> args(kNumBlockArgs);
args[kStatePointerArgNum] = LoadStatePointer(source_block);
args[kMemoryPointerArgNum] = LoadMemoryPointer(source_block);
args[kPCArgNum] = LoadProgramCounter(source_block);
// We may introduce variables like `__remill_jump_0xf00` that boils down to
// meaning the `__remill_jump` at offset `0xf00` within the lifted binary.
// Being able to know what jump in the lifted bitcode corresponds with a
// jump as a specific area in the binary is useful for introducing things
// switch instructions to handle statically known jump tables.
if (!llvm::isa<llvm::Function>(dest_func)) {
dest_func = ir.CreateLoad(dest_func);
}
llvm::CallInst *call_target_instr = ir.CreateCall(dest_func, args);
// Make sure we tail-call from one block method to another.
call_target_instr->setTailCallKind(llvm::CallInst::TCK_MustTail);
call_target_instr->setCallingConv(llvm::CallingConv::Fast);
ir.CreateRetVoid();
}
// Find a local variable defined in the entry block of the function. We use
// this to find register variables.
llvm::Value *FindVarInFunction(llvm::BasicBlock *block,
std::string name,
bool allow_failure) {
return FindVarInFunction(block->getParent(), name, allow_failure);
}
// Find a local variable defined in the entry block of the function. We use
// this to find register variables.
llvm::Value *FindVarInFunction(llvm::Function *function, std::string name,
bool allow_failure) {
for (auto &instr : function->getEntryBlock()) {
if (instr.getName() == name) {
return &instr;
}
}
CHECK(allow_failure)
<< "Could not find variable " << name << " in function "
<< function->getName().str();
return nullptr;
}
// Find the machine state pointer.
llvm::Value *LoadStatePointer(llvm::Function *function) {
CHECK(kNumBlockArgs == function->arg_size())
<< "Invalid block-like function. Expected two arguments: state "
<< "pointer and program counter in function "
<< function->getName().str();
static_assert(0 == kStatePointerArgNum,
"Expected state pointer to be the first operand.");
return &function->getArgumentList().front();
}
llvm::Value *LoadStatePointer(llvm::BasicBlock *block) {
return LoadStatePointer(block->getParent());
}
// Return the current program counter.
llvm::Value *LoadProgramCounter(llvm::BasicBlock *block) {
llvm::IRBuilder<> ir(block);
return ir.CreateLoad(ir.CreateLoad(FindVarInFunction(
block->getParent(), "PC")));
}
// Return the pointer to the current value of the memory pointer.
llvm::Value *LoadMemoryPointer(llvm::BasicBlock *block) {
llvm::IRBuilder<> ir(block);
return ir.CreateLoad(FindVarInFunction(
block->getParent(), "MEMORY"));
}
// Find a function with name `name` in the module `M`.
llvm::Function *FindFunction(const llvm::Module *module, std::string name) {
return module->getFunction(name);
}
// Find a global variable with name `name` in the module `M`.
llvm::GlobalVariable *FindGlobaVariable(const llvm::Module *module,
std::string name) {
return module->getGlobalVariable(name);
}
// Reads an LLVM module from a file.
llvm::Module *LoadModuleFromFile(llvm::LLVMContext *context,
std::string file_name) {
llvm::SMDiagnostic err;
auto mod_ptr = llvm::parseIRFile(file_name, err, *context);
auto module = mod_ptr.get();
mod_ptr.release();
CHECK(nullptr != module)
<< "Unable to parse module file: " << file_name << ".";
module->materializeAll(); // Just in case.
std::string error;
llvm::raw_string_ostream error_stream(error);
CHECK(!llvm::verifyModule(*module, &error_stream))
<< "Error reading module from file " << file_name << ". " << error << ".";
return module;
}
// Store an LLVM module into a file.
void StoreModuleToFile(llvm::Module *module, std::string file_name) {
std::string error;
llvm::raw_string_ostream error_stream(error);
CHECK(!llvm::verifyModule(*module, &error_stream))
<< "Error writing module to file " << file_name << ". " << error << ".";
std::error_code ec;
llvm::tool_output_file bc(file_name.c_str(), ec, llvm::sys::fs::F_RW);
CHECK(!ec)
<< "Unable to open output bitcode file for writing: " << file_name << ".";
llvm::WriteBitcodeToFile(module, bc.os());
bc.keep();
CHECK(!ec)
<< "Error writing bitcode to file: " << file_name << ".";
}
} // namespace remill