mirror of
https://github.com/lifting-bits/remill
synced 2026-06-21 13:56:07 +00:00
fb018c96e96dcfc933517d017a9b12f01b8a9708
22 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Alex Cameron
|
fb018c96e9 |
PowerPC Support (#645)
* Add skeleton for PPC
* Copyright notices
* Fill in some details for the PPC arch
* Start building a (wrong) PPC runtime
* Begin populating state structure
* First pass for EIS state structure
* Map registers to Sleigh register names
* More fixes
* add optional param
* Create handle unsupported and invalid instruction isels
* Correct typo
* Get a basic `remill-lift` invocation running without failure
* Fix capitalisation
* Set vle context reg
* Fix SleighDecoder signatures
* Set VLE context register in the Sleigh engine in addition to our
internal context reg mapping
* Capitalize reg names
* Add the flag registers for XER and CR
* Rename bitflag structures in PPC state
* PPC Sleigh patches (#643)
* Modified sleigh patch script to generate patches for multiple .sinc files
* update README with new examples of sleigh patch script invocation
* add ppc register definition
* add ppc sleigh patches
* fix issue with remill_insn_size definition
* regenerate sleigh patches for PPC
* update CMakeLists.txt to include PPC patches
* Add TEA signal as a register in the PPC state
* Uppercase the stack pointer register name
* Fix PPC instruction sizes
* initial PPC tests
* remove duplicate tests
* fix tests for e_stmvgprw/e_ldmvgprw
* add tests for loading/storing from special registers
* add tests with internal conditionals in pcode
* fix for pc reg and addr width not being the same... I suspect this issue is going to come up elsewhere
* add heuristic for flow from normal intrainstruction flow
* rework tests to allow testing for different sized registers
* add tests for overflow and record add
* fix bug with log printout
* add intrafunction control flow lifting
* handle edge case where there is no pcode op at the zero index
* Fix another inconsistency with mismatching address and PC reg size
* Allocate unique ptrs in the entry block
* Fix `INT_LEFT` and `INT_RIGHT` impl where shift exceeds bit width
* fix supiece lift?
* Add PPC emulate instruction to hyper call
* fix for pc reg and addr width not being the same... I suspect this issue is going to come up elsewhere
* add heuristic for flow from normal intrainstruction flow
* add intrafunction control flow lifting
* handle edge case where there is no pcode op at the zero index
* Fix another inconsistency with mismatching address and PC reg size
* fix supiece lift?
* Allocate unique ptrs in the entry block
* Fix `INT_LEFT` and `INT_RIGHT` impl where shift exceeds bit width
* fix int2float semantics
should use appropriate sized float based on the output size
* add tests for lifting int2float
* fix INT_{LEFT,RIGHT} semantics
should be `ICmpSGE` instead of `ICmpSGT`
* add cr0-7 registers
* fix formatting
* fix conditional branch test
* add test for compare
* re-enable rotate left word immediate and mask test
* genericize TestSpecOutput
* explicit instruction data size
* add test for syscall/callother (disabled)
* add tests for store/load word
* add test to convert from float to int
* specify intrinsic arg type, fixes null deref
* Add PPC emulate instruction to hyper call
* add headers + formatting
* remove old comment
* Map CRALL register
* Add basic LLVM data layout that specifies 32-bit addresses
* Remove unused variables
* convert auto* to auto when possible
* RegisterPrecondition -> RegisterCondition
* fix variable name
* convert any to variant
* use std::move
* bump to c++20, use concepts
* set arch in constructor since class isn't generic anyways
* formatting
* make type aliases
* bump cxx-common
* add comment
* clang format
* throw exception if register not found
* use const ref
* use shorthand for lambda capture values
* Add more detail to data layout to include proper stack alignment
* Compare to the correct size for SUBPIECE impl
* Add Sleigh message to error
* throw exception in else case
* throw runtime error if register value has incorrect type
* use reference instead of value
* get rid of unnecessary type alias
* formatting
* Propagate VLE context reg value into Sleigh
* Remove unnecessary whitespace
* Remove stale TODO and NOTE comments
* add additional parameter to test runner to specify decoding context
* drop llvm 14, bump macos version
* bump cxx-common, fix ci.yml mac build
* add test for unconditional relative negative branch
* add missing space to pcode debug log
* fix bug due to unordered_map, iteration order matters
* add error log in case we aren't able to adjust PC value
* use helper for getting register reference
* Revert "add optional param"
This reverts commit
|
||
|
Alex Cameron
|
d5c5035728 |
Get Remill building with LLVM 15 (#631)
* Get Remill building with LLVM 15 * Add missing header for x86 lift test * Fix `enableOpaquePointers` calls * Define a non-extern `__remill_state` in each Instructions module * Remove `__remill_state` variables in tests * Update build script to support LLVM 15 * Build with LLVM 15 in CI * Bump CXX Common version * Update Docker script * Add comment explaining the definition of the state variable * Correct wording |
||
|
2over12
|
854c73e06a |
Ian/instruction carry lifter (#615)
* add lifter field * arch has intrinsics * use insn lifter in trace lifter * limit default lifters to ops only * init lifters * pass arch in tests * fix bug in swap * stop using archbase decode in sleigh arches |
||
|
Alex Cameron
|
752ddf3ec0 | Enable opaque pointers in each spot where we make an LLVM context | ||
|
Lukas Korencik
|
c8e058c8f4 | test: Reflect changes in remill api. | ||
|
Sonya
|
6322f794d1 |
New arch support (#461)
* This branch contains support for new architectures. * Initial start to support for AArch 32 * Progress * Forgot the new files * Added all data Integer processing instructions without S + ADDS and started ANDS * Updated * Finished Integer Data Processing with three registers, added integer data processing with 2 regs + immediate, started MUL instructions * UMULL, UMULLS, UMLAL, UMLALS * Corrected condition for addend or 0 immediate for UMULL/UMLAL + SMULL/SMLAL instructions * Correct ops in Binary.cpp * UMAAL * SMULL, SMULLS, SMLAL, SMLALS + corrected acc was missing shift left in concatination * Updated decoding instructions based on top level encodings * Update returns around kDataProcessingRI and kDataProcessingI with comments to explain the correlation to the instruction rep * Added appropriate inst.category flags to Multiply and accumulate * Load/Store Word, Unsigned Byte (immediate, literal) && start of Logical Arithmetic (three register, immediate shift) * Was missing UMAAL DEF_ISEL in Binary.cpp * AddAddrRegOp * Logical Arithmetic (three register, immediate shift) without accounting for the possible PC jump * Made DecodeA32ExpandImm much much smaller * Replaced some imm ops with AddImmOp calls * Created AddShiftOp * Added interpreter for evaluating new PC value at decoding time to handle direct jumps and conditional jumps * Created EvalPCDest added PC evaluation to Logical Arithmetic Instructions * AddShiftOp -> AddShiftOp, AddShiftThenExtractOp, AddExtractThenShiftOp * Cleaned up some formatting, Renamed DecodeA32ExpandImm to ExpandTo32AddImmAddCarry and added a clarifying comment * Added comment to EvalPCDest for clarity * Cleaned up some things, updated the decoding semantics and semantics for the logical instructions * Shortened kLogArithEvaluators and fixed a bug * Updates from testing instructions * Fixed DEF_ISEL for pre/post index instructions in MEM.cpp * Integer Test and Compare (two register, immediate shift) * Logical Arithmetic (two register and immediate) * Integer Test and Compare (one register and immediate) * Added to the top level encoding infrastructure to handle the Data-processing register (register shift) set of instructions and 3 corresponding subsets * Add structs for the 3 subsets of Data-processing register (register shift) * Code status before refactoring operand types * This branch contains support for new architectures. * Initial start to support for AArch 32 * Progress * Forgot the new files * Added all data Integer processing instructions without S + ADDS and started ANDS * Updated * Finished Integer Data Processing with three registers, added integer data processing with 2 regs + immediate, started MUL instructions * UMULL, UMULLS, UMLAL, UMLALS * Corrected condition for addend or 0 immediate for UMULL/UMLAL + SMULL/SMLAL instructions * Correct ops in Binary.cpp * UMAAL * SMULL, SMULLS, SMLAL, SMLALS + corrected acc was missing shift left in concatination * Updated decoding instructions based on top level encodings * Update returns around kDataProcessingRI and kDataProcessingI with comments to explain the correlation to the instruction rep * Added appropriate inst.category flags to Multiply and accumulate * Load/Store Word, Unsigned Byte (immediate, literal) && start of Logical Arithmetic (three register, immediate shift) * Was missing UMAAL DEF_ISEL in Binary.cpp * AddAddrRegOp * Logical Arithmetic (three register, immediate shift) without accounting for the possible PC jump * Made DecodeA32ExpandImm much much smaller * Replaced some imm ops with AddImmOp calls * Created AddShiftOp * Added interpreter for evaluating new PC value at decoding time to handle direct jumps and conditional jumps * Created EvalPCDest added PC evaluation to Logical Arithmetic Instructions * AddShiftOp -> AddShiftOp, AddShiftThenExtractOp, AddExtractThenShiftOp * Cleaned up some formatting, Renamed DecodeA32ExpandImm to ExpandTo32AddImmAddCarry and added a clarifying comment * Added comment to EvalPCDest for clarity * Cleaned up some things, updated the decoding semantics and semantics for the logical instructions * Shortened kLogArithEvaluators and fixed a bug * Updates from testing instructions * Fixed DEF_ISEL for pre/post index instructions in MEM.cpp * Integer Test and Compare (two register, immediate shift) * Logical Arithmetic (two register and immediate) * Integer Test and Compare (one register and immediate) * Added to the top level encoding infrastructure to handle the Data-processing register (register shift) set of instructions and 3 corresponding subsets * Add structs for the 3 subsets of Data-processing register (register shift) * Code status before refactoring operand types * Finished updates off master * Start of operand refactor * Finished Expression Operand Support * Fix the .gitignore to add AArch32 to lib/Arch && removed all extra rrx ops from semantics * Updated .gitignore again, Added AddShiftRegRegOperand, Updated AddShiftRegImmOperand, Finished Register shift instructions for Integer Test and Compare, Logical Arithmetic, Integer Data Processing * Updated ROR in AddShiftRegRegOperand * Created ExtractAndZExtExpr * Fixed comment formatting in if else statements * Created RORExpr * Small fixes * Small fix in Logical Arithmetic (two register and immediate) * Corrected AddShiftRegRegOperand and cleaned it up. Split the carry op into a separate function. * conditional support + Start of Branch instructions * Created AddExprOp, cleaned up some expressions in reg shifted reg, and updated some occurances of ShiftThenExtractOp with ExtractAndZExtExpr * Updates from testing register shifted by register value inst * Fix to ROR in AddShiftRegCarryOperand * Corrected negation in DecodeCondition * DecodeCondition edit * DecodeCondition and AddShiftRegCarryOperand edits * Updated arch_for_decode to arch * Halfword Multiply and Accumulate * Edits from testing Halfword Multiply and Accumulate * Changed order of operands in Halfword Multiply and Accumulate to better reflect inst format + updated inst errors * Branch (Imm) & BX/BXL * Update aarch32 cmake * cmake update * CLZ * Forgot BITBYTE.cpp * MOVT * Integer Saturating Arithmetic * updated semantics in SMLAWh & SMLAh to use Select for setting PSTATE.Q * Started Load/Store Word, Unsigned Byte (register) & fixed MOV halfword * Load/Store Word, Unsigned Byte (register) * Finished testing load/Store Word, Unsigned Byte (register) * Load/Store Dual, Half, Signed Byte (register) * Rest of Extra load store: Load/Store Dual, Half, Signed Byte (immediate, literal) * Finished testing all the Load/store additions * Signed multiply, Divide * Cleaned up SExt some * Saturate Insts and Start of Load Store Multiple - STMDB and LDM (aliases which support PUSH and POP of multiple regs) * Condensed args in STMDB and LDM semantics * Rest of Multiple Load/Store that do not execute in a different mode * Bitfield Extract * Extend and Add * fix * NOP * Small fix * Simplified the bit reps in TryMoveSpecialRegisterAndHintsI * Moved Bitfield extract semantics out of BINARY and into BITBYTE * Finished correcting S/ZExt and Trunc use * Ran scripts/format-files to format * Smoke Test * Add false delay slot to kCategoryConditionalDirectFunctionCall * CI: Use single packaging job, add changelog support (#491) * CI: Add tag handler (#492) * Delay slot fixes to TraceLifter Co-authored-by: Peter Goodman <peter.goodman@gmail.com> Co-authored-by: Alessandro Gario <5714290+alessandrogario@users.noreply.github.com> |
||
|
Peter Goodman
|
3808e9951d |
Refactor and add sparc (#454)
* Refactors the code to improve directory layout, use more std::string_view in place of std::string, deletes some deprecated functions, deprecates some other functions for eventual deletion, and includes semantics for sparcv8 (sparc32) and sparcv9 (sparc64) * Update new dir layout with llvm 11 support * Whoops missing files * Drop llvm 800 from workflow * Minor fix * Move where the install directives are in CMake * Minor fixes * Rename tools/ to bin/. * Minor tweaks * Should fix issues |
||
|
Peter Goodman
|
99df2e19d4 |
Running clang-format on files with some additional custom scripts for… (#444)
* Running clang-format on files with some additional custom scripts for my style * Fix missing unique_ptr in remill/BC/Optimizer.h * Fixes and selective disabling of clang-format |
||
|
Peter Goodman
|
5af0122196 |
General API improvements (#424)
* General API improvements * Minor changes * New name for existing isel * Error compatibility with llvm8 * llvm 10 to build script Co-authored-by: kumarak <iit.akshay@gmail.com> |
||
|
Aiethel
|
d5ba741c5b |
Util improvement (#399)
* Use std::unique_ptr to manage llvm::Module after it is loaded. * Remove calls from Util that are internally dependent on global flags. In reality they only save a few characters for API users and eventually this kind of code can lead to tightly coupled code with global state. * Remove code duplication. * Reflect API change in tests. |
||
|
Peter Goodman
|
d6e3b10aa7 | Support code for ANVILL decompiler (#361) | ||
|
Peter Goodman
|
508ae0f8da |
Trace lifter api (#302)
* Adds an initial implementation of the trace lifter API, and remill-lift tool that uses the API, and also makes the x86 test suite lifter use the API. * Minor fixes. * Minor fix to conditional async hyper call lifting. Some added comments. * Minor fix to cmakelists * Add missing headers * Make a more convenient (and portable) function for saving IR to a file. * Minor fix. * Minor fix on LLVM version compatibility. |
||
|
Alessandro Gario
|
85dc1282b7 |
WIP: Add Windows support (#235)
* CMake: Make the bitcode generator Visual Studio friendly * CMake: Always recompile the runtimes when the semantics change * CMake: Style changes * CMake: Fix missing variable when re-configuring * CMake: Windows/Visual Studio fixes * Windows: Add missing FPU macros * Fix alignment issues with Visual Studio * CMake: Fix the install target for Windows * CMake: Add support for find_package(remill) * Build script: Use verbose makefile * CMake: Add missing ADDRESS_SIZE_BITS in BC compiler * CMake: X86 Tests project refactor * Windows: Port of BC/Utils and OS/FileSystem * CMake: Update settings.cmake * CMake: Generate the semantics path in remillConfig.cmake * CMake: Only use C++14 when compiling on Windows * Rename FPUFlags to Float and collect all FPU flags there * Travis: Add macOS, test all LLVM versions, add clang static analyzer * Travis: Disable LLVM50 tests, include verbose output for failed tests * CMake: Update the find_package handler, add support for fcd headers * Dead store elimination (draft) (#257) * Add initial prototying of DeadStoreEliminator * Improve upon DeadStoreEliminator prototype Update `DeadStoreEliminator` prototyping with additional comments and proper function prototype return values. Fix a small warning in `Lifter.h` by adding `llvm::Value` and `llvm::BasicBlock`. * Add initial code for DeadStoreEliminator Provide namespaced `remill::StateSlots` function which visits fields of the module's state struct and returns `StateSlot` objects. Remove `comment` field as the `State` object (an `llvm::StructType`) does not track names in a useful way. * Update state analyzer and fix type errors Update `DeadStoreEliminator.cpp` to properly produce a vector of `StateSlot`s without type errors (i.e. it compiles). Code still needed for other container types besides structs. * Add remill::VisitSequential for arrays and vectors Add state analyzer code for LLVM's `ArrayType` and `VectorType`. Add debugging use of `llvm::Type::dump()`. Consider refactoring as a `StateVisitor` class. * Begin refactor of Visit funcs to StateVisitor * Fix segfault in StateVisitor::visit, code style * Prototype ForwardAliasVisitor for alias analysis Add code for ForwardAliasVisitor subclass of `llvm::InstVisitor`, to be used for performing alias analysis of lifted functions. * Add non-working visit functions for alias analysis * Fix compilation of alias analysis visit funcs * Change ForwardAliasVisitor to RetTy=bool Update ForwardAliasVisitor to use booleans for return types to track when we should add the instruction to the next_wl. Add code to simplify StateSlots for vecs of ints. * Add progress tracking to AliasAnalysis * Correct progress tracker, use BasicBlockFunction() * Change ForwardAliasAnalysis<RetTy = AliasResult> Get some of those sweet sweet enums in there! * Add FAV state pointer field, PHINode impl * Allow non-const add and sub in FAV Allow add and sub instructions with two pointers in the offset map. Add to implementation of visitPHINode. * Update PHINode impl in FAV * Clean up use of StateSlots to create AAMDNodes Add code to create AAMDNodes from StateSlot elements. Change creation of StateSlot vector to have elements for every byte offset of the state structure for fast indexing. * Complete addition of AAMDNodes for load and store Move function defs up for AAMDNode ops. Add AliasMap typedef. Finish generateAAMDNodesFromSlots. * Add GenerateLiveSet func * Initialize live set, add AAMDNodes to stores * Change LiveSet creation to a block visitor class * Add to_remove set to LSBV * Update build script to use os-release for OS detection * Update DSE code to conform to pag's review * Refactor LiveSetBlockVisitor to one LiveSet per block * Update VisitBlock to better check instruction type * Fix VisitBlock CallInst and InvokeInst cases * Stack allocate AAMDInfo in AnalyzeAliases * Add remove pass option for VisitBlock * Add DOT digraph generation * Fix bugs in dot digraph * Fix various bugs in AAMDNodes and LSBV Move AAMDNodes functions later to match their use. Clean up code conventions. Fix small bugs in various spots in the code. * Fix various function prototypes, overflow checks * Merge AliasMap and OffsetMap * Change add/sub insts to be safer Add OpType enum class to replace use of `plus` bool. Add more straightforward bounds checking on AddInst or SubInst values. * Refactor GetUnsignedOffset style * Fix illegal instruction error * Add offset checking for GEP, provide log messages Write a log message for the cases where `GetUnsignedOffset` returns false (indicating an overflow or underflow). * Fix APInt initialization in VisitGEP * Re-add dead store elimination and alias map * Move LSBV into alias analysis, expand callinst Expand definition of cases where a CallInst should be considered to touch the state struct or otherwise revive a slot. * Improve DOT creation, fix errors for callinsts Fix small errors in LiveSetBlockVisitor::CallAccessesState. Clean up DOT digraph generation further. * Clean up code per @pag's comments * Clean up code, add selectinst, fix compile errors Deal with a few small corner cases with FAV Load and Store instructions, add SelectInst visitor. Inline MarkLiveArgs to avoid the complications of C++ generics. * Clean up logging * More changes to please Peter Clean up select and PHI node cases to improve circular dependency handling. * Correct errors in visitSelect * Add load forwarding code * Add code to run FBV * Fix map usage error in FBV * Add call, invoke cases for FBV * Here are the changes whoops * Minor API change * Minor tweaks to DOT digraph printing, as well as attempts to handle the case where the forward analysis pass is incomplete. Still don't have it guarantee completion on everything, but seems 'good enough' for now. * Change log level * Fix compile errors due to messy merge * Pag dead store (#262) * Fix to script calling wrong function. * Makes sure that value names are preserved (#249) * Adds LLVM_VERSION() to accomodate llvm::LLVMContext::setDiscardValueNames() in <3.9 (#250) * Makes sure that value names are preserved * Adds LLVM_VERSION() macros to accomodate * Update README.md * Update README.md * set state and memory as noalias (#254) * Implements some ring 0 instructions in terms of hyper calls and new I/O port intrinsics. (#252) * Random tests. * More decode error info * more playing around * more system instructions. instrinsics for accessing I/O ports. Split our writes to individual control regs for better identification via hyper calls. * CR8 read/write support (#255) * Check argument index of function (#256) * Fix NoAlias Attributes for older LLVM versions * Fix for LLVM 4.0 and 3.9 * Fix typo * Follow remill coding style * Here are the changes whoops * Minor API change * Minor tweaks to DOT digraph printing, as well as attempts to handle the case where the forward analysis pass is incomplete. Still don't have it guarantee completion on everything, but seems 'good enough' for now. * Change log level * Fix compile errors due to messy merge * Improve call/invoke case of FBV * Improve call/invoke case of FBV * Removes a level of indirection in the __remill_basic_block function * Create dedicated stats tracker Plus clean up the FBV visitor a teensy bit more. * Move call/invoke LiveSet gen to static func This commit is to pave the way for future improvements to where this information is calculated (in FAV instead of LSBV). * Add code to FAV for calls and invokes Move call and invoke arg-based livesets to FAV to allow for module-level LSBV code. * Begin change of LSBV to module-level * Minor bug fixes * Remove some dead code * Begin adding more code for call/invoke LSBV * Add entry block checks for LSBV call/invoke * Info about register names, as well as printing them in the digraphs. * Make DOT printing only happen per function, as opposed to printing every function per DOT file. Minor tweaks to interprocedural analysis. * Bug fixes and DOT printing improvements. * Bug fixes and DOT printing improvements. * Also print out DOT digraphs of functions after removing stuff * Tried to make it treat everything before a call to __remill_error as dead but that didn't work out. * Use datalayout and type sizes to handle the number of elements in a sequential type for LLVM 3.8 compatibility, also check for pointer type. * More stats, hopefully fixes a bitcast issue. * Minor bug fixes, and more DOT printing to help diagnose when the offset analysis terminates but there is still stuff in the work list. * Some possible bug fixes * Minor bug fix * Travis build fixes for earlier compatibility * Fix for LLVM less than 3.8 compatibility. * LLVM 3.5 compatibility * Missing condition in forwarding code that does casting. * LLVM 5.0 compatibility * Add LLVM 6.0 to build.sh. Change default install location to /usr/local * Add back in vmill. Add some extra flags to the building runtimes. * Maybe works * Disable dse across indirect call (#267) * Make DSE sensitive to indirect function calls * FE_DENORM issue * Added some compat code that implements the futimens syscall on mac os 10.12 for travis support. * Disable dse across indirect call (#268) * Make DSE sensitive to indirect function calls * FE_DENORM issue * Added some compat code that implements the futimens syscall on mac os 10.12 for travis support. * Playing with CACHE and PARENT_SCOPE * Minor stack address size check * Fix to DSE I think. * Minor DSE tweak * Update Run.cpp * Update travis.sh Adds in 32-bit libraries for Linux builds. |
||
|
Mike Myers
|
d94d09efd6 |
Issue 187 xeon kvm support (#192)
* Fix FPU IP state tracking * Fix LZCNT, TZCNT tests on unsupporting CPUs * Preserve segment registers CS and DS, fixing failing tests |
||
|
Mike Myers
|
4dc9f3712e | Enhance log messages for inst. decoding failures | ||
|
Peter Goodman
|
927998635a |
Implements more accurate tracking of x87 exception flags, last instru… (#163)
* Implements more accurate tracking of x87 exception flags, last instruction pointer, last data pointer, and floating point instruction opcode. * Fix that masks out the MXCSR register from being compared at the end of tests. We don't yet have good enough info to do this well. Moves the FCMOV instruction set to be under the purview of X87.cpp and its tests. * Tests should pass now. * Addressing issues Mike mentioned. |
||
|
Peter Goodman
|
b06fd79cd6 | llvm 5.0 compatibility, see Issue #139 | ||
|
Peter Goodman
|
ee8b295f97 | Minor refactorings. | ||
|
Peter Goodman
|
bef332fd2a |
Aarch64 tester (#128)
* Added in more aarch64 instructions. Fixed some x86 instructions. The x86 test cases now exercise each test through every possible combination of flags. * Adding missing files * Another missing file * Rename file * Fixup some macros * IPR * More improvements on the test runner * Test runner fixes related to me not being familiar with aarch64 assembly * Fixing default data layout * Trying to use llc to compile bitcode to aarch64 assembly. wth. * Revert back to using the CMAKE_BC_COMPILER for building the test assembly file instead of the whole CMAKE_LL_COMPILER stuff, now that I've adjusted cxx-common to use the right build target for aarch64. * Documentation updates. Fixes for aarch64. * Making progress. The native tests can run, but the first lifted test faults. Not yet sure why. * Weirdest issue is happening on aarch64. A pointer argument is being compiled to an integer, and that's really screwing things up. * Add caching of the libraries path to the main cmakelists to avoid having to re-run build.sh all the time when the TRAILOBITS_LIBRARIES env var is not globally defined. Experimenting with trying to force the semantics to be compiled using the x86_64 target, regardless of host arch, or modelled arch of the semantics. This is to try to get around the issue where a single-element struct containing a pointer is lowered into a uintptr_t when passed by value as an argument on aarch64. * Alright, falling back on handling this problem in the lifter (for now, at least). Really not ideal. * Test runner works afaict |
||
|
Peter Goodman
|
592b6cbe8d |
Aarch64 support and other stuff (#113)
* This includes most of Alessandro's wonderful cmake changes, and starts into AArch64 support, though not using Capstone anymore. * Trying to make travis work * Minor additions, still not completely working on my ubuntu 14.04 vm or on travis * Making progress on getting travis to build * Some documentation improvements * I push the travis button |
||
|
Peter Goodman
|
e1bdbdba9e |
Alessandro/refactor/llvmmultiversion cmake (#111)
* Change copyright notice in all the places. Trying to get cmake to download all the things * CMake refactor * Added the lib repository installer * Fixed a couple of paths in library repository scripts * Spelling and a missing .gitignore file. * The google test library was not correctly linked * Removed the pre-compiled XED library * Removed unused files; fixed the INSTALL directives. Added automatic protobuf generation. * Added a package generator script for ArchLinux * Merged in Alessandro's cmake magic * Cleanups, compatibility changes * CMake cleanup. o Removed the cmake folder as it is no longer used. o Removed the library_repository_installer directory and replaced it with a submodule to the newly introduced cxx_common repository. o Added the 'use_remill_semantics' branch of mcsema as a submodule under tools/cmake. o Updated the README instructions (added --recursive to the git clone command). Warning: the mcsema submodule is pointing to my fork of the repository! * CMake: avoid re-defining the C/CXX/ASM compiler. This will prevent CMake from looping forever when using submodules. * The tools/mcsema submodule now points to the official repository. * mcsema submodule update * Updated the mcsema git submodule to track the newest changes. * Various CMake fixes (see details). o External include headers were not correctly marked as SYSTEM. This caused them to output warnings and break the build. o The PROJECT_SOURCE CMake variable has been replaced with CMAKE_SOURCE_DIR. o Updated the mcsema submodule to point to the latest CMake fixes. * Getting closer to having the test case runner work again without using the CFG protobufs. * Trying to make things use ubuntu clang/llvm packages * Update to the mcsema submodule. * Minor changes for llvm version compatibility * Test cases should run now. Had to compile the lifted testcases to a .S file, as with the .bc file, they were being optimized away.. I think. The compilation to assembly seems unusually slow, though. * Minor change * Some stuff for llvm 4.0 support * Remove mcsema sub-module * Remove cxx-common submodule * First steps toward getting travis working again * Attempt at getting travis working again * Using https for cloning instead of ssh * Minor build script update * Added ISEL_ prefix to isels to make it easier for ForEachISel to find them. Commented out the defer_inlining intrinsics. * Changes related to mcsema2 * Simplify runtime targets generation. (#110) * CMake refactor: Added a new language 'BC' for the bitcode (see details). The language is used to generate the runtimes used by the architecture modules. The required executables (clang++ and llvm-link) are automatically detected in the same way as other compilers are. A new CMake function has been added and it can be used to easily generate runtime targets in a way similar to add_executable: add_runtime(<name> SOURCE <source list> ADDRESS_SIZE <n> DEFINITIONS <definition list>) Additionally, all files ending with the *.bcpp extensions will automatically invoke the bitcode compiler when listed in an active target. This should open support for parallel compilation! You just have to list all your .cpp files when calling add_runtime. * CMake: Fixes to the BC language handler. * CMake/BC: Use '.bo' for object files. X86 runtime: Add -g/-O flags. |
||
|
Peter Goodman
|
7dd87902ea |
Vmill (#106)
* Initial commit of x86 program snapshotter. Haven't yet figured out how to save a core dump to a specific file. Now finds and saves core dumps (in a sketchy way). Kind of but didn't really fix remill-disass with core dumps. It almost seems like a bunch of stuff is missing from within a core dump. It may be the case that it will be simpler to use binary ninja directly on snapshot files. Working toward the executor. Have snapshot creation and loading working. Next up: decoding the first few instructions! Starting to serverize remill and setting up lines of communicating between vmill and remill. Going to move on to finally implementing remill-opt. Changed a bazillon things. Fixed the dead register backward data-flow analysis. Fixed soem bugs, added GEP re-association, working toward inserting stores that will kill values. Got interprocedual dead store elimination working! :-D Finally...dead store elimination remill-opt is done. Added Lifter interface Spec'ing out the translation engine. Got the dynamic decoder working! Got bitcode caching working Added caching layer for bytecode compilation. Decided on how to access reg state and allocas in functions. The bytecode will treat the state struct and the alloca'd data as a contiguous, opaque, byte-addressible area. Compiler seems to work Refactor Great progress...goodnight Made it up to the first syscall Fixed call to a hypercall intrinsics Bug fixes, minor change to the CFG proto. Refacotorings and changes * Switching to trying nativeexec, and made memory32 map snapshot into low 32-bits of address space, preserving original addresses of program. * About to make some interesting changes, so save save save * Refactor done, now time to produce shared libraries * Got initial execution of some stuff in PHP working.. it either seems like the code is in an infinite loop, or just horribly slow, not sure. Otherwise, amazing progress. * Got initial compiling to a runtime dynamic library working * Separated most DEF_ISEL_SEMs and tests still pass. Goodnight! * Got incremental optimization and compilation working. * Got caching of the bitcode file to disk working and periodically collapsing the shared libraries into a single library. OUT. * Fixed some bugs * Commit before the storm * Made the JIT work again, still not that fast though. * Minor logging fix * Log an error that we're executing a missing instruction. * JIT compile the whole module first, then incrementally JIT compile function partitions. Also, link in libm. * Added new syscall * Added breakpoint sync hypercall, useful for testing. * Got a php5.4 unserialize bug to reproduce. Added remill-pinshot, which will use PIN to take a snapshot and print out a register trace. This is useful for debugging divergences. Fixed up the semantics of some instructions, and added semantics and tests for PSRLDQ. * Simplifications to remill that removes all the various basic block arrays, and uses meta-data instead. * Made cmake take over the test system * Working on mega refactor to eventually permit klee support * Made all semantics code return memory pointer; I think this makes LLVM's optimizer a bit happier. Made the __remill_sub_N things into global constants, referencing the actual functions, that have private linkage. Slowly getting back in the direction of execution. I've got some bitcode translation working (with a hard-coded address for my local php version). The address space stuff seems to work so far. * Missing files. * Other minor fixes * rtti-related fixes for gtest * minor travis config change * minor travis config change * sendfile no longer needed * Disable building vmill on non-linux platforms * Disable 32-bit test builds on macOS. * Trying to get symbol names on mac right. * Minor fix to save state code for tests, disable testing on macOS builds because of symbol mismatches in gtest. * Fixup remill-disass to use the simplified proto structure. Something may be wrong with remill-opt, or installing remill opt. Remove caching from travis. |