5 Commits

Author	SHA1	Message	Date
Peter Goodman	b96aec8fd8	Issue 137 lift bin ls (#155) ... * [WIP] Float Instructions for /bin/ls (#146) * Added bunch of float instructions required by `ls` Not all variants implemented but should follow same structure * FADD * FMUL * FMOV * FCVTZU * FCMPE Fixed bugs with FMOV_n_to_n Implemented FMOV_D64 variant Fixed up FADD, need to doublecheck edgecases FMUL Added Weird interaction with checking for denormalized output fpclassify(prod) gives you FP_SUBNORMAL, but in the native code the idc bit on SR and FPSR is not set so commented out for now FCVTZU added - FPSR flags not being caught and set Type fix In<float64_t> Fixes for FCVTZU Added FCMPE Sn Cleanup testing * Check unallocated encoding for float insts * Minor modifications. * Trying to wrap up some fpu operations inside of a function that can test the inputs and outputs. * Semantics, but no tests, for some variants of FMUL, FADD, and FDIV. * Tests for FDIV and FSUB. Additional tests for FADD and FMUL. * Minor tweak. * Minor tweak to using fetestexcept only once * Added UNTESTED implementation of mrs with fpsr. * FMOV vector instructions, some fixes (#149) * FMOV vector instructions, some fixes * More tests for FMOV_VECTOR * Fix to STRH isel, uses the wrong semantics. Minor modification to add DEF_CONDs to the ForEachIsel iterator. * Various instructions for aarch64 lifting WIP * Tests working except where commented Specifically STRH test throws some lifter error FCVT has issues with flag determiniation * Bug fix with oxc fpsr bit in print state * FCVTS, FMOV_S, FMOV_D * FNEG, FMADD_S/D For FMADD still seeing weird issues with flag setting on multiplication step when using denormals as factor, look into later * More precise definition of segment selectors. Directory traversal utility function. * Minor tweaks to existing APIs. * FMADD has a bug with one type * Fixed reimport of test * Semantics and tests for BICS. * Minor fix to tests. * Semantics and tests for integer REV instructions. * Minor fix. * Minor fixes, forgot an ISEL. * Tests for MOVK. * Tests for MOVZ. * Forgot test file. * More STR-related tests. * Minor fix * Minor fix * Post-decoder fixes for STR. * Minor fix? * Semantics and tests for general simd DUP. * Minor fix for arrangement specifier. Found a possible bug in capstone/keystone. * Wrong isel in test * Semantics and tests for simd ADD and SUB. * Semantics and tests for LD1_ASISDLSEP_I2_I2 * Whoops, forgot TEST_INPUTS to test. * Semantics and tests for LD1_ASISDLSEP_I1_I1 * Used wrong isel names in tests. * Tests and semantics for two more variants of post-index LD1 * Minor fix * Minor fix to a check for large op sizes. Regenerated AArch64 Extract.cpp, ordering extractors by most constrained first. I observed an issue where LD1_ASISDLSEP_I3_I3 was incorrectly extracted as LD1_ASISDLSEP_R3_R3. * Minor fix of a decoder routine for loading multiple structures. * Semantics and tests for RBIT, SMOV, and UMOV. Also semantics, but NO tests for a large number of SIMD instructions. * Tests for simd smax, smin, umax, umin. * Tests for sminp, smaxp, uminp, umaxp. * Tests for ADDP. * Tests for the simd vector register of cmeq, cmgt, and cmge. * Minor fix. * Tests for cmeq, cmgt, and cmge that compare a vector against zero.	2017-11-04 15:52:23 -04:00
Peter Goodman	e1bdbdba9e	Alessandro/refactor/llvmmultiversion cmake (#111) ... * Change copyright notice in all the places. Trying to get cmake to download all the things * CMake refactor * Added the lib repository installer * Fixed a couple of paths in library repository scripts * Spelling and a missing .gitignore file. * The google test library was not correctly linked * Removed the pre-compiled XED library * Removed unused files; fixed the INSTALL directives. Added automatic protobuf generation. * Added a package generator script for ArchLinux * Merged in Alessandro's cmake magic * Cleanups, compatibility changes * CMake cleanup. o Removed the cmake folder as it is no longer used. o Removed the library_repository_installer directory and replaced it with a submodule to the newly introduced cxx_common repository. o Added the 'use_remill_semantics' branch of mcsema as a submodule under tools/cmake. o Updated the README instructions (added --recursive to the git clone command). Warning: the mcsema submodule is pointing to my fork of the repository! * CMake: avoid re-defining the C/CXX/ASM compiler. This will prevent CMake from looping forever when using submodules. * The tools/mcsema submodule now points to the official repository. * mcsema submodule update * Updated the mcsema git submodule to track the newest changes. * Various CMake fixes (see details). o External include headers were not correctly marked as SYSTEM. This caused them to output warnings and break the build. o The PROJECT_SOURCE CMake variable has been replaced with CMAKE_SOURCE_DIR. o Updated the mcsema submodule to point to the latest CMake fixes. * Getting closer to having the test case runner work again without using the CFG protobufs. * Trying to make things use ubuntu clang/llvm packages * Update to the mcsema submodule. * Minor changes for llvm version compatibility * Test cases should run now. Had to compile the lifted testcases to a .S file, as with the .bc file, they were being optimized away.. I think. The compilation to assembly seems unusually slow, though. * Minor change * Some stuff for llvm 4.0 support * Remove mcsema sub-module * Remove cxx-common submodule * First steps toward getting travis working again * Attempt at getting travis working again * Using https for cloning instead of ssh * Minor build script update * Added ISEL_ prefix to isels to make it easier for ForEachISel to find them. Commented out the defer_inlining intrinsics. * Changes related to mcsema2 * Simplify runtime targets generation. (#110) * CMake refactor: Added a new language 'BC' for the bitcode (see details). The language is used to generate the runtimes used by the architecture modules. The required executables (clang++ and llvm-link) are automatically detected in the same way as other compilers are. A new CMake function has been added and it can be used to easily generate runtime targets in a way similar to add_executable: add_runtime(<name> SOURCE <source list> ADDRESS_SIZE <n> DEFINITIONS <definition list>) Additionally, all files ending with the *.bcpp extensions will automatically invoke the bitcode compiler when listed in an active target. This should open support for parallel compilation! You just have to list all your .cpp files when calling add_runtime. * CMake: Fixes to the BC language handler. * CMake/BC: Use '.bo' for object files. X86 runtime: Add -g/-O flags.	2017-05-15 13:00:33 -04:00
Peter Goodman	7dd87902ea	Vmill (#106) ... * Initial commit of x86 program snapshotter. Haven't yet figured out how to save a core dump to a specific file. Now finds and saves core dumps (in a sketchy way). Kind of but didn't really fix remill-disass with core dumps. It almost seems like a bunch of stuff is missing from within a core dump. It may be the case that it will be simpler to use binary ninja directly on snapshot files. Working toward the executor. Have snapshot creation and loading working. Next up: decoding the first few instructions! Starting to serverize remill and setting up lines of communicating between vmill and remill. Going to move on to finally implementing remill-opt. Changed a bazillon things. Fixed the dead register backward data-flow analysis. Fixed soem bugs, added GEP re-association, working toward inserting stores that will kill values. Got interprocedual dead store elimination working! :-D Finally...dead store elimination remill-opt is done. Added Lifter interface Spec'ing out the translation engine. Got the dynamic decoder working! Got bitcode caching working Added caching layer for bytecode compilation. Decided on how to access reg state and allocas in functions. The bytecode will treat the state struct and the alloca'd data as a contiguous, opaque, byte-addressible area. Compiler seems to work Refactor Great progress...goodnight Made it up to the first syscall Fixed call to a hypercall intrinsics Bug fixes, minor change to the CFG proto. Refacotorings and changes * Switching to trying nativeexec, and made memory32 map snapshot into low 32-bits of address space, preserving original addresses of program. * About to make some interesting changes, so save save save * Refactor done, now time to produce shared libraries * Got initial execution of some stuff in PHP working.. it either seems like the code is in an infinite loop, or just horribly slow, not sure. Otherwise, amazing progress. * Got initial compiling to a runtime dynamic library working * Separated most DEF_ISEL_SEMs and tests still pass. Goodnight! * Got incremental optimization and compilation working. * Got caching of the bitcode file to disk working and periodically collapsing the shared libraries into a single library. OUT. * Fixed some bugs * Commit before the storm * Made the JIT work again, still not that fast though. * Minor logging fix * Log an error that we're executing a missing instruction. * JIT compile the whole module first, then incrementally JIT compile function partitions. Also, link in libm. * Added new syscall * Added breakpoint sync hypercall, useful for testing. * Got a php5.4 unserialize bug to reproduce. Added remill-pinshot, which will use PIN to take a snapshot and print out a register trace. This is useful for debugging divergences. Fixed up the semantics of some instructions, and added semantics and tests for PSRLDQ. * Simplifications to remill that removes all the various basic block arrays, and uses meta-data instead. * Made cmake take over the test system * Working on mega refactor to eventually permit klee support * Made all semantics code return memory pointer; I think this makes LLVM's optimizer a bit happier. Made the __remill_sub_N things into global constants, referencing the actual functions, that have private linkage. Slowly getting back in the direction of execution. I've got some bitcode translation working (with a hard-coded address for my local php version). The address space stuff seems to work so far. * Missing files. * Other minor fixes * rtti-related fixes for gtest * minor travis config change * minor travis config change * sendfile no longer needed * Disable building vmill on non-linux platforms * Disable 32-bit test builds on macOS. * Trying to get symbol names on mac right. * Minor fix to save state code for tests, disable testing on macOS builds because of symbol mismatches in gtest. * Fixup remill-disass to use the simplified proto structure. Something may be wrong with remill-opt, or installing remill opt. Remove caching from travis.	2017-03-23 15:49:33 -04:00
Peter Goodman	1a2e4d11f9	Vmill backport (#81) ... * Backporting changes from vmill to remill. One set of changes is to remove DEF_ISEL_SEM, requiring that one use DEF_ISEL and DEF_SEM separately. This makes some simple instruction semantics more tedious to implement. The main goal is for ALL semantics themselves to be treated as private, non-exported symbols. In the case of those semantics defined via DEF_ISEL_SEM, those weren't private, and this inhibited inlining optimizations. Another big change is that function cloning has been rewritten and no longer depends on LLVM's function cloning implementation. Another big change is that the meat of the optimizer has been put into the remill/BC subdirectory and wrapped in a class, enabling it to be used independently of remill-opt. There are some new instruction semantics, and fixes to those. Some new utility functions for iterating over lifted blocks in the module. Attach/detach intrinsics have been removed, as they don't (yet) explicitly fit within the current model. The AssemblyWriter, which would produce a .S file and debug info linking lifted bitcode to the .S file via source line debug info has been removed. Despite it being really cool, the actual benefit is not obvious, given that optimization of the bitcode doesn't actually propagate enough of that information after inlining. * Eliminate sendfile dependency.	2017-01-05 12:22:10 -05:00
Peter Goodman	65f3812cce	Opt (#76) ... * Implements remill-opt * Change back to google namespace * ANother namespace change * Should fix test runner. * Try to fix deprecated use of fstat64 on macOS	2016-11-28 20:44:02 -05:00