Commit Graph

44 Commits

Author SHA1 Message Date
Peter Goodman b0f7790f43 Sparc support (#709)
* Adds sparc support; formats files

* Update the readme and walkthrough

* Minor tweaks to readme/walkthrough

* Update ci.yml
2020-11-05 12:59:20 -05:00
Peter Goodman c002b8505c Formats the code with clang-format and my additional stylistic mojojojo (#682) 2020-08-05 16:14:01 -04:00
Peter Goodman 2d55d02cc3 API improvements. Must be used with the api_improvements branches of Remill and Anvill (#666)
* API improvements. Must be used with the api_improvements branch of both Remill and McSema

fixes for x86 and running the lifted code with klee

* Update dockerfile to clone anvill

* update remill commit id

* Add python3 to dockerfile

* update python3

* disable abi script

* Updated cmake to find anvill

* Update main.cpp

* update find_package for anvill

* WIP:updated prebuild cfg

* update prebuild cfg files

* enable abi build for testsuite

* Fix memory leak

* install missing package for testcases

* frontend: Reflect cfg file changes in dyninst frontend.

* frontend: Update local files copyrights to reflect overall change to agplv3.

* fix failing testcases

* update test cfgs

* Fix test failure with local state pointer

* set the flag to use local state_ptr in default mode

Co-authored-by: kumarak <iit.akshay@gmail.com>
Co-authored-by: Lukas Korencik <xkorenc1@fi.muni.cz>
2020-07-23 02:14:56 -04:00
Aiethel 8ecfe165ef Various improvements (#628)
* bc: Inline calls to semantic functions and wrappers manually.

* bc: Inline calls to semantics and wrappers only if some flags are set.

* Refactor code that is responsible for abi libs loading.

* Wrap global pointers in RAII.

* Update .remill_commit_id

Co-authored-by: Peter Goodman <peter.goodman@gmail.com>
2020-02-10 13:20:05 -05:00
Peter Goodman 566976ac9b Issue 477 dse explicit args (#617)
* Devirtualization can only happen in read_only segment

* Disable inlining with explicit_args because of dse

Co-authored-by: Aiethel <korencikl@gmail.com>
2020-01-10 01:06:31 -05:00
Peter Goodman 46362c1f69 Function annotations (#619)
* Annotate lifted functions

* Annotate and Tie entrypoints

* Annotate external functions

* Annote _attach_call

* Update comments

* Simplify code that generated VerifyRegState

Co-authored-by: Aiethel <korencikl@gmail.com>
2020-01-08 17:35:35 -05:00
Peter Goodman e13d8ffbc9 Llvm 900 (#605)
* Support for LLVM 8.0.0 and 9.0.0

* remill script fixes

* Update to master
2019-11-18 21:53:23 -05:00
Aiethel bc0e132a65 Various fixes (#572)
* Correct constness when appending to global ctors

* Remove obsolete commit.

* Return type for native -> lifted is now pointer type size.

* Main returns i32
2019-06-01 21:02:45 -04:00
Peter Goodman 39a10d007c Minor fix to not recognizing gmon_start. (#480)
* Minor fix to not recognizing gmon_start.

* function wrapper name for native execution
2018-10-10 21:58:25 -04:00
Peter Goodman 69cb35cc16 Call early init lazy xrefs (#466)
* More broadly applies lazy xref initialization to all initializations where the xref size is less than teh value size of the xref itself (pointer size, usually). Moves initialization of lazy xrefs into a new function, __mcsema_early_init, which guards itself against multiple executions. Makes sure that __mcsema_constructor calls __mcsema_early_init first. Finally, makes sure that all native-to-lifted entrypoints call __mcsema_early_init as well, as we have observed cases where the lifted binary contains weak implementations of c++ standard library functions, and these functions are called by native libraries initialized before the call to __mcsema_constructor, thereby resulting in re-entrancy issues.

* Update Function.cpp

* Make sure the __mcsema_early_init guard is in the module.

* Make sure the __mcsema_early_init guard is in the module.

* Fixes for __gmon_start__ and stuff.

* Minor fix

* Adds some symbols

* Minor fix for ida7

* Try to resolve things like calls through PLT thunks via xref entry lookups when a flow cross-reference is missing.
2018-09-23 15:25:14 -07:00
Peter Goodman fd3bce8809 Issue 395 abi libs complex numbers (#436)
* -abi-libraries can work with function returning complex types

* -abi-libraries can now work with most functions taking complex numbers

* Code style improvements

* LLVM 3.5 compatibility

* long double complex numbers are now correctly passed to external function

* Code style changes, fix of float complex return for 64bit

* Older version of llvm allow pointer to be casted to type it is pointing to

* Add option to return integers in two registers if calling convention supports it.

* Byval attribute fix, small code style changes

* Floats are now correctly unpacked from xmm

* Add test for basic calls into complex.h

* Complex number test is now C-program

* Adding new prebuilt CFGs for unit tests

* WIP: Rework. Vector type is now one of the "simple" types. Easier code structure to support vector types in external libs

* Skeleton for vector passing

* WIP: Unpolished version of remake of constrant tables. Add constraint table also for return variables

* Code style changes, some small reworks.

* Incompatible return type now ends with error message instead of crash

* Code style

* Tables with information about registers used by different calling conventions are created only once.

* Changed severity of log, default for reg vectors are now xmm

* Win64 now uses only 4 registers to transfer arguments

* Code style improvements

* CallingConventionInfo now initialize ConstraintTable/s for specific gArch

* Remove kNoArgs since ConstraintTable can now use std::vector::size()

* Fixed iteration of ext_func arguments

* 32bit can now return i64 via two registers

* Code style
2018-08-30 16:12:31 -04:00
Aiethel 7a0e60d857 Issue 424 explicit args crash (#446)
* Using both --libc_ctor/dtor and --explicit_args no longer crashes

* --explicit_args now works with functions calling indirectly into lifted code

* Basic support for thread_local global stack and state

* Single function to insert call to __mcsema_verify_reg_state as first inst of block

* State is now zeroinitialized

* Extra comment

* Remove inline attribute from __detach_call_value

* No need for InsertVerifyFunction at beginning of ext_* functions
2018-08-20 10:40:39 -04:00
Peter Goodman 23800ebb8a Fstack protector (#434)
* Adds a new command-line option that adds a request for stack protection on lifted functions.

* Add the stack protector request to more locations

* Add gets to the libc abi.
2018-07-10 09:49:44 +10:00
Peter Goodman bda3af4c56 Dead store elim (#428)
* Update to use remill's dead store elimination, and be compatible with the indirection-removal done by Arch::PrepareModule.

* Fixes

* Update .remill_commit_id

* Update .remill_commit_id

* Update .remill_commit_id

* Update .remill_commit_id

* Update .remill_commit_id

* Update .remill_commit_id
2018-06-28 21:05:31 -04:00
kumarak a3483e4bf1 C++ exception handling support (#391)
* Initial changes to parse the eh_frame and exception table

* linearly scan the function to find missing blocks

* Update protobuf format for lp; need to tag instructions or basic blocks to associate it with the lp

* tag instructions with lp; removed debug logs

* updated comments and protobuf; use utility functions for pointer size;

* travis build failure; make lp_ea optional field

* Collect the exception handling bb chunks; Create a passthrough for the lifted landingpad;

* Generate the passthrough IR and attach the landing pad with function invoke

* Generate the eh_frame/exception table and add personality function with one raising the exception.

* catch/cleanup wrapper for the exception handling; Disable CFG simplification pass temporarily for invoke inst;

* invoke wrapper to fix stack and base pointers

* recover type indices of the exception type

* fix exception table offset

* Add the function definition for setting the rsp/rpb correctly after eh return;

* Changes for abi_library support in exception handling

* Remove unwind attribute from ext_* functions to generate eh table

* Support the lifting of multiple exception types thrown;

* Fix the multiple declaration of personality function in lifted bitcode;

* Add --recover_exception flag to lift the exception table

* Update the list of external functions with c++ ABI's; Fix basic block split issue for exception chunks;

* Weak linkage for external function causing the garbage value for out_of_range destructor. Temporarily disabled;

* cleanup redundant changes and resolve conflicts

* Update --std-defs list with the c++ exception ABIs

* Fix merge fail

* exception test binaries for integration test; Move to clang++ for recompiling the bitcode.

* Fix the exception handling support for llvm 3.5

* Disable the optimization passes for the function having landingpad;

* Temporary changes to add ABI libraries for exception tests.

* Remove the noreturn attribute to fix the problem with -03 optimization

* review comment changes

* Change wrapper function name across invoke;

* review changes

* Updated comments text

* Remove duplicate clauses from the list; Add arguments for personalityfn and fix review comments.

* Update the flag name to --exception_personalityfn;

* Update the flag name to --exception_personality_func;

* Follow the variable naming convention;

* Update comments

* travis build fix for unused variable

* Create the map of type indices from original binary. Runtime routine to fix the RDX with correct type index;

* Add _Unwind_Resume to --abi_libraries functions list;

* Update comments & code restructure

* Exception handling changes and utility functions for 32 bit;

* DWORD for 32-bit system
2018-04-12 14:05:45 -04:00
kumarak 1cc571b11f Support for --abi_library flag to call the external functions with explicit arguments. (#387)
* Add initial support for --abi_library flags

* support for abi_library

* Remove the weak linkage for __mcsema_debug_get_reg_state.

* Fix num of agruments for __cxa_allocate_exception

* Remove --library flag and cleanup

* Don't reload/overwrite mcsema::gModule after loading in the protobuf.

* Use PrepareModuleDataLayout to avoid checks for __remill_basic_block function.

* Bump the remill commit id to support --abi_library flag

* Fix travis build failure

* Initialize the test log filename;

* trim newline char from the eof
2018-03-22 11:09:13 -04:00
josh2059 5d1b7fb7be Making the stack start a bit further back reduces things like KLEE messing up (#373) 2018-01-16 14:14:48 -05:00
Peter Goodman 8a9856ada3 Klee maze example (#369)
* In progress. Working on an example of using KLEE on a Maze, but with the maze program being compiled to x86, amd64, and aarch64.

* Making lots of progress on getting lifting and runnning an aarch64 maze program on amd64, but using --explicit_args. The key thing I'm working through right now is a jump offset table, but where the offset is a block pc, rather than a table base. Also adding various bits of code here and there to making runnning with klee more directly doable, and working on a debugging facility to track down when the emulated program counter gets out of sync with the original program.

* Fixed a subtle @PAGE and @PAGEOFF-related reference bug on AArch64. Partially disabled the special jump offset table handling I had in table.py, as it doesn't (yet) handle the shifted table values. However, I still have the code there, so that it can recognize that a basic block address is used as a possible offset, so that I can remove the block address as a reference, which permits a new heuristic on the C++ side to work. On the C++ side, when there's a jump instruction that isn't associated with a cross-reference flow, I try to auto-augment it with addition switch cases, targeting blocks with no predecessors (as present in the CFG). This seems to work reasonably well.

* Improved the scripts and updated the READMEs.

* Minor rephrase

* Minor rephrase
2018-01-13 23:47:55 -05:00
Peter Goodman c5ceac27e8 Use 80-column lines (#352) 2017-12-11 14:32:19 -05:00
Peter Goodman 5134e5898f This commit adds support for the x86 stdcall calling convention to the ABI argument passing constraint table. Previously I must have assumed it would come in as the C calling convention. This commit also adds support for caller cleanup of arguments when using --explicit_args. (#334) 2017-11-20 17:11:22 -05:00
Alessandro Gario a8b85fdb8a Add support for the new cxx-common and llvm35 (#320)
* Add support for the new cxx-common and llvm35

This closes #317

* Code review changes

* Code review changes
2017-11-09 14:52:10 -05:00
Peter Goodman 71c7e939df LLVM 5.0 compatibility. 2017-09-19 15:13:24 -04:00
Scott Carr 3bd31da9c4 accumulate gep offset in value of width ptr size (#296) 2017-09-13 16:17:27 -04:00
Peter Goodman 8b465717e4 Made the pc annotation configurable, and auto-set when --legacy_mode is enabled 2017-08-23 12:49:01 -04:00
Peter Goodman 497d4dbe04 Pretty substantial changes. I removed --partition_segments. Now all idautils.Entries that are in the data sections are treated as segment split points. This is all to handle Apache, which exports its symbols to its shared libs (apr and apr-util). 2017-08-23 00:10:49 -04:00
Peter Goodman 872ad5efcf Minor fix for explicit args, where calling out to an external didn't fixup the stack pointer after the call. 2017-08-20 13:40:49 -04:00
Peter Goodman a45a1e9816 LLVM 3.6 compatability. 2017-08-19 20:26:49 -04:00
Peter Goodman 04e675ac48 Remove tail calls, fast calling convention, and fixup get_cfg.py to better support segmenting existing segments using global variables 2017-08-18 22:41:49 -04:00
Peter Goodman df1678412b Adds support for --legacy_mode, where drivers are added for the entrypoint and the callbacks 2017-08-18 15:09:25 -04:00
Peter Goodman 18abc11714 Added new --legacy_mode operation that tries to produce bitcode that is more similar to old bitcode. Also updates get_cfg.py to disassemble in such a way that mcsema2 will be able to support global variable recovery as though they were distinct segments 2017-08-16 23:06:53 -04:00
Peter Goodman d908fa4128 Minor fix to the ELF64 runtime where RDI is clobbered in one spot. 2017-08-15 15:10:19 -04:00
Peter Goodman e4d87c7154 Updates to some API changes in remill. 2017-08-04 15:27:13 -04:00
Peter Goodman b6ab3f9c43 Swapped order of args to the lifted subroutines. Updated the callback assembly stubs to use an inline assembly feature I was previously unaware of. That inline asm change fixes one issue related to using llvm 3.6, which is that the callback functions would be compiled without their tail calls, and that would cause a mess of issues. 2017-07-19 17:35:07 -04:00
Peter Goodman ebfdb22f44 Improvements to building with gcc. Updates readme to use new build script. Updated the 64-bit elf runtime to work with GNU as, which doesn't support all the same kinds of relocation arithmetic as llvm-as does. 2017-07-19 12:50:52 -04:00
Peter Goodman 6ced3bd807 Transitioned to using a table-based approach for explicit arguments and return values. 2017-07-14 15:30:18 -04:00
Peter Goodman 834adb7991 More progress on calling convention stuff. Moved some existing code around. Added implementations for the runtime argument getting stuff. Added functions for setting the return value. Not well tested. (#267) 2017-07-10 11:57:10 -04:00
car bauer 82d70afe5c Explicit args [WIP] (#265)
* explicit args to external APIs
* Make AArch64 disassembly detect refs to external functions
* Fix awful bug where code would become undefined since our callinst was not same convention as the called function
* Emit enough code so that AArch64 bitcode gets emitted
2017-06-29 13:34:51 -04:00
Peter Goodman c1bd1668f4 Minor fixes here and there. May not yet build! 2017-05-25 11:00:02 -04:00
AkshayK 36914c64a4 Minor changes in mcsema branch to avoid compile-time error due to new arch name; The major changes will go into remill; revisit mcsema-lift after having basic support for ARM instructions set; 2017-05-06 19:53:03 -04:00
Peter Goodman b5ef96fabe Lifting and recompiling /bin/ls now works. Yay 2017-05-02 18:28:46 -04:00
Peter Goodman 939f408d69 Lifting and recompiling a simple hello world program is working. This uses a new runtime that does more direct register conversion between lifted and native. Updated the .gdbinit debugging script to work using the remill State structure. 2017-05-01 22:51:41 -04:00
Peter Goodman 5e508962f8 Fixes for lifting. Simple lifted binaries can now be recompiled when using the alessandro/refactor/llvmmultiversion-cmake branch of remill as a base. 2017-04-29 22:06:24 -04:00
Peter Goodman 0c8f2ec21f Minor updates. Trying to diagnose an issue where llvm optimizes a ton of stuff into unreachable insts. 2017-04-27 21:04:00 -04:00
Peter Goodman 3ca5995e83 Working toward bringing up the mcsema runtime. Need to add --entrypoint support to get_cfg.py, so that something like main can be specified as the key entrypoint. Right now it's pulling in things like init, start, etc. 2017-04-25 11:45:32 -04:00