209 Commits

Author SHA1 Message Date
Peter Goodman 65f3812cce Opt (#76)
* Implements remill-opt

* Change back to google namespace

* ANother namespace change

* Should fix test runner.

* Try to fix deprecated use of fstat64 on macOS
2016-11-28 20:44:02 -05:00
JP Smith 708f8fd4a0 OSX gtest debugging (#74)
* move gtest headers around

* start debugging travis' weird gtest issues

* fix directory

* put libraries where they should be

* skip testing on OS X
2016-11-22 11:37:50 -05:00
JP Smith b533f750e4 move gtest headers around (#69) 2016-11-08 10:35:33 -05:00
JP Smith c8e0c37710 OS X build (#61)
* Start adding OS X compatibility

* wrong unzip command

* update README

* first shot at build.sh

* clean up CMakeLists

* unnecessary CMake flag

* build.sh finally works (on my machine)

* start work on travis integration with new build.sh

* fix directory issues

* fix script name typo

* fix unnecessary brew stuff for os x

* build.sh should work on travis now

* make tests not fail on apple

* forgot to add file to fix mac/macos confusion
2016-11-04 10:35:41 -04:00
Peter Goodman 284ef26018 Issue 50 ida get cfg (#67)
* Starting work on Issue #50, implementing an IDA Python script for producing a CFG file. The approach taken is to fill in a generic data structure, and some other code will deal with the production of the CFG structure itself. This will enable moving beyond the protobuf format. The script can be used via remill--disass. Eventually Binary Ninja will support a batch mode, and I will work it into remill-disass when that happens.

* remill-disass can now be installed and executed directly. There were some issues with the PYTHONPATH but I think those are resolved.

* Added in command-line flag to change the log level.

* Added conditions to prevent non-termination, especially in the presence of noreturn calls.

* Try to fix travis

* remill-disass seems to work reasonably well now.
2016-11-01 18:00:42 -04:00
Peter Goodman 9a5f684dd3 Attempted rebase against master with a bit of restructuring of the MMX semantics
Issue 44 mmx support (#58)

* Should fix Issue #56. Used -isystem instead of -I, as it's an include of <cstdint> instead of "stdint.h". @erhlee-bird does this work on your system?

* Minor changes

* Remove old 'test' function.

* Start to MMX PUNPCK instructions.

* Initial example for a PUNPCK instruction.

* Cleanups to Operators.h. Specifically, to the signed versions of vector operations. They now more strictly enforce type safety. This caught a few bugs. Operators.h is arguable simpler as a result.

* MMX Instruction support(PADD/PSUB); Fix for handling signed operations;

* Should fix Issue #56. Used -isystem instead of -I, as it's an include of <cstdint> instead of "stdint.h". @erhlee-bird does this work on your system?

* Minor changes

* Cleanups to Operators.h. Specifically, to the signed versions of vector operations. They now more strictly enforce type safety. This caught a few bugs. Operators.h is arguable simpler as a result.

* merged conflict;

* MMX Instructions (PAVG/PHSUB/PMULHRS)

* Testcases for PADD/PSUB; Fix for signed/unsigned saturation;

* Test cases for PAVG/PABS/PACKSS; Fixes for MMX instructions handling;

* Fixed PACKSSWB instruction handling;

* PHADD/PHADDSW Instruction;

* PMAXSW/PMAXUB/PMINSW/PMINUB Instructions

* PCMP/PUNPCK/PMUL Instructions;

* Corrected space indentation;

* Fixed build issue;

* reimplemntation of instructions with saturation arithmetic

Minor syntax fixes. Added beginning of an implementation for EMMS.

Issue 44 mmx support (#65)

* Remove old 'test' function.

* Start to MMX PUNPCK instructions.

* Initial example for a PUNPCK instruction.

* MMX Instruction support(PADD/PSUB); Fix for handling signed operations;

* Should fix Issue #56. Used -isystem instead of -I, as it's an include of <cstdint> instead of "stdint.h". @erhlee-bird does this work on your system?

* Minor changes

* Cleanups to Operators.h. Specifically, to the signed versions of vector operations. They now more strictly enforce type safety. This caught a few bugs. Operators.h is arguable simpler as a result.

* merged conflict;

* MMX Instructions (PAVG/PHSUB/PMULHRS)

* Testcases for PADD/PSUB; Fix for signed/unsigned saturation;

* Test cases for PAVG/PABS/PACKSS; Fixes for MMX instructions handling;

* Fixed PACKSSWB instruction handling;

* PHADD/PHADDSW Instruction;

* PMAXSW/PMAXUB/PMINSW/PMINUB Instructions

* PCMP/PUNPCK/PMUL Instructions;

* Corrected space indentation;

* Fixed build issue;

* reimplemntation of instructions with saturation arithmetic

* PSLL/PSRL/PMUL instructions;

* Add in some missing arg parsing to Run.cpp. Set the minimum default log level in the test runner so that Travis logs don't explode so much, causing Travis to cancel an otherwise 'good' job.

* Update README.md (#62)

Fixed install path for install_gtest.sh

* Fixed failing testcases for PSLL/PSRA/PSRL/PACKSS

* Cleanup and resolved TODO comments;

* Added new testcases for MMX instructions;

* Fixes Issues #52 and #53. Addresses first comment of Issue #45.

* MIssing file

* Renaming test cases

* In progress changes. Moduler verifier is complaining about lack of \!dbg location entries. Adding a SourceWriter to produce something semi-useful.

* Linking lifted instruction code to printed out assembly code now works

* Added new instruction and testcase for CVTPI2PS. Made general improvements:
- made it so that disassembled instructions can be printed to a file, and linked back to the bitcode via debug info.
- eliminated the unnecessary _read and _write suffixes for registers in __remill_basic_block.

Mostly cosmetic changes. For example, limiting line length, putting spaces before the opening parens of loops, etc. Renamed all the semantics files to be cpp source files instead of header files. That stops cpplint from complaining about anonymous namespaces in headers ;-)
2016-10-18 17:23:57 -04:00
kumarak b0055b6503 Issue 44 mmx support (#58)
* Should fix Issue #56. Used -isystem instead of -I, as it's an include of <cstdint> instead of "stdint.h". @erhlee-bird does this work on your system?

* Minor changes

* Remove old 'test' function.

* Start to MMX PUNPCK instructions.

* Initial example for a PUNPCK instruction.

* Cleanups to Operators.h. Specifically, to the signed versions of vector operations. They now more strictly enforce type safety. This caught a few bugs. Operators.h is arguable simpler as a result.

* MMX Instruction support(PADD/PSUB); Fix for handling signed operations;

* Should fix Issue #56. Used -isystem instead of -I, as it's an include of <cstdint> instead of "stdint.h". @erhlee-bird does this work on your system?

* Minor changes

* Cleanups to Operators.h. Specifically, to the signed versions of vector operations. They now more strictly enforce type safety. This caught a few bugs. Operators.h is arguable simpler as a result.

* merged conflict;

* MMX Instructions (PAVG/PHSUB/PMULHRS)

* Testcases for PADD/PSUB; Fix for signed/unsigned saturation;

* Test cases for PAVG/PABS/PACKSS; Fixes for MMX instructions handling;

* Fixed PACKSSWB instruction handling;

* PHADD/PHADDSW Instruction;

* PMAXSW/PMAXUB/PMINSW/PMINUB Instructions

* PCMP/PUNPCK/PMUL Instructions;

* Corrected space indentation;

* Fixed build issue;

* reimplemntation of instructions with saturation arithmetic
2016-10-04 11:06:47 -04:00
Peter Goodman 839f6cb882 Remove old build scripts. 2016-10-01 19:12:08 -04:00
Peter Goodman 08f5384a94 Issue 51 third party code (#60)
* Working on factoring out external dependencies

* Making progress

* CMake for remill-lift seems to work :-)

* Improving build process and dependency list. Using a blank ubuntu 14.04 vm for testing.

* Still fighting with various dependencies.

* Making more progress with CMake stuff! Now have the semantics get compiled with cmake, and support installing stuff. I have also attempted to auto-find the semantics files when --bc_in is not provided.

* Adding protobuf compilation back to the steps.

* Improving readme

* Improving readme

* Switch to using the  command.

* Updated travis. Updated README to include new build badge icons, to change where the ldconfig step goes, and to add a CMake notice for ubuntu trusty.

* Added script to generate tests. Added auto-yes in apt-add-repository commands.

* Minor fix

* Enable auto-finding of --bc_in based on --arch_in.

* Adding more test generation and compilation. Unfortunately compiling tests isn't working due to llvm::CloneFunctionInto not quite working anymore.

* llvm::CloneIntoFunction seems to work, which may resolve Issue #55. I'll need to double check with the official release version of LLVM; right now I'm using a custom debug build to try to catch issues. Added code to build and install gtest. Directly referenced which LLVM libraries must be linked. Added backup paths for deducing the semantics bitcode files based on the input arch. Made remill-lift report simple flag errors via stderr instead of using glog to abort the program. Fixed an issue where, when I added the various arch versions, xed was decoding avx-enabled instructions as 64-bit because the comparison condition deciding which decode state to use was too narrow.

* Tests generation and running scripts now work for me. Gotta test in fresh vms now.

* Addresses Issue #55. The problem was that the release version of clang 3.9 isn't guaranteed to preserve c++ variables name in the bitcode. I've added some stuff to re-introduce the variable names based on debug info.

* Build the test runner with rtti so that gtest doesn't complain on some versions of ubuntu.

* Fixing up Travis build stuff

* I am dumb; forgot to add the os-generic install file for travis to pick up.

* Change the trusty build type to generic, otherwise Travis uses a YUGE vm with all the things.

* Disable aptitude upgrading. dhclient is getting in the way.

* Re-order the linux-specific travis install script. Hope that protocol buffers gets installed correctly this time. Remove test generation and running from end of install script -- it doesn't belong there.

* Try to purge dhcp client, then add it back in, then do the normal update + upgrade

* Something is very dumb about Travi CI's generic linux build environment. It seems to be having trouble with the protobuf-compiler package -- that or somehting is going wrong just before that package is installed.

* Use the cpp build language -- I think this defaults to that mega vm though. Print out better OS information at the beginning of the travis install. Individually install needed packages, as opposed to doing it all at once.

* OK, using cpp language got me further, but builds take much longer. I was still missing cstdint, though. This time I'll try building with the generic trusty travis image, and with build-essential and the g++ 4.8 stdlib dev files installed.

* Lets try using the libstdc++ dev files

* Add in the x32 versions of the dev files. Hopefully that'll resolve the missing cstdint for the 32-bit builds of some of the instruction semantics bitcode files.

* Hopefully these are right!

* Install g++ multilib package.

* Simplifications to installer, and also to the README to now use g++-multilib.
2016-10-01 19:10:37 -04:00
Peter Goodman 6178c83166 Merge pull request #57 from trailofbits/issue_56_missing_cstdint
Should fix Issue #56. Used -isystem instead of -I, as it's an include…
2016-09-22 19:17:48 -04:00
Peter Goodman d07d065fb6 Enabled more aggressive compiler warnings. This caught one bug in COND_BR.h. Added some pragmas into the types and state structure related files to make sure that extra padding added to structures is reported as a fatal error. The rest of the changes are downcasts from 64-bit values to 32-bit values. 2016-09-21 14:05:07 -04:00
Peter Goodman 50d64ef5a1 Should fix Issue #56. Used -isystem instead of -I, as it's an include of <cstdint> instead of "stdint.h". @erhlee-bird does this work on your system? 2016-09-21 11:08:54 -04:00
Peter Goodman 3d74805dab Switch to ncurses 2016-09-07 17:39:57 -04:00
Peter Goodman da643fa3cb Merge pull request #39 from rgov/mac
Fix build on macOS
2016-09-07 11:57:28 -04:00
Peter Goodman 7c60cd3860 Build changes 2016-09-07 11:35:11 -04:00
Peter Goodman d755a870cf Added potential support for openSUSE builds 2016-09-03 16:06:41 -04:00
Peter Goodman f283e05616 Added semantics and tests for CVTDQ2PD and CVTTPD2DQ 2016-09-01 20:57:39 -04:00
Peter Goodman 773919bd40 Added NOPs. 2016-08-31 20:46:02 -04:00
Peter Goodman c7d9122e26 Updated binja script for simpler cfg protobuf format. Fixed X86/Arch.cpp to record x87 ST(n) regs as 64 bits. Updated llvm version to 3.8.1. Updated bootstrap to choose the release of the installed ubuntu version (mine is now 16.04, previously 14.04). 2016-08-31 17:47:31 -04:00
Peter Goodman 5eef7ead59 Re-introducing cfg::Instr. Marked extern funcs as not builtin so that they won't conflict with builtins needed to implement fpu instructions. Updated the address translation to have the scale be signed, and to implement the displacement as an add for positive numbers, and a subtract for negative numbers. 2016-08-29 12:47:28 -04:00
Peter Goodman 1b012f75f3 Added tests for MOVQ. Starting implementation of FPU instructions and tests. Change how instruction lifting is done. The arch side of lifting is now separate from all bitcode concerns. remill/BC/Translator.cpp is now heavier weight in what it does. 2016-08-26 15:59:52 -04:00
Kareem El-Faramawi d9ede09aa7 Add function references as indirect blocks 2016-08-08 17:08:47 -04:00
Kareem El-Faramawi 5380487bde Add jump table entries as indirects 2016-08-08 17:08:47 -04:00
Kareem El-Faramawi c08ca9f50f Clean up how indirects are added 2016-08-08 17:08:47 -04:00
Kareem El-Faramawi 2e01103ded Process instructions, create indirect blocks at terminators 2016-08-08 17:08:47 -04:00
Kareem El-Faramawi 497eac715e Process basic blocks 2016-08-08 17:08:47 -04:00
Kareem El-Faramawi 8740835f78 Use update_analysis_and_wait 2016-08-08 17:08:47 -04:00
Kareem El-Faramawi 3e46dc0498 Add basic function processing 2016-08-08 17:08:47 -04:00
Kareem El-Faramawi d6c94946e1 Add arguments 2016-08-08 17:08:47 -04:00
Kareem El-Faramawi 17da66e732 Create binja_get_cfg.py 2016-08-08 17:08:47 -04:00
Peter Goodman ba427a3b4b Starting to add some more FPU stuff, though untested. Decided on a simpler way of handling conversions between float80_ts and float64_ts. The idea is to just punt on float80_ts, with the idea that all sources of them will come via memory locations, or at attach/detach time. 2016-08-08 17:05:44 -04:00
Peter Goodman 81df2663d9 Changes to how conditional branches are handled. Made the x86 translator code even less OO. Removed the conditional_branch and create_program_counter intrinsics. 2016-08-08 17:05:44 -04:00
Peter Goodman 4a3307bc6c Got all BITBYTE instructions and tests working. 2016-08-08 17:05:43 -04:00
Peter Goodman 8f8ed65af1 DATAXFER and BINARY tests are now passing. 2016-08-08 17:05:43 -04:00
Peter Goodman a7c2d11ce5 In-progress, doesn't build, but shows where things are going and I wanted to back my changes up before my vacation. 2016-08-08 17:05:43 -04:00
Ryan Govostes 9b4759a3d2 Set a default SDKROOT on Darwin 2016-07-28 21:55:32 -07:00
Ryan Govostes 85f568f072 Fix test for existince of libxed on Darwin 2016-07-28 21:52:03 -07:00
Ryan Govostes 0e249223cb Don't run install_name_tool on libxed.a 2016-07-28 21:40:39 -07:00
Kareem El-Faramawi b61450fe53 Add function references as indirect blocks 2016-07-05 11:19:44 -04:00
Kareem El-Faramawi b62756dd6f Add jump table entries as indirects 2016-07-05 11:07:19 -04:00
Kareem El-Faramawi e69c448d69 Clean up how indirects are added 2016-07-05 10:51:43 -04:00
Kareem El-Faramawi 2810edb565 Process instructions, create indirect blocks at terminators 2016-06-30 15:13:27 -04:00
Kareem El-Faramawi 141c5566b1 Process basic blocks 2016-06-30 14:26:43 -04:00
Kareem El-Faramawi 0e9b69ede0 Use update_analysis_and_wait 2016-06-30 13:54:08 -04:00
Kareem El-Faramawi d21fbd28b4 Add basic function processing 2016-06-29 18:18:42 -04:00
Kareem El-Faramawi 002dd06981 Add arguments 2016-06-29 14:54:51 -04:00
Kareem El-Faramawi 614973b68d Create binja_get_cfg.py 2016-06-28 13:55:57 -04:00
yan 3f815efcbd Rename all references from mcsema2 to remill 2016-06-23 11:39:36 -04:00
Peter Goodman 10b5af884d Fix to build system for dry runs. 2016-04-22 15:48:47 -04:00
Peter Goodman fb5ac1bd6c Very early FPU support. Changes to libOptimize. Added a libFinalize and a scripts/finalize_bitcode.sh to get around the issue of LLVM and dead arg elimination. This is annoying because I had to use the Naked attribute on all functions, but then this affects codegen, so the finalize step is needed to make sure that code is correctly compiled. 2016-04-08 16:35:48 -04:00