First step towards transitioning away from cxx-common's `pkgman.py` dependency management system towards compatibility with and use of vcpkg (https://github.com/microsoft/vcpkg) to manage dependencies.
This commit attempts to support both the new and old build systems until we can phase out the old completely.
Please see the updated trailofbits/cxx-common repository or run the `./scripts/build.sh` script to fetch the required dependencies.
* Refactors the code to improve directory layout, use more std::string_view in place of std::string, deletes some deprecated functions, deprecates some other functions for eventual deletion, and includes semantics for sparcv8 (sparc32) and sparcv9 (sparc64)
* Update new dir layout with llvm 11 support
* Whoops missing files
* Drop llvm 800 from workflow
* Minor fix
* Move where the install directives are in CMake
* Minor fixes
* Rename tools/ to bin/.
* Minor tweaks
* Should fix issues
* Append project name to the check_git target to fix compound builds (#446)
* Change the way we get registers for __remill_basic_block so that it's the Arch class of the target that actually populates the function, and isn't based on reverse-engineering what's visible in the function as compiled by llvm
* Renamed a cmake var to reduce likelihood of conflicts, and tries to make sure remill only links against static llvm libraries
* Fix instruction dominance issues
* Minor fix on the type of PC on x86
* Added new implementation (in Arch.cpp) of the AArch64 __remill_basic_block function
Co-authored-by: Artem Dinaburg <artem@trailofbits.com>
Co-authored-by: Eric Kilmer <eric.d.kilmer@gmail.com>
* Emit Version Information
* Cut new versions in CI on passing master builds
* Emit detailed version information via --version
* add VERSION to gitignore
* Running clang-format on files with some additional custom scripts for my style
* Fix missing unique_ptr in remill/BC/Optimizer.h
* Fixes and selective disabling of clang-format
* General API improvements
* Minor changes
* New name for existing isel
* Error compatibility with llvm8
* llvm 10 to build script
Co-authored-by: kumarak <iit.akshay@gmail.com>
* Support for llvm 8.0.0 and llvm 9.0.0. Thanks to Peter Garba and John Regehr.
* Fixes for macOS to bake the SDK root into the config for the bc compiler
* Script fixes
* Update travis.sh
* Update travis.sh
* Windows: Rename the internal getpid to avoid conflicts with the ucrt
* Scripts: Remove the LLVM toolchain integration script
The mcsema README has been updated with the new instructions. It is
better to always download the extension as it changes at each update.
Additionally, the installation folder depends on what the user is
using (full Visual Studio vs just the build tools)
* CI/Travis: Add Windows support
* Add support for llvm 8.0 to the build script
Also adds a new compiler option, `--use-host-compiler`, which looks for `$CC` and `$CXX` in place of using our build of Clang. In the case of LLVM < 5.0, using the host compiler is the default behaviour.
* Update DeadStoreEliminator.cpp
* Disables build failures when clang is not the specified compiler. Really the core requirement is that clang is the bitcode compiler.
* Update travis.sh
* CMake: Make the bitcode generator Visual Studio friendly
* CMake: Always recompile the runtimes when the semantics change
* CMake: Style changes
* CMake: Fix missing variable when re-configuring
* CMake: Windows/Visual Studio fixes
* Windows: Add missing FPU macros
* Fix alignment issues with Visual Studio
* CMake: Fix the install target for Windows
* CMake: Add support for find_package(remill)
* Build script: Use verbose makefile
* CMake: Add missing ADDRESS_SIZE_BITS in BC compiler
* CMake: X86 Tests project refactor
* Windows: Port of BC/Utils and OS/FileSystem
* CMake: Update settings.cmake
* CMake: Generate the semantics path in remillConfig.cmake
* CMake: Only use C++14 when compiling on Windows
* Rename FPUFlags to Float and collect all FPU flags there
* Travis: Add macOS, test all LLVM versions, add clang static analyzer
* Travis: Disable LLVM50 tests, include verbose output for failed tests
* CMake: Update the find_package handler, add support for fcd headers
* Dead store elimination (draft) (#257)
* Add initial prototying of DeadStoreEliminator
* Improve upon DeadStoreEliminator prototype
Update `DeadStoreEliminator` prototyping with additional comments and
proper function prototype return values.
Fix a small warning in `Lifter.h` by adding `llvm::Value` and
`llvm::BasicBlock`.
* Add initial code for DeadStoreEliminator
Provide namespaced `remill::StateSlots` function which visits fields of
the module's state struct and returns `StateSlot` objects.
Remove `comment` field as the `State` object (an `llvm::StructType`)
does not track names in a useful way.
* Update state analyzer and fix type errors
Update `DeadStoreEliminator.cpp` to properly produce a vector of
`StateSlot`s without type errors (i.e. it compiles).
Code still needed for other container types besides structs.
* Add remill::VisitSequential for arrays and vectors
Add state analyzer code for LLVM's `ArrayType` and `VectorType`.
Add debugging use of `llvm::Type::dump()`.
Consider refactoring as a `StateVisitor` class.
* Begin refactor of Visit funcs to StateVisitor
* Fix segfault in StateVisitor::visit, code style
* Prototype ForwardAliasVisitor for alias analysis
Add code for ForwardAliasVisitor subclass of `llvm::InstVisitor`,
to be used for performing alias analysis of lifted functions.
* Add non-working visit functions for alias analysis
* Fix compilation of alias analysis visit funcs
* Change ForwardAliasVisitor to RetTy=bool
Update ForwardAliasVisitor to use booleans for return types to track
when we should add the instruction to the next_wl.
Add code to simplify StateSlots for vecs of ints.
* Add progress tracking to AliasAnalysis
* Correct progress tracker, use BasicBlockFunction()
* Change ForwardAliasAnalysis<RetTy = AliasResult>
Get some of those sweet sweet enums in there!
* Add FAV state pointer field, PHINode impl
* Allow non-const add and sub in FAV
Allow add and sub instructions with two pointers in the offset map.
Add to implementation of visitPHINode.
* Update PHINode impl in FAV
* Clean up use of StateSlots to create AAMDNodes
Add code to create AAMDNodes from StateSlot elements.
Change creation of StateSlot vector to have elements for every
byte offset of the state structure for fast indexing.
* Complete addition of AAMDNodes for load and store
Move function defs up for AAMDNode ops.
Add AliasMap typedef.
Finish generateAAMDNodesFromSlots.
* Add GenerateLiveSet func
* Initialize live set, add AAMDNodes to stores
* Change LiveSet creation to a block visitor class
* Add to_remove set to LSBV
* Update build script to use os-release for OS detection
* Update DSE code to conform to pag's review
* Refactor LiveSetBlockVisitor to one LiveSet per block
* Update VisitBlock to better check instruction type
* Fix VisitBlock CallInst and InvokeInst cases
* Stack allocate AAMDInfo in AnalyzeAliases
* Add remove pass option for VisitBlock
* Add DOT digraph generation
* Fix bugs in dot digraph
* Fix various bugs in AAMDNodes and LSBV
Move AAMDNodes functions later to match their use.
Clean up code conventions.
Fix small bugs in various spots in the code.
* Fix various function prototypes, overflow checks
* Merge AliasMap and OffsetMap
* Change add/sub insts to be safer
Add OpType enum class to replace use of `plus` bool.
Add more straightforward bounds checking on AddInst or SubInst values.
* Refactor GetUnsignedOffset style
* Fix illegal instruction error
* Add offset checking for GEP, provide log messages
Write a log message for the cases where `GetUnsignedOffset` returns
false (indicating an overflow or underflow).
* Fix APInt initialization in VisitGEP
* Re-add dead store elimination and alias map
* Move LSBV into alias analysis, expand callinst
Expand definition of cases where a CallInst should be considered to
touch the state struct or otherwise revive a slot.
* Improve DOT creation, fix errors for callinsts
Fix small errors in LiveSetBlockVisitor::CallAccessesState.
Clean up DOT digraph generation further.
* Clean up code per @pag's comments
* Clean up code, add selectinst, fix compile errors
Deal with a few small corner cases with FAV Load and Store instructions,
add SelectInst visitor.
Inline MarkLiveArgs to avoid the complications of C++ generics.
* Clean up logging
* More changes to please Peter
Clean up select and PHI node cases to improve circular dependency
handling.
* Correct errors in visitSelect
* Add load forwarding code
* Add code to run FBV
* Fix map usage error in FBV
* Add call, invoke cases for FBV
* Here are the changes whoops
* Minor API change
* Minor tweaks to DOT digraph printing, as well as attempts to handle the case where the forward analysis pass is incomplete. Still don't have it guarantee completion on everything, but seems 'good enough' for now.
* Change log level
* Fix compile errors due to messy merge
* Pag dead store (#262)
* Fix to script calling wrong function.
* Makes sure that value names are preserved (#249)
* Adds LLVM_VERSION() to accomodate llvm::LLVMContext::setDiscardValueNames() in <3.9 (#250)
* Makes sure that value names are preserved
* Adds LLVM_VERSION() macros to accomodate
* Update README.md
* Update README.md
* set state and memory as noalias (#254)
* Implements some ring 0 instructions in terms of hyper calls and new I/O port intrinsics. (#252)
* Random tests.
* More decode error info
* more playing around
* more system instructions. instrinsics for accessing I/O ports. Split our writes to individual control regs for better identification via hyper calls.
* CR8 read/write support (#255)
* Check argument index of function (#256)
* Fix NoAlias Attributes for older LLVM versions
* Fix for LLVM 4.0 and 3.9
* Fix typo
* Follow remill coding style
* Here are the changes whoops
* Minor API change
* Minor tweaks to DOT digraph printing, as well as attempts to handle the case where the forward analysis pass is incomplete. Still don't have it guarantee completion on everything, but seems 'good enough' for now.
* Change log level
* Fix compile errors due to messy merge
* Improve call/invoke case of FBV
* Improve call/invoke case of FBV
* Removes a level of indirection in the __remill_basic_block function
* Create dedicated stats tracker
Plus clean up the FBV visitor a teensy bit more.
* Move call/invoke LiveSet gen to static func
This commit is to pave the way for future improvements to where this
information is calculated (in FAV instead of LSBV).
* Add code to FAV for calls and invokes
Move call and invoke arg-based livesets to FAV to allow for module-level
LSBV code.
* Begin change of LSBV to module-level
* Minor bug fixes
* Remove some dead code
* Begin adding more code for call/invoke LSBV
* Add entry block checks for LSBV call/invoke
* Info about register names, as well as printing them in the digraphs.
* Make DOT printing only happen per function, as opposed to printing every function per DOT file. Minor tweaks to interprocedural analysis.
* Bug fixes and DOT printing improvements.
* Bug fixes and DOT printing improvements.
* Also print out DOT digraphs of functions after removing stuff
* Tried to make it treat everything before a call to __remill_error as dead but that didn't work out.
* Use datalayout and type sizes to handle the number of elements in a sequential type for LLVM 3.8 compatibility, also check for pointer type.
* More stats, hopefully fixes a bitcast issue.
* Minor bug fixes, and more DOT printing to help diagnose when the offset analysis terminates but there is still stuff in the work list.
* Some possible bug fixes
* Minor bug fix
* Travis build fixes for earlier compatibility
* Fix for LLVM less than 3.8 compatibility.
* LLVM 3.5 compatibility
* Missing condition in forwarding code that does casting.
* LLVM 5.0 compatibility
* Add LLVM 6.0 to build.sh. Change default install location to /usr/local
* Add back in vmill. Add some extra flags to the building runtimes.
* Maybe works
* Disable dse across indirect call (#267)
* Make DSE sensitive to indirect function calls
* FE_DENORM issue
* Added some compat code that implements the futimens syscall on mac os 10.12 for travis support.
* Disable dse across indirect call (#268)
* Make DSE sensitive to indirect function calls
* FE_DENORM issue
* Added some compat code that implements the futimens syscall on mac os 10.12 for travis support.
* Playing with CACHE and PARENT_SCOPE
* Minor stack address size check
* Fix to DSE I think.
* Minor DSE tweak
* Update Run.cpp
* Update travis.sh
Adds in 32-bit libraries for Linux builds.
* Add initial prototying of DeadStoreEliminator
* Improve upon DeadStoreEliminator prototype
Update `DeadStoreEliminator` prototyping with additional comments and
proper function prototype return values.
Fix a small warning in `Lifter.h` by adding `llvm::Value` and
`llvm::BasicBlock`.
* Add initial code for DeadStoreEliminator
Provide namespaced `remill::StateSlots` function which visits fields of
the module's state struct and returns `StateSlot` objects.
Remove `comment` field as the `State` object (an `llvm::StructType`)
does not track names in a useful way.
* Update state analyzer and fix type errors
Update `DeadStoreEliminator.cpp` to properly produce a vector of
`StateSlot`s without type errors (i.e. it compiles).
Code still needed for other container types besides structs.
* Add remill::VisitSequential for arrays and vectors
Add state analyzer code for LLVM's `ArrayType` and `VectorType`.
Add debugging use of `llvm::Type::dump()`.
Consider refactoring as a `StateVisitor` class.
* Begin refactor of Visit funcs to StateVisitor
* Fix segfault in StateVisitor::visit, code style
* Prototype ForwardAliasVisitor for alias analysis
Add code for ForwardAliasVisitor subclass of `llvm::InstVisitor`,
to be used for performing alias analysis of lifted functions.
* Add non-working visit functions for alias analysis
* Fix compilation of alias analysis visit funcs
* Change ForwardAliasVisitor to RetTy=bool
Update ForwardAliasVisitor to use booleans for return types to track
when we should add the instruction to the next_wl.
Add code to simplify StateSlots for vecs of ints.
* Add progress tracking to AliasAnalysis
* Correct progress tracker, use BasicBlockFunction()
* Change ForwardAliasAnalysis<RetTy = AliasResult>
Get some of those sweet sweet enums in there!
* Add FAV state pointer field, PHINode impl
* Allow non-const add and sub in FAV
Allow add and sub instructions with two pointers in the offset map.
Add to implementation of visitPHINode.
* Update PHINode impl in FAV
* Clean up use of StateSlots to create AAMDNodes
Add code to create AAMDNodes from StateSlot elements.
Change creation of StateSlot vector to have elements for every
byte offset of the state structure for fast indexing.
* Complete addition of AAMDNodes for load and store
Move function defs up for AAMDNode ops.
Add AliasMap typedef.
Finish generateAAMDNodesFromSlots.
* Add GenerateLiveSet func
* Initialize live set, add AAMDNodes to stores
* Change LiveSet creation to a block visitor class
* Add to_remove set to LSBV
* Update build script to use os-release for OS detection
* Update DSE code to conform to pag's review
* Refactor LiveSetBlockVisitor to one LiveSet per block
* Update VisitBlock to better check instruction type
* Fix VisitBlock CallInst and InvokeInst cases
* Stack allocate AAMDInfo in AnalyzeAliases
* Add remove pass option for VisitBlock
* Add DOT digraph generation
* Fix bugs in dot digraph
* Fix various bugs in AAMDNodes and LSBV
Move AAMDNodes functions later to match their use.
Clean up code conventions.
Fix small bugs in various spots in the code.
* Fix various function prototypes, overflow checks
* Merge AliasMap and OffsetMap
* Change add/sub insts to be safer
Add OpType enum class to replace use of `plus` bool.
Add more straightforward bounds checking on AddInst or SubInst values.
* Refactor GetUnsignedOffset style
* Fix illegal instruction error
* Add offset checking for GEP, provide log messages
Write a log message for the cases where `GetUnsignedOffset` returns
false (indicating an overflow or underflow).
* Fix APInt initialization in VisitGEP
* Re-add dead store elimination and alias map
* Move LSBV into alias analysis, expand callinst
Expand definition of cases where a CallInst should be considered to
touch the state struct or otherwise revive a slot.
* Improve DOT creation, fix errors for callinsts
Fix small errors in LiveSetBlockVisitor::CallAccessesState.
Clean up DOT digraph generation further.
* Clean up code per @pag's comments
* Clean up code, add selectinst, fix compile errors
Deal with a few small corner cases with FAV Load and Store instructions,
add SelectInst visitor.
Inline MarkLiveArgs to avoid the complications of C++ generics.
* Clean up logging
* More changes to please Peter
Clean up select and PHI node cases to improve circular dependency
handling.
* Correct errors in visitSelect
* Add load forwarding code
* Add code to run FBV
* Fix map usage error in FBV
* Add call, invoke cases for FBV
* Here are the changes whoops
* Minor API change
* Minor tweaks to DOT digraph printing, as well as attempts to handle the case where the forward analysis pass is incomplete. Still don't have it guarantee completion on everything, but seems 'good enough' for now.
* Change log level
* Fix compile errors due to messy merge
* Pag dead store (#262)
* Fix to script calling wrong function.
* Makes sure that value names are preserved (#249)
* Adds LLVM_VERSION() to accomodate llvm::LLVMContext::setDiscardValueNames() in <3.9 (#250)
* Makes sure that value names are preserved
* Adds LLVM_VERSION() macros to accomodate
* Update README.md
* Update README.md
* set state and memory as noalias (#254)
* Implements some ring 0 instructions in terms of hyper calls and new I/O port intrinsics. (#252)
* Random tests.
* More decode error info
* more playing around
* more system instructions. instrinsics for accessing I/O ports. Split our writes to individual control regs for better identification via hyper calls.
* CR8 read/write support (#255)
* Check argument index of function (#256)
* Fix NoAlias Attributes for older LLVM versions
* Fix for LLVM 4.0 and 3.9
* Fix typo
* Follow remill coding style
* Here are the changes whoops
* Minor API change
* Minor tweaks to DOT digraph printing, as well as attempts to handle the case where the forward analysis pass is incomplete. Still don't have it guarantee completion on everything, but seems 'good enough' for now.
* Change log level
* Fix compile errors due to messy merge
* Improve call/invoke case of FBV
* Improve call/invoke case of FBV
* Removes a level of indirection in the __remill_basic_block function
* Create dedicated stats tracker
Plus clean up the FBV visitor a teensy bit more.
* Move call/invoke LiveSet gen to static func
This commit is to pave the way for future improvements to where this
information is calculated (in FAV instead of LSBV).
* Add code to FAV for calls and invokes
Move call and invoke arg-based livesets to FAV to allow for module-level
LSBV code.
* Begin change of LSBV to module-level
* Minor bug fixes
* Remove some dead code
* Begin adding more code for call/invoke LSBV
* Add entry block checks for LSBV call/invoke
* Info about register names, as well as printing them in the digraphs.
* Make DOT printing only happen per function, as opposed to printing every function per DOT file. Minor tweaks to interprocedural analysis.
* Bug fixes and DOT printing improvements.
* Bug fixes and DOT printing improvements.
* Also print out DOT digraphs of functions after removing stuff
* Tried to make it treat everything before a call to __remill_error as dead but that didn't work out.
* Use datalayout and type sizes to handle the number of elements in a sequential type for LLVM 3.8 compatibility, also check for pointer type.
* More stats, hopefully fixes a bitcast issue.
* Minor bug fixes, and more DOT printing to help diagnose when the offset analysis terminates but there is still stuff in the work list.
* Some possible bug fixes
* Minor bug fix
* Initial support for compilation on OS X
* Special-cased the size_t code for Apple, since it errors on Linux
* Support for OS X
* MACH-compatible ASM for OS X
* Working on macOS platform support for Remill.
* Support for OS X in the build script
* Use curl instead of wget.
* This clarifies the difference between kCategoryInvalid and kCategoryError for instruction categories.
* Removes the --use-system-compiler option, and requires that clang always be used.
* Some general improvements, as well as semantics and tests for PACKUSWB.
* Implements more accurate tracking of x87 exception flags, last instruction pointer, last data pointer, and floating point instruction opcode.
* Fix that masks out the MXCSR register from being compared at the end of tests. We don't yet have good enough info to do this well. Moves the FCMOV instruction set to be under the purview of X87.cpp and its tests.
* Tests should pass now.
* Addressing issues Mike mentioned.
* Added in more aarch64 instructions. Fixed some x86 instructions. The x86 test cases now exercise each test through every possible combination of flags.
* Adding missing files
* Another missing file
* Rename file
* Fixup some macros
* IPR
* More improvements on the test runner
* Test runner fixes related to me not being familiar with aarch64 assembly
* Fixing default data layout
* Trying to use llc to compile bitcode to aarch64 assembly. wth.
* Revert back to using the CMAKE_BC_COMPILER for building the test assembly file instead of the whole CMAKE_LL_COMPILER stuff, now that I've adjusted cxx-common to use the right build target for aarch64.
* Documentation updates. Fixes for aarch64.
* Making progress. The native tests can run, but the first lifted test faults. Not yet sure why.
* Weirdest issue is happening on aarch64. A pointer argument is being compiled to an integer, and that's really screwing things up.
* Add caching of the libraries path to the main cmakelists to avoid having to re-run build.sh all the time when the TRAILOBITS_LIBRARIES env var is not globally defined. Experimenting with trying to force the semantics to be compiled using the x86_64 target, regardless of host arch, or modelled arch of the semantics. This is to try to get around the issue where a single-element struct containing a pointer is lowered into a uintptr_t when passed by value as an argument on aarch64.
* Alright, falling back on handling this problem in the lifter (for now, at least). Really not ideal.
* Test runner works afaict
* Working on issue #122 that removes sudo from the build script and some other little things
* Some stuff relating to Issue #122, some relating to aarch64. My commits are all over the place. Fundamentally, I'm working toward getting travis stuff kind of working. One issue atm is getting the C++ abi right when trying to build remill using old versions of LLVM.
* Adding vector regs to aarch64 basic block function. Also, seems that most things can compile with gcc, although there are some issues compiling the mcsema runtime
Eric Kilmer