* initial WIP sleigh based SPARC32 arch
* add install option to build script
* fix register names
* fix incorrect index into current_bytes in loadFill implementation
* add didrestore register
* dlog register adding
* ignore categories
* remove decompile_mode from llvm
* fix register mapping for flags
* remove nextpc/npc since it doesn't appear to be used and breaks anvill
* overlay NPC with NEXT_PC
* remove next_pc correctly since it was still breaking stuff
* Don't mark instruction as "invalid" when we can't determine a control
flow category
* fix pcode cfg for calls and branches
* add double registers to sparc state structure
* match up with ghidra register names
* fix missing header
* add starts for blocks after control flow ops to guarentee a block split for a term
* remove sparc64_sleigh, fix initialization order
* lift floating point numbers using ghidra names
---------
Co-authored-by: Alex Cameron <asc@tetsuo.sh>
Co-authored-by: 2over12 <ian.smith@trailofbits.com>
* llvm 17 fix
* more instances
* more fixes
* add inline
* bump ci
* bump cxx common
* add 17 case
* bump xcode version
* macos 13
* xcode 15
* bump docker to llvm 17
* change actual matrix value
* opaque pointers are the default now
* actually remove the option
* debug size
* install tree
* fix install
* fix install?
* get tree after
* build dir?
* tree above
* accumulate
* bug?
* try to remove intermediate packaging
* fix opt
* only inline
* give up on function inlining
* llvm 17
* remove debug tasks
* Update scripts/build.sh
Co-authored-by: William Tan <1284324+Ninja3047@users.noreply.github.com>
* Revert "Update scripts/build.sh"
This reverts commit 6c727a539d.
* debug
* try clear out old build
* retest
* try to save more space
* up
* Clear space and don't use cxx-common docker image (#689)
* Clear space and don't use cxx-common docker image
* Run on all PRs
* Fix perms
* Remove some unnecessary apt commands
* Install LLVM version as root
* Allow writing to external target
* Build before running tests
* Install with sudo
* Simplify with container volume mounts
---------
Co-authored-by: William Tan <1284324+Ninja3047@users.noreply.github.com>
Co-authored-by: Eric Kilmer <eric.d.kilmer@gmail.com>
* add sleigh state
* remove bad include
* add volatile
* dont run tests in docker deploys
* remove ccache
* Revert "remove ccache"
This reverts commit 221b70a1bc.
* Revert "dont run tests in docker deploys"
This reverts commit 72229acd6e.
* disable packaging in dockerfile
* fix block packagingA
* fix syntax
* Add skeleton for PPC
* Copyright notices
* Fill in some details for the PPC arch
* Start building a (wrong) PPC runtime
* Begin populating state structure
* First pass for EIS state structure
* Map registers to Sleigh register names
* More fixes
* add optional param
* Create handle unsupported and invalid instruction isels
* Correct typo
* Get a basic `remill-lift` invocation running without failure
* Fix capitalisation
* Set vle context reg
* Fix SleighDecoder signatures
* Set VLE context register in the Sleigh engine in addition to our
internal context reg mapping
* Capitalize reg names
* Add the flag registers for XER and CR
* Rename bitflag structures in PPC state
* PPC Sleigh patches (#643)
* Modified sleigh patch script to generate patches for multiple .sinc files
* update README with new examples of sleigh patch script invocation
* add ppc register definition
* add ppc sleigh patches
* fix issue with remill_insn_size definition
* regenerate sleigh patches for PPC
* update CMakeLists.txt to include PPC patches
* Add TEA signal as a register in the PPC state
* Uppercase the stack pointer register name
* Fix PPC instruction sizes
* initial PPC tests
* remove duplicate tests
* fix tests for e_stmvgprw/e_ldmvgprw
* add tests for loading/storing from special registers
* add tests with internal conditionals in pcode
* fix for pc reg and addr width not being the same... I suspect this issue is going to come up elsewhere
* add heuristic for flow from normal intrainstruction flow
* rework tests to allow testing for different sized registers
* add tests for overflow and record add
* fix bug with log printout
* add intrafunction control flow lifting
* handle edge case where there is no pcode op at the zero index
* Fix another inconsistency with mismatching address and PC reg size
* Allocate unique ptrs in the entry block
* Fix `INT_LEFT` and `INT_RIGHT` impl where shift exceeds bit width
* fix supiece lift?
* Add PPC emulate instruction to hyper call
* fix for pc reg and addr width not being the same... I suspect this issue is going to come up elsewhere
* add heuristic for flow from normal intrainstruction flow
* add intrafunction control flow lifting
* handle edge case where there is no pcode op at the zero index
* Fix another inconsistency with mismatching address and PC reg size
* fix supiece lift?
* Allocate unique ptrs in the entry block
* Fix `INT_LEFT` and `INT_RIGHT` impl where shift exceeds bit width
* fix int2float semantics
should use appropriate sized float based on the output size
* add tests for lifting int2float
* fix INT_{LEFT,RIGHT} semantics
should be `ICmpSGE` instead of `ICmpSGT`
* add cr0-7 registers
* fix formatting
* fix conditional branch test
* add test for compare
* re-enable rotate left word immediate and mask test
* genericize TestSpecOutput
* explicit instruction data size
* add test for syscall/callother (disabled)
* add tests for store/load word
* add test to convert from float to int
* specify intrinsic arg type, fixes null deref
* Add PPC emulate instruction to hyper call
* add headers + formatting
* remove old comment
* Map CRALL register
* Add basic LLVM data layout that specifies 32-bit addresses
* Remove unused variables
* convert auto* to auto when possible
* RegisterPrecondition -> RegisterCondition
* fix variable name
* convert any to variant
* use std::move
* bump to c++20, use concepts
* set arch in constructor since class isn't generic anyways
* formatting
* make type aliases
* bump cxx-common
* add comment
* clang format
* throw exception if register not found
* use const ref
* use shorthand for lambda capture values
* Add more detail to data layout to include proper stack alignment
* Compare to the correct size for SUBPIECE impl
* Add Sleigh message to error
* throw exception in else case
* throw runtime error if register value has incorrect type
* use reference instead of value
* get rid of unnecessary type alias
* formatting
* Propagate VLE context reg value into Sleigh
* Remove unnecessary whitespace
* Remove stale TODO and NOTE comments
* add additional parameter to test runner to specify decoding context
* drop llvm 14, bump macos version
* bump cxx-common, fix ci.yml mac build
* add test for unconditional relative negative branch
* add missing space to pcode debug log
* fix bug due to unordered_map, iteration order matters
* add error log in case we aren't able to adjust PC value
* use helper for getting register reference
* Revert "add optional param"
This reverts commit 51ed49f8cf.
* Remove remaining LLVM 14 compatibility code and configuration
* Add padding between CR and XER flags
* Use `enum class`
* Remove void cast
* Remove unnecessary variable
* Use initialiser lists where appropriate
* Remove redundant `else`
* Prefer `CHECK` over `assert`
* Polish PowerPC function initialisation with lambda
* zero out xer_so to fix tests
* log error when we see claim_eq with no usages
* Collapse namespace blocks
* Remove unnecessary `this->`
* Use `auto` where appropriate
* Remove unnecessary `else`
* Use `emplace` over `insert` for `std::map`
Co-authored-by: lkorenc <lukas.korencik@trailofbits.com>
* Use `constexpr` for VLE reg name
* Use module verification util
* Add `VerifyFunction` util and use where applicable
* Extract lambda to improve readability of flow categorisation
* Use type alias for context values
* Introduce type alias for block exit
* Create type alias for optional branch taken
* Refactor `PcodeCFGBuilder`
* Use lambda to avoid conditional mutation
* Extract duplicated bit-shift code generation into helper
* Simplify flow with ternary
* Add `GetBlock` helper
* Rename variables
* Move statement for clarity
* Create helpers for working with Sleigh context register values
* Convert loop to `std::copy`
* Add a comment explaining the use of set to de-duplicate and sort
* Refactor `IntraProcTransferCollector`
* Expose static method to easily use `IntraProcTransferCollector`
* Rename PPC related variables to include address width
* add docs to intrainstructionindex
* remove llvm 14 ifdefs
* don't log error if no claim_eqs were used
* update comments
* Cleanup exit visitors
---------
Co-authored-by: 2over12 <ian.smith@trailofbits.com>
Co-authored-by: William Tan <1284324+Ninja3047@users.noreply.github.com>
Co-authored-by: lkorenc <lukas.korencik@trailofbits.com>
* re-enable broken float code
* try to cast floats as integers
* fix assingment for function calls
* actually fix calls
* make sure to set branch not taken
* add initial neon registers
* fix missing unit128 fields
* add more debug
* do armv8 since that's what we need right now
* add complete fpscr
* make FPSCR capital
* initialize return_pc
* fix build on linux (#633)
* undo superfluous change (#634)
* revert aarch64 linux changes for now
* fix off by 2 when defining INST_NEXT_PTR in patches for thumb
* fix off by 2
* fix typo
* fix typo
* use maybe float directly
* fix test
* add additional float types
Co-authored-by: William Tan <1284324+Ninja3047@users.noreply.github.com>
* Get Remill building with LLVM 15
* Add missing header for x86 lift test
* Fix `enableOpaquePointers` calls
* Define a non-extern `__remill_state` in each Instructions module
* Remove `__remill_state` variables in tests
* Update build script to support LLVM 15
* Build with LLVM 15 in CI
* Bump CXX Common version
* Update Docker script
* Add comment explaining the definition of the state variable
* Correct wording
* stuff
* make decoders standalone from arch
* compiles
* fix lazy initialization of intrinsic table
* add back sleigh arch
* added switching between arches... need to update contexts
* add instructin sizes
* thumb triplet
* back out of lambda approach
* updating context in thumb case
* fix differential tester to initialize lifter
* add back lambda approach
* aarch32 context updates
* add tests
* fix assetions in tests
* fix address alignment for tests
* add better decode function
* fix tests
* add conditional tests
* fix uninitialized context in contextupdater
* restore old contexts on call...
* null out branch taken arch on indirects
* fix fallthrough for conditional where neither src evaluates properly
* initialize is interproc
* fix initial context arm
* fix boolop bug where params would be lifted when not bool op
* fix comparison
* remove debug
* make missing context non fatal
* bump sleigh
* add insn flow variant
* stub out computing categories
* add control flow structuring
* adding unconditoinal abnormal
* start refactoring flows to share more structure
* rework conditionals to express a combination of a condition with an abnormal flow
* finish basic flow analysis
* add context updater
* implement eq
* stop passing back contexts through return value
* add tests
* rework constructors, make flow usage consistent
* implement flows in aarch32
* add implementation of flows for old cateogries and no context
* make sleigh lifter backwards compatible by applying flows to instructions
* fix bug in generating coarse flows
* refactor to allow lifter to bring along decoding state
* rework insertion of branch taken vats
* fix off by one
* fix size
* dont reset bytes before use
* disable broken float ops
* back of ambiguity to non fatal error, also add logging of callother's encountered
* update names
* remove debug logging
* add comment about x86 context
* move to headers
* move eqs
* order of ctors, fields, methods
* noop subtype of normal
* remove direct constructors
* remove more duplication
* more duplication in noop
* remove unused field
* add utility for is thumb
* absolute path
* remove duplication
* absolute paths
* review fixes
* make const refs
* make enums match style guide
* add constructor to flow
* non ref
* make refs
* move map out of optional
* refer to enum
* move curr id out
* fix BOOL_NEGATE for non 1 bit
* type alias for decoding result of branch taken and flow
* more delog
* this language makes me incredibly happy
* remap flags and treat metavars more like how the instructionlifter does it
* more fixes to metavars
* fix pc reg definition so consistent with value during isntruction execution
* fix alignment of test
* add todo
* extract condition
* add comment
* Add `syscall` case to `__remill_sync_hyper_call`
* Cross compile 32 bit and 64 bit x86 appropriately
* Support `sysenter` and `sysexit`
* Pass in full set of registers for syscall instructions
* Address review comments
* Create initial implementation of `__remill_sync_hyper_call`
* Fill in a few more cases
* Use `state.addr_to_load` for LGDT and LIDT operands
* Fix variable names
* Cross-compile the Remill runtimes
* Create temp variable for `lgdt` and `lidt` handling
* Add intrinsics for SPARC emulate instruction calls
* Create intrinsics for remaining hyper calls
* Remove `__remill_sync_hyper_call` implementations in tests
* Create the target triple based on the provided arch
* Provide ARCH parameter for SPARC archs
* CMake formatting
* Switch the conditions around
* Adjust target triple
* Cross-compile the hyper calls and then link them into the runtime
* Cleanup
* Prefix int types with namespace
* Include `<limits>`
* Fix `lgdt` and `lidt` handling
* Mark new intrinsics as "used"
* Add placeholder intrinsic definitions to tests
* Complete list of intrinsic definitions
* Mark hyper call definition with `always_inline`
* Use `_BitInt` if available
* Use `__builtin_unreachable` instead of `abort`
* Leave comment explaining `always_inline` attribute
* add empty contexts
* add include
* make function const
* add helper for uniform mappings
* expose cache clearing for operand lifter
* decoding context documentation:
* move virtual inheritance down
* remove unused var names
* add type alias
* remove underscores
* make sure we have poetry
* check version in CI
* try specify python3
* newer poetry install script
* fail fast
* try use pythons pip
* upgrade pip?
* install directly
* update in linux too
* add lifter field
* arch has intrinsics
* use insn lifter in trace lifter
* limit default lifters to ops only
* init lifters
* pass arch in tests
* fix bug in swap
* stop using archbase decode in sleigh arches
* cmake: Bring in SLEIGH as a dependency
* sleigh: Boilerplate for adding a new arch
* sleigh: Begin passing instruction sequences into SLEIGH
* cmake: Rename target to be X86 specific
* sleigh: Copy over more X86 runtime code and get things running
* sleigh: Begin populating operands in the returned instruction
* sleigh: Set instruction category for a few opcodes
* sleigh: Initial attempt at generating LLVM IR for P-Code
* sleigh: Implement enough opcodes to run the `sleigh-lift` example
* sleigh: Get things building with an up-to-date Remill tree
* sleigh: Use the new SLA helpers
* sleigh: Clear operands between invalid lifts
* added thumb2
* stub out thumb
* need to refactor to relift the instruction
* factored lifter out of handler:
* allow lifters to decide wether to use sleigh
* relift bytes
* fixed both lifter contexts
* hey a copy lifted :)
* need to refactor to use parameterptrs to unify interface to memory
* refactored to use parameter abstraction
* lift at correct location
* add control flow to pc and next
* add categories mostly
* fix direct semantics add cbranch
* track cbranch metavar also lift returns
* cmake: Fix SleighArch.h path
* sleigh: Support more binary ops
* sleigh: Implement ZEXT
* sleigh: Support SEXT op
* sleigh: Implement 2COMP and INT_NEGATE
* mutex around parsing sleigh specs
* fixed typos in custom image
* added handling for exceptions, need to fix issue with xml parsing
* sleigh: Support boolean binops
* sleigh: Support float binary ops
* sleigh: Support a few more float pcode ops that require LLVM intrinsics
* sleigh: Add entries for ops that require overflow intrinsics
* sleigh: Implement more float unary ops
* sleigh: Reduce duplication in ops that require float intrinsics
* expose mutex
* fix fallthough and format of control flow resolution
* sleigh: Support CARRY, SCARRY and SBORROW
* Remove duplicate THUMB2 arch in enum
* Remove duplicate THUMB2 entry in archnames array
* Implement STORE op
* Fix incorrect index
* Support PIECE and SUBPIECE ops
* Support remaining set of PCode ops in the base set. Now to do the
pseudo ops.
* Implement additional ops
* Partially implement a few pseudo ops + a few missing from the docs
* Fix the variadic op condition
* Include <mutex> to build on Linux
* stop lying about insn categories/functions
* set function insns with mnemonic
* fix fallthrough for ind calls
* More virtualization, fewer pimples
* switch back to x86 normal
* Fix off-by-eight issue with ADR in AArch32
* Get rid of mutex in sleigh, make a globally-available remill locker for handling sleigh issues
* CHange allow 32 bit shift to true, not sure if valid but oh well
* Use the node size when creating constants
* fix memory
* store memory state back
* fix calling intrinsic table
* fix comp sizes
* load pspec data
* add pspec names
* add differential test
* fix tests start pretty printer
* add runner
* added really slow memory
* add comparison of memory state
* do reset
* fix uninitialized module
* fixed lift crash
* add amd64 runtime
* unfix fs, gs, and pc
* added whitelist file
* fix memory to update state with uninitialized reads
* better pc handling
* add main.py
* x86 compiles
* fixup patch generation
* add replacements for mem locs and constant varnodes, now need to handle special branches
* added replacement for direct branches
* add context clears
* add handler for claim_eq
* comment out clears for now
* added ambiguity check
* handle duplicate names in same constructor
* mantain sleigh invariant of 1 or 0 in flag
* added uint8t memory intrinsics
* extend shift value when needed
* handle cmovs
* update patch generation
* removed deprecated load
* fix for returned type
* builds
* add logging
* add comment on why not compute GEPAccessors in Arch.cpp
* enable opaque pointers
* fix memory state update
* add mem_16 impls
* not preserve 1 bit width
* fix CR changes
* fix L and R for shift operands bigger than target
* revert use after move
* more CR fixes
* remove lookahead... lets try this again
* rework to create internal function that is inlined to allow for early return control flow
* terminate conditionally
* avoid large iteration counts for .REP insns
* fix patch for rep insns
* handle inst_start in patches
* handle inserting insn_size constructor outside of macros
* hint size of insn_size
* refactor bool functions to make more sense
* just use ;
* remove delete of operands when fail to lift
* remove copy and paste
* remove unused deps
* add patch file to sleigh list
* prep git user
* refer to checkout
* maybe shell type?
* fix layout
* fix format again
* fix
* update dockerfile
* fix gitwatcher to point to current project:
* Revert "fix gitwatcher to point to current project:"
This reverts commit 711da11e6f.
* fix ninja builds
* disable sleigh tests
* remove duplicates
* refactor
* fix non reset context in sleigh lifter
* remove debugging prints
* wrote CI runner
* add handwritten test infra
* refactor to allow sharing JIT test running between differential tester and hand written tests
* allow for internal ownership of semantics module
* handwritten test framework
* fix hand written tests
* pc rel testruns
* add pc rel regression
* fix script for pc rel
* fix names for priors to ignore whitespace
* fix action ops
* add xor to repl ops
* add xors to patch
* add arm patch to build
* fix test for semantics
* allow cross platform tests to run
* add running diff tester
* 2 underscores?
* run tests on macos too
* fix command
* fix whitelist path
* remove unused test data
* install python deps for differential tests
* update script to git patches
* update patches to git format
* attempt to fixup patches
* update ref for sleigh
* install test depends docker
* point docker to correct dir
* install pydeps in macos too
* initialize address field
* exports, also tag master on sleigh
* add comments and whitelist undefined value OF in shifts
* add of accessor to whitelist impl
* change variables to non alloca and remove dead code
* remove dead code:
* replace dump with print
* code quality
* make non null into ref
* type alias
* remove allocas from unit tests
* Address copypasta comment.
* Address PR comment.
* Address PR comment.
* fix build
* unify diff modules into single structure
* remove extra newline
* lock sleigh to specific commit
* remove new lines
* remove useless fenv headers
* caps
* copyright notices
* update more copyright
* fix reviews
* early returns out of accessors
* early return
* auto
* return getarchbyname
* insert register rather than modify reference
* just return
* only log in the assembly logger
* prefer functional style
* remove commented code
* move defaults to header
* that's not how arch switches work in pcode
* informative names
* simplify control flow in fill
* early return
* fix early exit condition
* refactor register default into function
* make one liner
* early return
* unary instead of unop
* construct pair with {}
* move cbranch into binops instead of integer binops
* refactor float ops into getter
* factor out float type
* early return
* separate out callother handling
* do pointer extensions cleaner
* braces
* add private headers to lifters
* refactor redirecting control flow out of instruction
* use constant check function
* remove has_value
* structure for preconditions
* return success when applying eq claim
* expose arch base and move shared functionality into x86 base
* dedup x86 code
* dedup aarch32
* remove needless assignment, also remove else after an if that returns
Co-authored-by: Alex Cameron <asc@tetsuo.sh>
Co-authored-by: Artem Dinaburg <artem@trailofbits.com>
Co-authored-by: Peter Goodman <peter.goodman@gmail.com>
Duncan Ogilvie