63 Commits

Author SHA1 Message Date
cctv130 b6fc70f0f4 CMake: update to C++20, clean up properties, optimize fetch content (#721)
- Enable C++20 on both Windows and Linux
- Remove unnecessary set_property calls to silence LOCATION warnings
- Replace duplicate -fPIC flags with INTERFACE_POSITION_INDEPENDENT_CODE
- Set FETCHCONTENT_BASE_DIR to speed up sleigh and ghidra-fork downloads/builds
- fixed: Does not match the generator used previously: Unix Makefiles,use ninja instead
2025-09-05 17:55:23 -04:00
William Tan 17cff6b4df SPARC support via Sleigh (#681)
* initial WIP sleigh based SPARC32 arch

* add install option to build script

* fix register names

* fix incorrect index into current_bytes in loadFill implementation

* add didrestore register

* dlog register adding

* ignore categories

* remove decompile_mode from llvm

* fix register mapping for flags

* remove nextpc/npc since it doesn't appear to be used and breaks anvill

* overlay NPC with NEXT_PC

* remove next_pc correctly since it was still breaking stuff

* Don't mark instruction as "invalid" when we can't determine a control
flow category

* fix pcode cfg for calls and branches

* add double registers to sparc state structure

* match up with ghidra register names

* fix missing header

* add starts for blocks after control flow ops to guarentee a block split for a term

* remove sparc64_sleigh, fix initialization order

* lift floating point numbers using ghidra names

---------

Co-authored-by: Alex Cameron <asc@tetsuo.sh>
Co-authored-by: 2over12 <ian.smith@trailofbits.com>
2023-11-10 07:44:27 -05:00
2over12 0183248218 llvm 17 fixes (#688)
* llvm 17 fix

* more instances

* more fixes

* add inline

* bump ci

* bump cxx common

* add 17 case

* bump xcode version

* macos 13

* xcode 15

* bump docker to llvm 17

* change actual matrix value

* opaque pointers are the default now

* actually remove the option

* debug size

* install tree

* fix install

* fix install?

* get tree after

* build dir?

* tree above

* accumulate

* bug?

* try to remove intermediate packaging

* fix opt

* only inline

* give up on function inlining

* llvm 17

* remove debug tasks

* Update scripts/build.sh

Co-authored-by: William Tan <1284324+Ninja3047@users.noreply.github.com>

* Revert "Update scripts/build.sh"

This reverts commit 6c727a539d.

* debug

* try clear out old build

* retest

* try to save more space

* up

* Clear space and don't use cxx-common docker image (#689)

* Clear space and don't use cxx-common docker image

* Run on all PRs

* Fix perms

* Remove some unnecessary apt commands

* Install LLVM version as root

* Allow writing to external target

* Build before running tests

* Install with sudo

* Simplify with container volume mounts

---------

Co-authored-by: William Tan <1284324+Ninja3047@users.noreply.github.com>
Co-authored-by: Eric Kilmer <eric.d.kilmer@gmail.com>
2023-10-12 16:37:45 -04:00
2over12 a6abbb818c Mirror Sleigh's Byte-width Flags (#668)
* add sleigh state

* remove bad include

* add volatile

* dont run tests in docker deploys

* remove ccache

* Revert "remove ccache"

This reverts commit 221b70a1bc.

* Revert "dont run tests in docker deploys"

This reverts commit 72229acd6e.

* disable packaging in dockerfile

* fix block packagingA

* fix syntax
2023-07-10 12:07:42 -04:00
Alex Cameron 031305d519 Get Remill building with LLVM 16 (#665)
* Get Remill building with LLVM 16

* Update scripts and README

* Refactor `MoveConstantIntoModule` check to reduce duplication

* Remove hack since we don't plan to clone functions across contexts

* Remove call to deprecated LLVM function

* Add LLVM 16 to CI

* Bump cxx-common version

* Fix scripts to work with new `cxx-common`

* Don't add unsupported `readnone` attribute with LLVM 16

* Use `memory(none)` with LLVM 16 instead of `readnone`

* Add LLVM 16 to docker lifter entrypoint
2023-04-28 10:29:34 +10:00
Eric Kilmer b677387f6f Test on Ubuntu 22.04 (#660)
* Only support and test Ubuntu 22.04

* Fix regex matching in CI

* Remove unnecessary addr size check

---------

Co-authored-by: Alex Cameron <asc@tetsuo.sh>
2023-03-30 10:46:25 -04:00
Alex Cameron fb018c96e9 PowerPC Support (#645)
* Add skeleton for PPC

* Copyright notices

* Fill in some details for the PPC arch

* Start building a (wrong) PPC runtime

* Begin populating state structure

* First pass for EIS state structure

* Map registers to Sleigh register names

* More fixes

* add optional param

* Create handle unsupported and invalid instruction isels

* Correct typo

* Get a basic `remill-lift` invocation running without failure

* Fix capitalisation

* Set vle context reg

* Fix SleighDecoder signatures

* Set VLE context register in the Sleigh engine in addition to our
internal context reg mapping

* Capitalize reg names

* Add the flag registers for XER and CR

* Rename bitflag structures in PPC state

* PPC Sleigh patches (#643)

* Modified sleigh patch script to generate patches for multiple .sinc files

* update README with new examples of sleigh patch script invocation

* add ppc register definition

* add ppc sleigh patches

* fix issue with remill_insn_size definition

* regenerate sleigh patches for PPC

* update CMakeLists.txt to include PPC patches

* Add TEA signal as a register in the PPC state

* Uppercase the stack pointer register name

* Fix PPC instruction sizes

* initial PPC tests

* remove duplicate tests

* fix tests for e_stmvgprw/e_ldmvgprw

* add tests for loading/storing from special registers

* add tests with internal conditionals in pcode

* fix for pc reg and addr width not being the same... I suspect this issue is going to come up elsewhere

* add heuristic for flow from normal intrainstruction flow

* rework tests to allow testing for different sized registers

* add tests for overflow and record add

* fix bug with log printout

* add intrafunction control flow lifting

* handle edge case where there is no pcode op at the zero index

* Fix another inconsistency with mismatching address and PC reg size

* Allocate unique ptrs in the entry block

* Fix `INT_LEFT` and `INT_RIGHT` impl where shift exceeds bit width

* fix supiece lift?

* Add PPC emulate instruction to hyper call

* fix for pc reg and addr width not being the same... I suspect this issue is going to come up elsewhere

* add heuristic for flow from normal intrainstruction flow

* add intrafunction control flow lifting

* handle edge case where there is no pcode op at the zero index

* Fix another inconsistency with mismatching address and PC reg size

* fix supiece lift?

* Allocate unique ptrs in the entry block

* Fix `INT_LEFT` and `INT_RIGHT` impl where shift exceeds bit width

* fix int2float semantics

should use appropriate sized float based on the output size

* add tests for lifting int2float

* fix INT_{LEFT,RIGHT} semantics

should be `ICmpSGE` instead of `ICmpSGT`

* add cr0-7 registers

* fix formatting

* fix conditional branch test

* add test for compare

* re-enable rotate left word immediate and mask test

* genericize TestSpecOutput

* explicit instruction data size

* add test for syscall/callother (disabled)

* add tests for store/load word

* add test to convert from float to int

* specify intrinsic arg type, fixes null deref

* Add PPC emulate instruction to hyper call

* add headers + formatting

* remove old comment

* Map CRALL register

* Add basic LLVM data layout that specifies 32-bit addresses

* Remove unused variables

* convert auto* to auto when possible

* RegisterPrecondition -> RegisterCondition

* fix variable name

* convert any to variant

* use std::move

* bump to c++20, use concepts

* set arch in constructor since class isn't generic anyways

* formatting

* make type aliases

* bump cxx-common

* add comment

* clang format

* throw exception if register not found

* use const ref

* use shorthand for lambda capture values

* Add more detail to data layout to include proper stack alignment

* Compare to the correct size for SUBPIECE impl

* Add Sleigh message to error

* throw exception in else case

* throw runtime error if register value has incorrect type

* use reference instead of value

* get rid of unnecessary type alias

* formatting

* Propagate VLE context reg value into Sleigh

* Remove unnecessary whitespace

* Remove stale TODO and NOTE comments

* add additional parameter to test runner to specify decoding context

* drop llvm 14, bump macos version

* bump cxx-common, fix ci.yml mac build

* add test for unconditional relative negative branch

* add missing space to pcode debug log

* fix bug due to unordered_map, iteration order matters

* add error log in case we aren't able to adjust PC value

* use helper for getting register reference

* Revert "add optional param"

This reverts commit 51ed49f8cf.

* Remove remaining LLVM 14 compatibility code and configuration

* Add padding between CR and XER flags

* Use `enum class`

* Remove void cast

* Remove unnecessary variable

* Use initialiser lists where appropriate

* Remove redundant `else`

* Prefer `CHECK` over `assert`

* Polish PowerPC function initialisation with lambda

* zero out xer_so to fix tests

* log error when we see claim_eq with no usages

* Collapse namespace blocks

* Remove unnecessary `this->`

* Use `auto` where appropriate

* Remove unnecessary `else`

* Use `emplace` over `insert` for `std::map`

Co-authored-by: lkorenc <lukas.korencik@trailofbits.com>

* Use `constexpr` for VLE reg name

* Use module verification util

* Add `VerifyFunction` util and use where applicable

* Extract lambda to improve readability of flow categorisation

* Use type alias for context values

* Introduce type alias for block exit

* Create type alias for optional branch taken

* Refactor `PcodeCFGBuilder`

* Use lambda to avoid conditional mutation

* Extract duplicated bit-shift code generation into helper

* Simplify flow with ternary

* Add `GetBlock` helper

* Rename variables

* Move statement for clarity

* Create helpers for working with Sleigh context register values

* Convert loop to `std::copy`

* Add a comment explaining the use of set to de-duplicate and sort

* Refactor `IntraProcTransferCollector`

* Expose static method to easily use `IntraProcTransferCollector`

* Rename PPC related variables to include address width

* add docs to intrainstructionindex

* remove llvm 14 ifdefs

* don't log error if no claim_eqs were used

* update comments

* Cleanup exit visitors

---------

Co-authored-by: 2over12 <ian.smith@trailofbits.com>
Co-authored-by: William Tan <1284324+Ninja3047@users.noreply.github.com>
Co-authored-by: lkorenc <lukas.korencik@trailofbits.com>
2023-02-02 10:05:28 -05:00
Alex Cameron a8ead7b584 Move to explicitly setting CMake toolchain file (#639)
* Move to CMake toolchain file

* Update docs and scripts

* Fix `build.sh` to manually specify the `VCPKG_TARGET_TRIPLET`

* Use x64 instead of amd64 in the VCPKG target triplet

* Remove references to VCPKG in Remill config

* Update cmake/remillConfig.cmake.in

Co-authored-by: Eric Kilmer <eric.d.kilmer@gmail.com>

* Update cmake/remillConfig.cmake.in

Co-authored-by: Eric Kilmer <eric.d.kilmer@gmail.com>

* Fix suggested edit

* Stop specifying compilers in CMake presets

* Fix typo

* Remove trailing comma from JSON file

Co-authored-by: Eric Kilmer <eric.d.kilmer@gmail.com>
2022-12-02 15:45:48 +11:00
Alex Cameron d5c5035728 Get Remill building with LLVM 15 (#631)
* Get Remill building with LLVM 15

* Add missing header for x86 lift test

* Fix `enableOpaquePointers` calls

* Define a non-extern `__remill_state` in each Instructions module

* Remove `__remill_state` variables in tests

* Update build script to support LLVM 15

* Build with LLVM 15 in CI

* Bump CXX Common version

* Update Docker script

* Add comment explaining the definition of the state variable

* Correct wording
2022-10-30 16:42:23 -04:00
William Tan 8f34768ad4 collapse case 2022-08-29 14:11:50 -04:00
William Tan 3056b3bcb4 extra v 2022-08-29 13:59:22 -04:00
William Tan a49b737f52 forgot v 2022-08-29 13:53:46 -04:00
William Tan c1c993f5f9 detect arm64 in addition to aarch64 for m1 2022-08-29 13:38:03 -04:00
William Tan f1443739ec bump cxx-common 2022-08-29 13:37:41 -04:00
Alex Cameron 66ef46be8a Remove compatibility for LLVM < 14 (#606)
* Remove compatibility with LLVM < 14

* Remove redundant VectorType compat file

* Remove redundant Attributes compat file

* Remove redundant BitcodeReaderWriter compat file

* Remove redundant Verifier compat file

* Remove redundant Cloning compat file

* Remove redundant DebugInfo compat file

* Remove redundant GlobalValue compat file

* Remove redundant FileSystem compat file

* Remove redundant JITSymbol compat file

* Remove redundant TargetLibraryInfo compat file

* Remove redundant ScalarTransforms compat file

* Actually remove the file this time

* Remove redundant ToolOutputFile compat file

* Remove redundant DataLayout compat file

* Move Error helpers out of Compat directory

* Remove redundant CallSite compat file

* Remove redundant RuntimeDyld compat file

* Remove redundant CTypes compat file

* Modify build script to exclude LLVM 13

* Update references to LLVM in README
2022-06-28 10:21:54 -04:00
Alex Cameron 81478b4dbe Add LLVM 14 to CI (#598)
* Add LLVM 14 to CI

* Update `build.sh`

* Fix docker entrypoint

Co-authored-by: Eric Kilmer <eric.d.kilmer@gmail.com>
2022-06-07 17:07:13 -04:00
2over12 9b05e6f3ad Fix SSE Instructions for XED v2022.04.17 (#597)
* added logging

* add logging

* add op corresponding to read

* same fix for movhps

* add fillers to MOVLPD

* add filler to MOVLPS

* add filler to CVTSD2SS

* filler for CVTSS2SD

* remove logging

* bump cxx-common

* drop llvm-12 support

* drop in other runs

* add dummy read to MOVHLPS

* change ISEL to match size from xed update

* fix MOVLHPS

* fix VMOVDDUP_XMMdq_XMMdq size
2022-06-04 10:36:34 -04:00
Eric Kilmer e91f18e059 Remove Ubuntu 18.04
The move to c++17 is so much easier to test and support if we just
remove official support for Ubuntu 18.04.

Ubuntu 22.04 is almost here also
2022-04-14 14:07:51 -04:00
Peter Goodman d5928a0faa llvm 13 support (#577)
* llvm 13 support

* Add llvm 13

* Update build script

* Update build script

* Modify remill-lift docker entrypoint
2022-02-16 19:13:17 -05:00
Artem Dinaburg edbc6523f4 Build via CMakePresets.json (#570)
* Add a script that builds via CMake Presets and some default CMake presets
* Bump cxx-common past the one slightly broken release.

Co-authored-by: Eric Kilmer <eric.d.kilmer@gmail.com>
2022-01-03 14:54:22 -05:00
Eric Kilmer 75a9e041d6 Test and officially support LLVM 12 only (#548)
* Test and officially support LLVM 12 only

* Remove tools that are already included in Docker image
2021-10-07 21:13:22 -04:00
kumarak 9006baf7db WIP: Fix FPU types (#536)
Update remill's FPU semantics to use 80-bit floats, at least on x86/amd64.

Co-authored-by: Artem Dinaburg <artem@trailofbits.com>
2021-08-25 12:26:21 -04:00
Eric Kilmer e768d25564 Fixup cxx-common links (#519)
Moved the repo from trailofbits to lifting-bits
2021-05-26 12:17:06 -04:00
Eric Kilmer 94722b1677 Use cxx-common 0.1.4 (#517)
Fixes for building with latest MacOS 11 development tools
2021-05-18 16:06:02 -04:00
Eric Kilmer 1b415d616b Use system compiler to build (#516)
* Don't use vcpkg clang to build remill in build script

* Don't manually set compiler within CMakeLists.txt & Fix ccache detection

* Simplify ccache usage

Don't use a macro since we just immediately invoke it

* Turn on CMAKE_ENABLE_EXPORTS to fix tests

Not sure exactly why the updated CMake version affected this, but it
did.

See https://cmake.org/cmake/help/latest/variable/CMAKE_ENABLE_EXPORTS.html#variable:CMAKE_ENABLE_EXPORTS

* Bump number of build jobs in Dockerfile
2021-05-17 15:13:17 -04:00
Eric Kilmer 76f3725f79 Build with LLVM 12 (#515)
* Build with LLVM 12

* Update scripts

* Fix syntax
2021-05-17 10:14:27 -04:00
Eric Kilmer d9255fa728 CI: Use v0.1.3 (latest) cxx-common prebuilt libraries (#511) 2021-04-28 16:31:58 -04:00
Eric Kilmer a9c1c9d735 Use clang from vcpkg in build script (#506)
We've lost support for building with gcc and this helps prevent build
errors if someone isn't using clang
2021-04-15 13:35:32 -04:00
Alessandro Gario 9c315668c8 CI: Auto-generate releases when pushing tags (#487) 2021-02-20 14:31:51 +01:00
Alessandro Gario 53f30c4ceb CMake: Add packaging support (#486) 2021-02-18 16:36:02 +01:00
Eric Kilmer 37741957d6 Support vcpkg dependency management (#463)
First step towards transitioning away from cxx-common's `pkgman.py` dependency management system towards compatibility with and use of vcpkg (https://github.com/microsoft/vcpkg) to manage dependencies.

This commit attempts to support both the new and old build systems until we can phase out the old completely.

Please see the updated trailofbits/cxx-common repository or run the `./scripts/build.sh` script to fetch the required dependencies.
2020-12-10 17:28:40 -05:00
Peter Goodman 3808e9951d Refactor and add sparc (#454)
* Refactors the code to improve directory layout, use more std::string_view in place of std::string, deletes some deprecated functions, deprecates some other functions for eventual deletion, and includes semantics for sparcv8 (sparc32) and sparcv9 (sparc64)

* Update new dir layout with llvm 11 support

* Whoops missing files

* Drop llvm 800 from workflow

* Minor fix

* Move where the install directives are in CMake

* Minor fixes

* Rename tools/ to bin/.

* Minor tweaks

* Should fix issues
2020-10-28 15:11:42 -04:00
Artem Dinaburg 4aea5a8f2a Add LLVM11 to CI, remove ubuntu 1910 (#453)
* Add LLVM11 to CI, remove ubuntu 1910
* LLVM11 Compatibility fixes
* Bump LLVM version in scripts
2020-10-27 14:01:20 -04:00
Peter Goodman 5af0122196 General API improvements (#424)
* General API improvements

* Minor changes

* New name for existing isel

* Error compatibility with llvm8

* llvm 10 to build script

Co-authored-by: kumarak <iit.akshay@gmail.com>
2020-06-30 21:42:23 -04:00
Eric Kilmer 15a9178e5f Small final docker image (#416)
* Small final docker image
* Add headers needed by McSema
2020-05-16 01:56:23 -04:00
Artem Dinaburg 14ab273f32 Update build script to fetch from Github Releases and not S3 (#415) 2020-05-07 14:54:39 -04:00
van Hauser 1395dd1c89 kali support (#390) 2020-01-03 23:39:04 -05:00
Peter Goodman 6729755cad Llvm 900 (#377)
* Support for llvm 8.0.0 and llvm 9.0.0. Thanks to Peter Garba and John Regehr.

* Fixes for macOS to bake the SDK root into the config for the bc compiler

* Script fixes

* Update travis.sh

* Update travis.sh
2019-11-18 20:01:52 -05:00
Aiethel c5e297fe04 Remove whitespaces (#358) 2019-08-16 15:32:22 -04:00
Aiethel 1e9d762eb4 Flag option in build.sh (#348) 2019-07-03 09:16:17 -04:00
Peter Goodman c0c0847966 Update build.sh 2019-04-17 16:19:27 -04:00
Peter Goodman 1f94663462 Update build.sh 2019-03-23 17:44:34 -04:00
Peter Goodman 00ce1ece90 Add support for llvm 8.0 to the build script (#341)
* Add support for llvm 8.0 to the build script

Also adds a new compiler option, `--use-host-compiler`, which looks for `$CC` and `$CXX` in place of using our build of Clang. In the case of LLVM < 5.0, using the host compiler is the default behaviour.

* Update DeadStoreEliminator.cpp
2019-03-22 16:20:47 -04:00
Kyle Martin 4b873d8c19 18.10 doesn't compile with 16.04 libraries, but does with 18.04 (#322) 2019-01-12 08:17:51 -05:00
Peter Goodman 8f7b62e72f Update build.sh 2019-01-09 21:25:40 -05:00
Peter Goodman 7104442d51 Fix win32 (#321)
* Attempt to fix remill-lift on win32. Also, enable ubuntu 18.04-specific cxx-common builds, instead of falling back on 16.04

* Minor fix
2019-01-07 20:38:29 -05:00
Frank Busse 62284e4833 build.sh: minor fixes (#316) 2018-11-20 16:31:56 +01:00
Peter Goodman be5d6c0bf7 Add Ubuntu 18.10 to build.sh. 2018-10-31 15:43:46 -04:00
artemdinaburg 7911af9cf1 Allow for extra args to cmake durinng build. Used to pass arguments to permit ABI libraries to McSema (#295) 2018-09-09 12:07:20 -04:00
Peter Goodman e338c188e6 Minor improvements. 2018-09-08 11:08:17 -04:00