Commit Graph

6895 Commits

Author SHA1 Message Date
Alessandro Di Federico d9123fcd49 PECOFFImporter: fix handling of relocations
This commit ensures relocations imported from PE/COFF binaries have
generic `Address`. The fact that this was not the case lead to not
detecting dynamic calls.

The commit also improves logging and fixes the handling of maximum depth
visit of the PDB importer.
2025-12-19 14:44:07 +01:00
Alessandro Di Federico 44fdba9958 JumpTargetManager: add read-from-pointer logger 2025-12-19 14:38:19 +01:00
Alessandro Di Federico 66b2374d70 model::Relocation: require Address to be generic 2025-12-19 14:38:19 +01:00
Alessandro Di Federico c18aa628e7 Minor changes 2025-12-19 14:38:19 +01:00
Alessandro Di Federico e1a7cc79eb Merge branch 'feature/fix-pypeline-crashes' 2025-12-19 14:37:24 +01:00
Giacomo Vercesi 971cea5bf1 pype-cli: better propagate pypeline errors
Convert the `RuntimeError`s emitted by the pipebox (e.g. when checking
preconditions or running analyses) into `PypelineError`s and have the
pypeline cli infrastructure handle them gracefully, without emitting
stacktraces.
2025-12-19 10:29:28 +01:00
Giacomo Vercesi e8b55583e6 isolate: outline function splitting in function
Outline the code responsible for splitting an isolated function into its
own `llvm::Module` in a separate function.
2025-12-19 10:29:28 +01:00
Giacomo Vercesi 664eb497c1 isolate: selectively clone helpers
When performing `cloneFiltered`, instead of cloning all the helper
functions only clone those that are transitively called from the isolated
function.
2025-12-19 10:29:28 +01:00
Giacomo Vercesi 7f7f9571a8 pypeline: implement container discarding
Add infrastructure to pypeline that allows containers to be notified
when they are being used last, this allows two things:
* `Pipe`s eagerly clearing those containers once they are done reading
  their contents
* `ScheduledTask`s clearing those out at the end of their execution in
  case the pipe did not do it
This overall should improve memory usage as container no longer take up
memory if they are no longer used as part of a `Schedule`.
2025-12-19 10:29:28 +01:00
Giacomo Vercesi 3d2015a233 isolate: implement trivial memory optimizations
Implement trivial memory optimizations on the `isolate` pipe:
* Clone the `root` module right away to the correct context before
  performing isolation
* Delete the `root` function's body right after isolation
* Delete the isolated function body from the cloned module after it has
  been `cloneFiltered`-ed into the output container
2025-12-19 10:28:29 +01:00
Giacomo Vercesi 265af3768f Binary importers: fix debug info detection
Fix the behavior of binary importers by propagating the path of the
input binary from revng2 downwards, allowing finding `.debug` files in
the correct paths.
2025-12-19 10:28:29 +01:00
Giacomo Vercesi e8f293271b BinariesContainer: page files to disk
Store the contents of the files deserialized by `BinariesContainer` to
disk via `TemporaryFile`. This also allows some pipes to avoid shuffling
data around since they needed a file on disk anyways.
2025-12-19 10:28:29 +01:00
Giacomo Vercesi 397df408be RemoveStackAlignment: allow missing InitFunction 2025-12-19 10:28:29 +01:00
Giacomo Vercesi eefdca934b pypeline: give better names to some attributes
Rename some attributes of pypeline objects to better-suited names:
* `TaskArgument.to_container_decl` -> `TaskArgument.declaration`
* `PipelineNode.arguments` -> `PipelineNode.argument_declarations`
* `PipelineNode.arguments_with_access` -> `PipelineNode.arguments`
2025-12-19 10:26:49 +01:00
Giacomo Vercesi a2a7e6442b pypeline: move PipelineNode.run
Inline the body of `PipelineNode.run` into `ScheduledTask.run`, making
`PipelineNode` a pure data structure. This also allows eliding all the
arguments since all of them as properties of `ScheduledTask`.
2025-12-19 10:18:10 +01:00
Giacomo Vercesi 9338ac3486 pypeline: expand ScheduledTask's constructor
Move the arguments with a longer lifetime from the arguments of
`ScheduledTask`'s `run` function to its constructor.
2025-12-19 10:17:37 +01:00
Giacomo Vercesi 12c180bfd9 pypeline: convert uses of pypeline_logger.log
The `pypeline_logger.log` functions should only be used to convey
information to the user, all other uses should fall under `debug_log`.
2025-12-17 11:58:46 +01:00
Alessandro Di Federico 75eb0fc0aa Merge branch 'feature/lift-fixes-for-pypeline' 2025-12-17 10:08:35 +01:00
Alessandro Di Federico 6175c43b0c Lift: zero initialize temporaries
Sometimes QEMU will emit code that reads a local temporary *before* it
is assigned in the `TranslationBlock`. This usually happens when lifting
non-code, but it's still a problem since it leads to phis around the
dispatcher, which we don't want.

For example, consider the following x86-64 code:

```
0000000140002000 <.text>:
140002000: f0    lock
140002001: 38 00 cmpb    %al, (%rax)
```

It leads to:

```
[libtcg] Translation starting from 0x140002000:Generic64
[libtcg]   ---- 0000000040002000
[libtcg]     mov_i64 loc2,rax
[libtcg]     mov_i64 loc1,rax
[libtcg]     mov_i64 cc_src,loc1
[libtcg]     mov_i64 loc8,loc0
[libtcg]     sub_i64 cc_dst,loc0,loc1
[libtcg]     discard cc_src2
[libtcg]     discard cc_op
```

Note how `loc0` is read before being initialized.

Currently we zero-initialize all the temporaries. It would be in
princple possible to detect those that need the initialization, which
would also enable us to emit a warning.
2025-12-16 17:42:07 +01:00
Alessandro Di Federico c7cdd22843 Lift: do not fail if newpc is absent 2025-12-16 17:41:55 +01:00
Alessandro Di Federico 5820908675 Remove and ban \file 2025-12-16 17:41:55 +01:00
Alessandro Di Federico dd84504321 Merge branch 'feature/pypeline-implement-second-half' 2025-12-12 16:33:36 +01:00
Giacomo Vercesi 1bcd1ef920 mass-testing-report: names for binaries
Instead of having all the binaries in the mass-testing-report be
downloaded with the name `input`, have them use a meaningful name
instead.
2025-12-12 15:46:49 +01:00
Giacomo Vercesi 94b9ecad6c mass-testing: exclude paths from flamegraph
Add a new option to the mass-testing `meta.yml` that allows skipping
some paths when generating the flamegraph. In the case of revng this
allows skipping the intermediate python frames.
2025-12-12 15:46:49 +01:00
Giacomo Vercesi 793b8999e9 Adapt mass-testing infrastructure to pypeline
Make the required changes so that `revng2` can be used with mass-testing
and the per-component stats are correct.
2025-12-12 15:46:49 +01:00
Giacomo Vercesi 6e7efc835b InitRevng: fix memory leak due to stack trace
When initializing `revng::InitRevng` it is assumed by its parent,
`InitLLVM`, that the lifetime of `Argv` and be equivalent to
static, because it might be later retrieved to print it in case of a
crash. This works with the traditional `main`, however in the pipebox
the string passed are not guaranteed to have that lifetime. This commits
moves ownership of the strings to a class that stores them to avoid this
problem.
2025-12-12 15:46:49 +01:00
Giacomo Vercesi 1632aeac73 Comment run* function with inlinable content
Add `TODO` comments to piperuns' `run*` functions that will have content
inlined once the old pipeline is removed.
2025-12-10 16:16:14 +01:00
Giacomo Vercesi 8af5e1d05c Implement revng2 quick artifact 2025-12-10 16:16:14 +01:00
Giacomo Vercesi e3b263d9da Implement revng2 project init 2025-12-10 16:16:14 +01:00
Giacomo Vercesi f7231a29da Add yield-cfg to pypeline 2025-12-10 16:16:14 +01:00
Giacomo Vercesi 255748228a Add yield-call-graph-slice to pypeline 2025-12-10 16:16:14 +01:00
Giacomo Vercesi 83e770551b Add yield-call-graph to pypeline 2025-12-10 16:16:14 +01:00
Giacomo Vercesi 4385f28c17 Add YieldCallGraph to pypeline 2025-12-10 16:16:14 +01:00
Giacomo Vercesi 9a202822f1 Add ProcessCallGraph to pypeline 2025-12-10 16:16:14 +01:00
Giacomo Vercesi 0d732f7a72 Add emit-cfg artifact to pypeline 2025-12-10 16:16:14 +01:00
Giacomo Vercesi 3a387238dc Add cleanup-ir to pypeline 2025-12-10 16:16:14 +01:00
Giacomo Vercesi a8ed75fdb1 Add CleanupIR to pypeline 2025-12-10 16:16:14 +01:00
Giacomo Vercesi d80c0a73f6 Add MergeLLVMModules to pypeline 2025-12-10 16:16:14 +01:00
Giacomo Vercesi 05dc279ef2 Add emit-c-as-single-file to pypeline 2025-12-10 16:16:14 +01:00
Giacomo Vercesi 0ffce5c5b2 Add emit-c to pypeline 2025-12-10 16:16:14 +01:00
Giacomo Vercesi e260d05082 Add EmitC to pypeline 2025-12-10 16:16:14 +01:00
Giacomo Vercesi 9b527c8b69 Add ImportModelNames to pypeline 2025-12-10 16:16:14 +01:00
Giacomo Vercesi cfe476c701 Add ModelVerifyClift to pypeline 2025-12-10 16:16:14 +01:00
Giacomo Vercesi 75455a339e Add PureMLIRPassesPipe to pypeline
In the same vein as `PureLLVMPassesPipe`, add a pipe to pypeline that
allows running arbitrarily-specified MLIR passes that do not depend on
the model.
2025-12-10 16:16:14 +01:00
Giacomo Vercesi 97fe9203e1 Add LLVMToClift to pypeline 2025-12-10 16:16:14 +01:00
Giacomo Vercesi 08154335e3 Add SwitchToStatements to pypeline 2025-12-10 16:16:14 +01:00
Giacomo Vercesi ee1fdebc4e Add SegregateStackAccesses to pypeline 2025-12-10 16:16:14 +01:00
Giacomo Vercesi 3cd011c146 Introduce CliftFunctionMixin
In equivalence with `LLVMFunctionMixin`, add a mixin that allows
writing a pipe that instead of defining the `runOnFunction` method it
defines the `runOnCliftFunction` which receives both the
`module::Function` and the `FunctionOp`.
2025-12-10 16:16:14 +01:00
Giacomo Vercesi 16e89416c5 Add CliftFunctionContainer
Add a container that owns one `mlir::ModuleOp` per function, to be used
in conjunction with the new clift backend.
2025-12-10 16:16:14 +01:00
Giacomo Vercesi 281484ad79 Add initial-auto-analysis list to pypeline 2025-12-10 16:16:14 +01:00