Commit Graph

891 Commits

Author SHA1 Message Date
Giacomo Vercesi c53e615fa0 Cleanup pypeline code
Remove some cruft and apply trivial changes to the existing pypeline
code, especially on the daemon side.
2026-03-31 17:00:48 +02:00
Giacomo Vercesi 773245b893 pypeline/cli: use dataclass for ctx.obj
Add typing to the `ctx.obj` object used to store pypeline-specific data
while parsing the command-line in click.
2026-03-31 17:00:48 +02:00
Andrea Gussoni 30a1396a64 EmitFieldAccesses: typedef pointer as array tests 2026-03-30 16:03:37 +02:00
Andrea Gussoni 4c27971ed8 EmitFieldAccesses: pointer as array tests 2026-03-30 16:03:37 +02:00
Ivan Krysak 979a384763 mlir_lit_tests: test unit attribute emission 2026-03-27 08:28:21 +00:00
Ivan Krysak 82e86cf167 Emit prototype attributes *before* the prototype 2026-03-27 08:27:52 +00:00
Ivan Krysak 69a6508ea7 mlir_lit_tests: introduce comment-specific tests 2026-03-27 08:21:58 +00:00
Ivan Krysak 8f747e7ff3 Name-import tests: be explicit about the ABI 2026-03-27 08:15:56 +00:00
Ivan Krysak 9524312ed4 C-Attributes: test typed arguments 2026-03-27 08:07:41 +00:00
Ivan Krysak 58257d5520 ImportModel: import _ABI 2026-03-27 08:04:10 +00:00
Ivan Krysak 105f15266b ImportModel: import _CAN_CONTAIN_CODE 2026-03-27 07:19:27 +00:00
Ivan Krysak f4e7d50361 Allow C-Attributes to accept integers and types 2026-03-27 07:18:49 +00:00
Ivan Krysak 73ddff7126 import-clift-types -> import-types 2026-03-26 16:28:48 +00:00
Ivan Krysak addfc543c8 import-model-names -> import-descriptive-info 2026-03-26 16:28:48 +00:00
Ivan Krysak 6dc6ad2cf3 importModelTypes -> importTypes 2026-03-26 16:28:48 +00:00
Ivan Krysak 1310b6ab62 Clift: append C to attribute classes we emit 2026-03-25 15:42:13 +00:00
Andrea Gussoni d54b8a5767 EmitFieldAccesses: array access argument and shl
Add two unit tests having a `struct` -> `array` -> `struct` access,
where the offset is computed through a `shl` operation, applied to a
dynamic index obtained through the argument.
2026-03-23 17:34:50 +01:00
Andrea Gussoni 732c880cd8 EmitFieldAccesses: array access argument and mul
Add a unit test having a `struct` -> `array` -> `struct` access, where
the offset is computed through a `mul` operation, applied to a dynamic
index obtained through an argument.
2026-03-23 17:34:50 +01:00
Andrea Gussoni cd3a30d346 EmitFieldAccesses: nested array access
Add a unit test for nested `array`s access, and a unit test for nested
`array`s access with a `struct` indirection level in the middle.
2026-03-23 17:34:50 +01:00
Andrea Gussoni 5a4a281677 EmitFieldAccesses: array with dynamic index test
Add a `EmitFieldAccess` unit test where we perform am `array` access,
using a dynamic index contained in another field of the `struct`
containing the `array` itself.
2026-03-23 17:34:50 +01:00
Andrea Gussoni 9e2528faec EmitFieldAccesses: ptr_add LinearCombination test
Add a test exercising the `LinearCombination` propagation through
`composePtrAdd`.
2026-03-23 17:34:50 +01:00
Andrea Gussoni b70c9aed66 EmitFieldAccess: ptr_add.offset as BasePointer
Additional test in to struct-ptradd-access.mlir that exercises the case
where the base pointer information comes from the offset operand (RHS)
of ptr_add rather than the pointer operand (LHS).

This tests the relaxed assumptions introduced in the previous commit
where `composePtrAdd` no longer requires the `PointerOperand` to be an
`Address` `PA`.
2026-03-19 17:31:42 +01:00
Andrea Gussoni 02cc6923d0 EmitFieldAccesses: implement pass
The `EmitFieldAccesses` pass transforms `clift` by taking pointer-typed
expressions computed via integerr arithmetic with type-safe field
accesses and array accesses.

The transformation is split in three main phases:
1) `PointerArithmetic` computation.
2) `BestTraversal` computation.
3) `FieldAccess` `clift` rewrite.

The high level driver is implemented in the `EmitFieldAccesses` header
and cpp, while the nested 3 phases are implemented respectively in
`PointerArithmetic`, `BestTraversal` and `FieldAccessReplacement`.

The `computerPointerArithmetic` phase is concerned with taking a
pointer-typed `ExpressionOp`, called `PointerToReplace`, and expressing
it in a `BasePointer+Offset` form.

The `computeBestTraversal` phase is concerned with computing the best
traversal of the type pointed to by `BasePointer`, that can be used to
rewrite the pointer arithmetic in `clift` with just field accesses and
array subscripts.

The `replaceFieldAccess` phase takes the `Traversal` computed at the
previous step, and actually rewrites in `clift` the `PointerToReplace`
in terms of field accesses and array accesses w.r.t. the `BasePointer`.
2026-03-18 17:25:13 +01:00
Andrea Gussoni cbb3f03910 Clift: fix CastKind::Decay verify
Perform `dealias` over argument operand of the `CastOp`.
2026-03-18 17:25:13 +01:00
Giacomo Vercesi ce60d8a1e5 pypeline: integrate model migration infrastructure
When calling `Model.deserialize` the model will be automatically be
migrated if it needs to.
2026-03-05 14:23:06 +01:00
Giacomo Vercesi f4d0573017 pypeline: StorageProvider works with Model
Change the `get_model` and `set_model` interface of `StorageProvider` so
that it is responsibility of the `StorageProvider` to
serialize/deserialize the model before returning to the caller.
This is in preparation to the model migration being implemented, since
it's now a responsibility of the storage provider to deserialize it it
can trivially re-save it to disk if it is migrated.
2026-03-05 14:23:06 +01:00
Giacomo Vercesi a6a3e15e6b pypeline: add env prefixes
Add the `PYPELINE_` prefix to all the pypeline environment variables.
2026-03-05 14:23:06 +01:00
Giacomo Vercesi 704c5e9bbb TupleTree: fix broken operator==
The `operator==` for keyed `StructDefinitions` was the comparison
between keys, which is misleading.
2026-03-04 14:58:02 +01:00
Giacomo Vercesi 9287c51d27 pypeline: fix -C option
Instead of executing `chdir` when using the `-C` option, which is
fragile when other paths are involved, store the option in the click
context and propagate it to the required code paths that require knowing
what the base directory is.
2026-03-04 14:58:02 +01:00
Ivan Krysak 69d006382b Fix revng build with coroutines disabled 2026-02-25 12:47:27 +01:00
Giacomo Vercesi e14d1cee6f Add native runner tests 2026-02-16 10:34:39 +01:00
Giacomo Vercesi df4f8b76ac pypeline: re-organize tests
Split the tests into two subdirectories: those that work in pure python
and those that use the C++ bindings.
2026-02-16 10:34:39 +01:00
Giacomo Vercesi f1878f1194 pypeline: drop TraceRunner
Drop the `pypeline-trace-run` executable, as its purposed has been
superseded by the combination of `--debug` and native pipe/analysis
runners.
2026-02-16 10:34:39 +01:00
Giacomo Vercesi 27f419c8cd Fix and ban namespace clobbering
Fix an instance where the `std` namespace got clobbered into the global
namespace by accident.
2026-02-16 10:34:38 +01:00
Lauri Vasama 57c49e4302 Implement Clift statement comment import 2026-02-13 08:41:39 +02:00
Lauri Vasama 46e8b64bb9 Implement Clift backend statement comment emission 2026-02-13 08:41:39 +02:00
Lauri Vasama 60e1684127 Generalize CommentPlacementHelper 2026-02-13 08:41:39 +02:00
Lauri Vasama dd75b0aebc Improve Clift local name import
* Names are now only imported for locals with valid handles.
* Stack frame variable is now handled separately.
2026-02-13 08:41:39 +02:00
Lauri Vasama 8b709ea532 Assign local variable and label handles in Clifter
Handles were previously assigned in the model name import pipe, but as
they are not expected to change often, they can be improted in the
Clifter. Additionally, in some cases it may be desirable to emit C
using generated names instead of those imported from the model, in which
case no handles would be available.
2026-02-13 08:41:39 +02:00
Lauri Vasama 811478771e Make local op name attributes inherent
The handle was already inherent, but was not being used. Discardable
attributes were instead previously used for both handle and name.
2026-02-13 08:41:39 +02:00
Lauri Vasama 757f278134 Add ptml::DoxygenEmitter, CDoxygenEmitter
These streams allow emitting Doxygen comments in C and other languages
using various Doxygen style options.
2026-02-13 08:41:39 +02:00
Lauri Vasama 6b764ccbd1 Use octal escapes instead of hexadecimal
A hexadecimal escape sequences consume an unlimited number of
hexadecimal digits, causing invalid code emission in the case that the
sequence is followed by an unrelated hexadecimal digit.
2026-02-11 12:48:33 +02:00
Giacomo Vercesi d578bc6cd2 isolate: clone root module conservatively
In the new pipeline the root module is split off in its individual
isolated modules at the end of `isolate`. Before splitting, there are a
lot of global variables in the root module and only a small part is
going to be needed after splitting for each module. To avoid excessive
memory usage employ `ConservativeModuleCloner` in `Isolate` so that
only the needed global variables are actually cloned when splitting off.
2026-02-05 10:20:45 +01:00
Giacomo Vercesi d7e605fe63 pypeline: add debug mode
Add the functionality the the pipeline infrastructure and CLI to run
individual pipe and analyses as subcommands instead of in-process. This
allow better debuggability of individual pipes.
2026-01-29 15:00:12 +01:00
Giacomo Vercesi fd14580619 pipebox: improve signal handling
Improve the handling of signals by leveraging the `Py_AtExit`
functionality to trigger cleanup when the interpreter exits.
When receiving SIGINT handle it specially because exceptions in python
are only thrown when the interpreter is running and not when C code is.
2026-01-20 12:29:12 +01:00
Alessandro Di Federico d79c66cb1b Introduce getOrCreateGlobal 2025-12-22 11:34:47 +01:00
Giacomo Vercesi 7f7f9571a8 pypeline: implement container discarding
Add infrastructure to pypeline that allows containers to be notified
when they are being used last, this allows two things:
* `Pipe`s eagerly clearing those containers once they are done reading
  their contents
* `ScheduledTask`s clearing those out at the end of their execution in
  case the pipe did not do it
This overall should improve memory usage as container no longer take up
memory if they are no longer used as part of a `Schedule`.
2025-12-19 10:29:28 +01:00
Giacomo Vercesi a2a7e6442b pypeline: move PipelineNode.run
Inline the body of `PipelineNode.run` into `ScheduledTask.run`, making
`PipelineNode` a pure data structure. This also allows eliding all the
arguments since all of them as properties of `ScheduledTask`.
2025-12-19 10:18:10 +01:00
Giacomo Vercesi 9338ac3486 pypeline: expand ScheduledTask's constructor
Move the arguments with a longer lifetime from the arguments of
`ScheduledTask`'s `run` function to its constructor.
2025-12-19 10:17:37 +01:00
Alessandro Di Federico 5820908675 Remove and ban \file 2025-12-16 17:41:55 +01:00