Split the `InlineHelpers` pass into the three phases:
- `link-helpers-to-inline` links the missing `revng_inline` helper
bodies from the `libtcg-helpers-to-inline-arch` module.
- `inline-helpers` inlines, in every function except those tagged
`Root` or `Helper`, the calls to `revng_inline` helpers whose
critical arguments on the call site are constant.
- `delete-helper-bodies` drops from the module the body of every
`revng_inline` function, leaving only the declaration.
The three passes live in a dedicated `revngInlineHelpers` library
(`lib/InlineHelpers/` + `include/revng/InlineHelpers/`), so that
libraries which do not depend on `revngFunctionIsolation` can use them
without creating cyclic dependencies.
Implement compression of objects before they are saved into the storage
provider. Each container type can specify which algorithm to use
(currently `none` or `zstd`) and the compression level.
Three closely-coupled changes that have to land together to keep
every caller compiling:
- Introduce a per-instance ExtraState recording surface (with a
NoExtraState default) and thread it through applyTransferFunction.
- Replace getMaximalFixedPoint's positional arg list with a single
MFPConfiguration struct, so callers spell out only the fields they
need.
- Replace the EntryLabels pointer with an Entry/All/vector* variant
so callers can ask for the graph entry, every node, or a custom
list without juggling extra arguments.
Update every existing caller for the new signatures and switch
Liveness::GraphType to a plain Function* (with the inverse
GraphTraits selected at call sites).
Introduce a pass which statically demotes some helpers from candidates
for inlining:
1) Helpers which are part of SCC on the callgraph.
2) Helpers which contains an `insertvalue`.
3) Helpers which _fail_ the `CriticalArguments` criterion.
A critical argument is defined starting from the critical operand
definition.
A critical operand is either the condition of a `switch` instruction or
the indices of a `getelementptr` instruction.
If the dataflow computing the the operand trace back to a formal
parameter of the function, that parameter will be considered critical
when attempting the inlining of the helper.
In case each condition is satisfied, the `revng.inline.policy` metadata
is attached to the function, so that it can be used at _inline_ time to
take the decision based on the critical arguments constantness.
The metadata is encoded as an `iN` integer with N = `arg_size() + 1`:
the extra most-significant bit is always zero so LLVM's signed-decimal
`iN` printer renders the value positively (avoiding e.g. `!{i2 -2}`).
The (de)serialisation primitives live in `revng/Support/IRHelpers.h`
as `serializeInliningPolicy` / `deserializeInliningPolicy` so the
runtime `revngFunctionIsolation` doesn't need to link the build-time
`revngHelperInliningAnalyses` to read the metadata back.
Introduce a build-time pass that tags every QEMU helper transitively
reaching a function defined under one of the configured runtime-library
directories (currently `fpu/` only) as `revng_inline`. The intent is to
make the lift pipeline surface "leaf-level" runtime-library calls (e.g.
softfloat ops) directly in the lifted IR by inlining away every wrapper
helper that sits between QEMU's `helper_*` boundary and the leaf.
We also gate the tagging by a per-function body-size budget.
The pass is run in the pipeline building the `libtcg-helpers-full-*.bc`
artifacts.
Switch out from using uvicorn to hypercorn for the pypeline daemon, make
the facilities create hypercorn commands in stand-alone functions, as
they will be re-used for other servers later on.
Expand the `ProjectMetadata` dataclass to include additional timestamps
and the hash of the pipeline description. Change all the
`StorageProvider`s so that all the new data is saved and returned.
Consolidate the saving of object dependencies and saving the actual
objects' data into a single function called `add_objects`. The previous
methods `add_dependencies`, `add_custom_invalidation_data` and `put`
have been removed.
Due to the drifting differences, split `ScheduledTask` into
`PipeScheduledTask` and `SavepointScheduledTask`, which allows each to
specialize on the task it has to accomplish.
Make the issuing of notification completely out of band with respect to
the `Pipeline`. Now the `StorageProvider` is responsible for providing
notifications to clients. Since some providers are local-only, there is
a `LOCAL_QUEUE` which allows sensing notifications through a local
queue, re-using the local revng daemon.
Split the `invalidate` method in two phases: in `invalidateCheck` it
just checks if the model diff warrants the execution of the actual
`invalidate` method, which requires fetching the custom invalidation
data from storage. The actual `invalidation` method remains the same.
This should make invalidation faster for storage providers that store
the invalidation remotely.
Add three more unit tests that mirror the previous "with_alloca"
variants but where the store of the extra argument %val into the
pre-existing alloca happens BEFORE the call instead of after it. The
expected number of NEW allocas after the pass is the same as in the
corresponding store-after-call variant: the position of the store
relative to the call does not change the picker's decision.
Add three more unit tests, one for each of the three call patterns
introduced in the previous commit. Each test takes an extra i64
argument %val, has an alloca pre-existing in the entry block, stores
%val into the alloca right after the call, and at least one branch
returns by loading from the alloca.
The pre-existing alloca/store/load pattern does not write to memory
the call could read (the alloca is local and the call's pointer
argument is unrelated) and the call cannot write to the alloca, so
the picker should reach the same conclusion as in the simpler
counterpart tests: the variant of @call_rw_one_use generates no new
alloca, @call_rw_two_uses generates one new alloca alongside the
pre-existing one, and @call_ro_two_uses generates none.
Cover three representative cases for how the picker should treat
function calls in non-legacy mode:
- A call to a read+write function with a single use (an icmp that
feeds a conditional branch). The call has only one use, so the
picker does not pick it and no new alloca is created.
- The same shape, but with two uses of the call (the icmp and a
return). Because the function may read+write memory and the call
has more than one use, the picker picks the call: a new alloca is
created at the top of entry, the result of the call is stored into
it, and every user of the call uses a load from the alloca.
- The same shape as the second case but with a function that only
reads memory (and is also nounwind willreturn so that
Instruction::mayHaveSideEffects returns false). The picker does
not pick the call and no new alloca is created.
Make cannotBePointer recurse through an Add when both of its
operands cannot be pointers. This lets canDisambiguatePointerOperand
identify the only viable base pointer of an outer Add even when the
offset side is itself an Add (or a deeper tree of Adds) whose leaves
are all clearly-non-pointer values, e.g.
%offset = add i64 (%a*%b), (%c*%d)
%result = add i64 %arg, %offset
Two unit tests cover the simple two-mul case and a deeper
four-mul binary-tree variant.
Cover several scenarios where a single base pointer is reachable
only after the discovery walk in ArithmeticToGEPPass propagates
through many layers of Add instructions, each of which has
ambiguous pointer operands and must be disambiguated separately:
- A chain of 8 chained adds with a varied set of clearly-non-pointer
offset opcodes (mul, ashr, shl, and, or, xor, sdiv, srem), with
discovery seeded from the !revng.pointers metadata on the return.
- A chain seeded from a pointer argument (forward propagation).
- A chain feeding the integer argument of a call whose argument is
a pointer (backwards discovery from the call site).
- A chain where the pointer side alternates between operand 0 and
operand 1 of each add.
- A chain whose offset side is hidden behind freeze instructions,
so the cannotBePointer recursion through transparent casts is
exercised.
- A chain where every add reuses the same multiplication as offset,
exercising independent disambiguation per-add over a shared
sub-expression.
- A chain interrupted by ptrtoint/inttoptr round-trips between adds.
Introduce the ArithmeticToGEP canonicalisation pass. It detects
pointers (LLVM-typed pointer values, model pointers flagged by the
"revng.pointers" metadata, and pointer-tagged extract values) and
rewrites integer arithmetic computed on top of them as i8 GEPs,
followed by a ptrtoint when the original use needs an integer.
For each obvious pointer the pass also walks backwards through Add
chains and disambiguates the pointer operand of each Add when the
other operand cannot be a pointer, so the rewrite extends to integer
arithmetic that only later flows into a pointer use.
Add an llvm-lit test suite for the new-PassManager
SwitchToStatements pass, exercised via
`revng opt -passes=switch-to-statements-test` (registered by the
pass's registerCallbacks hook). The tests cover load/store
serialisation, side-effecting calls and a few representative
patterns where an instruction must be picked for serialisation
because its memory read is no longer available at one of its uses.
Andrea Gussoni