Add experimental support for the [Typed Function References](https://github.com/WebAssembly/function-references)
proposal.
Closes https://github.com/wazero/wazero/issues/2483, follows up to the
refactoring in https://github.com/wazero/wazero/pull/2495 and prepares
to WasmGC.
Typed function references extend WebAssembly's type system with
non-nullable reference types and concrete function type indices (`(ref
$t)`, `(ref null $t)`), enabling direct calls through typed references
(`call_ref`, `return_call_ref`) and null-aware branching (`br_on_null`,
`br_on_non_null`, `ref.as_non_null`).
Excluding tests and spec suites, the feature amounts to roughly 1,900
lines of code.
Feature flag: `experimental.CoreFeaturesTypedFunctionReferences`
## What's the use for this?
Sadly, very little. This proposal is a pretty much just a prerequisite
for the GC proposal.
On the flip side, it completes the exception handling spec
(https://github.com/wazero/wazero/pull/2489):
1. two EH spec tests were previously skipped because they required
distinguishing nullable from non-nullable references. Those tests now
pass.
2. The fuzzer is now enabled for both exception handling and typed
function references. It could not be enabled for EH because the fuzzer
would generate func refs.
So I guess, technically, it has a use :D
## Type System
`ValueType` (already `uint64` introduced in
https://github.com/wazero/wazero/pull/2495) is now extended with bit
flags to encode nullability and concrete type indices:
| Bits | Purpose |
|-------|---------------------------------|
| 0-7 | Base type byte (same as before) |
| 8 | Non-nullable flag |
| 9 | Concrete ref flag |
| 32-63 | Type index (for `(ref $t)`) |
This encoding should be fine for WasmGC too and should not require
further changes in the near future.
Subtyping rules: non-nullable is a subtype of nullable (same
kind/index); concrete function refs `(ref $t)` are subtypes of
`funcref`.
## New Instructions
| Opcode | Hex | Description |
|-------------------|--------|----------------------------------------------------------|
| `call_ref` | `0x14` | Indirect call through a typed function reference
|
| `return_call_ref` | `0x15` | Tail-call variant of `call_ref` |
| `ref.as_non_null` | `0xd4` | Assert ref is non-null, trap otherwise |
| `br_on_null` | `0xd5` | Branch if null, push non-null ref on
fall-through |
| `br_on_non_null` | `0xd6` | Branch if non-null (carrying the ref),
fall-through on null |
## Validation
Tricky bits:
- Non-nullable local initialization tracking: `local.get` on a
non-nullable ref local is rejected unless a `local.set`/`local.tee` has
been executed in the same or enclosing block scope. State is
saved/restored at block boundaries per the spec (needed an additional
field to keep track of init'd values)
- `ref.func` now pushes `(ref $t)` (the concrete non-nullable type of
the referenced function) onto the validation type stack instead of plain
`funcref`, so that passing it to `call_ref` type-checks without an
upcast.
- Block types, element segments, table types, and const expressions all
support concrete ref types.
- The type section now validates forward references (standalone types
can only reference previously defined types; rec group members can
reference each other).
## Interpreter
The five new opcodes are compiled to new IR operations and executed in
the interpreter loop. `call_ref` / `return_call_ref` load the function
instance from the opaque reference pointer, null-check, and dispatch.
`br_on_null` / `br_on_non_null` pop the reference, check nullity, and
either branch or fall through with the appropriate stack state.
## Compiler (wazevo)
Implemented entirely as SSA-level lowering with no backend-specific
code:
- `call_ref` / `return_call_ref`: load executable and module context
pointers from the function instance, null-check via
`ExitIfTrueWithCode(ExitCodeNullReference)`, then dispatch as an
indirect call.
- `br_on_null` / `br_on_non_null`: compare against zero, branch with
trampoline blocks for try-table exits and listener support.
- `ref.as_non_null`: null-check with trap.
## Binary Decoding
- `decodeRefType` helper extracted and shared across `value.go`,
`element.go`, `table.go`, and `code.go` for consistent handling of `(ref
null ht)` / `(ref ht)` prefixes.
- Tables support the `0x40 0x00` prefix for initializer expressions
(required for non-nullable table element types).
- `DecodeBlockType` handles concrete ref types as block results.
## Cross-Module Linking
`call_indirect` uses `FunctionTypeID` for fast runtime type checks. The
existing `FunctionType.key()` method builds the key from raw `ValueType`
bytes, but with concrete refs `(ref $0)` in module A and `(ref $0)` in
module B may refer to structurally identical types at different local
indices. `structuralTypeKey` fixes this by replacing local type indices
with the already-assigned `FunctionTypeID` of the referenced type, so
two modules with the same structural signature share a single
`FunctionTypeID`.
## Spec Suite
The spec test suite uses `wasm-tools json-from-wast` (same as exception
handling). All 22 test files pass.
## Fuzzing
The fuzzer (`nodiff`) now enables both `CoreFeaturesExceptionHandling`
and `CoreFeaturesTypedFunctionReferences`. Dummy import generation
handles `exnref` and concrete ref types.
---------
Signed-off-by: Edoardo Vacchi <evacchi@users.noreply.github.com>
Add experimental support to the Exception Handling spec.
Exception handling adds structured error propagation to WebAssembly
through `tags` (typed exception signatures), `throw/throw_ref` (raising
exceptions), and `try_table` (catching exceptions with typed `catch`
clauses). A new value type, `exnref`, represents a reference to a caught
exception.
Excluding tests and spec suites, the entire feature amounts to a little
less than 2000 lines of code.
Feature flag: `experimental.CoreFeaturesExceptionHandling`
## What's the use for this?
In the spirit of always having a real use case for the specs we
implement, I did check this implementation against the one large Wasm
C++ codebase I am aware about, i.e.
https://github.com/klippa-app/go-pdfium, and I verified that the entire
suite, compiled with `emcc+wasm-opt` does work; incidentally, much of
the existing support code, based on Emscripten's longjmp/setjmp, can be
dropped (thx @jerbob92 for the help!)
Broadly speaking, I don't know how much EH-enabled Wasm code you will
find in the wild. For instance, code compiled using Clang's C++ backend
does not really throw new-style exceptions, unless you compile WASI-SDK
with some flags, or you build with `emcc` and translate old-style
exceptions using `wasm-opt` (see exceptions_test.go)
In short, this spec should be really seen as a step towards supporting
the GC spec. The design decisions (esp. regarding the test suite) follow
from this reasoning.
## Spec Suite
`wast2json` **does not** support the EH suite, but `wasm-tools` does,
albeit with a tiny quirk in the way negative numbers are rendered. In
this PR _I am not_ moving the other suites to `wasm-tools` though.
The spec test suite passes entirely, except for two tests that are
intentionally skipped.
These tests assume **a very small subset** of typed references is
implemented; specifically, the distinction between nullable and
non-nullable function references (`(ref null $t)` vs `(ref $t)`). I have
decided to just flatten all non-nullable refs to nullable refs at decode
time, so we cannot detect the type mismatch the spec expects in these
cases.
Passing the **full suite is actually straightforward** because it only
requires introducing a non-nullable variant for each ref type: strictly
speaking the 2 failing tests are only for related nullable vs
non-nullable **func refs** -- so I can special-case these and/or add
non-nullable types for each known value; however, we will **need** to
rethink the representation of Values for the GC proposal anyway (because
this will introduce custom type indices, and these won't fit our
currently byte-sized `ValueType` enum).
The subtype-checking functions (`isRefSubtypeOf`,
`isStrictRefSubtypeOf`) are kept as abstraction points but currently
they are just equalities.
Nevertheless, as proven by the CPP example and
https://github.com/klippa-app/go-pdfium this PR provides a starting
point that is already useful (even if it is for limited cases).
## Interpreter
Exception handling in the interpreter:
**Within a frame**, `try_table` blocks are compiled into a static
exception table with PC ranges. When `throw` or `throw_ref` executes,
`searchExceptionTable` scans backwards (inner handlers first) for a
catch clause whose PC range covers the current instruction. If a match
is found, `applyExceptionHandler` adjusts the operand stack depth and
jumps to the catch target PC directly.
**Across frames**, if no handler is found in the current frame,
`panic(&thrownException{...})` propagates up the Go call stack. Each
cross-function call goes through `callWithUnwind()`, which uses
`defer/recover` to intercept the panic. Its `canRestore` method unwinds
`ce.frames` to the caller's depth and searches that frame's exception
table. If a handler is found, `doRestore` applies it and
`callWithUnwind` returns `true` (frame unwound -- caller refreshes
locals and continues). If not, the panic re-propagates to the next outer
`callWithUnwind`.
Note on the implementation: this mechanism is unified with the existing
`snapshot/restore` API through the `restorable` interface: both
`*thrownException` and `*snapshot` implement `canRestore`/`doRestore`,
so `callWithUnwind` handles both with a single defer/recover path. The
short-circuit check (`len(exceptionTable) == 0 && no snapshotter`) skips
the defer/recover overhead entirely for frames that don't need it.
## Compiler
The compiler could not reuse the snapshot/restore mechanism directly,
but it follows a similar pattern. One cool thing is that exception
handling could be implemented entirely as an SSA-level lowering with new
exit codes; there is no backend-specific code.
Throwing uses a two-phase ABI:
1. `ExitCodeThrowAlloc`: exits to the Go runtime to allocate an
`Exception` struct on the heap and writes a pointer to its params array
into the execution context. Then we return control to compiled code,
which stores the tag parameters directly into the struct.
2. `ExitCodeThrow`: exits to the Go runtime again to search for a
matching handler, restore the stack checkpoint, and branch to the catch
target.
The idea is that we NEED to allocate the parameter slice dynamically, so
we delegate to the Go runtime. Incidentally, this also allows us to
unify `throw` and `throwRef`; in the latter case, we skip the `alloc`
phase, and we exit straight to `ExitCodeThrow`.
### Trampolines
The compiler uses four trampolines to return to host to manage exception
state:
| Trampoline | Exit Code | Purpose |
|---|---|---|
| **ThrowAlloc** | `ExitCodeThrowAlloc` | Allocates an `Exception`
struct on the Go heap, writes `&exn.Params[0]` into `execCtx` so
compiled code can store tag parameters directly. |
| **Throw** | `ExitCodeThrow` | Searches `tryHandlers` for a matching
catch clause, restores the stack checkpoint on match, sets
`caughtExceptionClauseIdx` for the compiled dispatch. Used for both
`throw` (after ThrowAlloc) and `throw_ref` (directly). |
| **TryTableEnter** | `ExitCodeTryTableEnter` | Clones the current stack
as a checkpoint, pushes a `tryHandler` with the catch clause table and
saved module instance. Sets `caughtExceptionClauseIdx = -1` (no
exception). |
| **TryTableLeave** | `ExitCodeTryTableLeave` | Pops the most recent
`tryHandler`. Emitted at `try_table` block ends, and before any branch
or return that exits a `try_table` scope. |
### Lowering
As mentioned earlier, this entirely resolved at the SSA-level. In
particular:
- **Entry**: when a `try_table` is encountered, the compiler emits a
call to the TryTableEnter trampoline. Control returns to compiled code,
which reads `caughtExceptionClauseIdx` from the execution context — if
it's -1 (no exception), execution continues into the `try_table` body.
- **Normal exit**: when execution reaches the `End` opcode of the
`try_table` block, a TryTableLeave trampoline call is emitted.
However, every TryTableEnter must have a matching TryTableLeave on every
control flow path that doesn't throw; so we also "TryTableLeaves" also
for the following:
- **Early exit (branch)**: a `br`, `br_if`, or `br_table` can jump to a
label outside the `try_table`, skipping its `End`.
`emitTryTableLeaves(depth)` walks the control frame stack and emits one
TryTableLeave for each `try_table` frame the branch crosses. For
`br_if`, this happens in a trampoline basic block that only executes on
the taken path.
- **Early exit (return)**: `return`, `return_call`, and
`return_call_indirect` call `emitTryTableLeaves` with the full control
stack depth, popping all active handlers before leaving the frame.
- **Throw**: on `throw`/`throw_ref`, compiled code exits to the dispatch
loop via ExitCodeThrow. `doHandleException` searches `tryHandlers`
innermost-to-outermost. On match, it restores the cloned stack
checkpoint, sets `caughtExceptionClauseIdx`, and re-enters compiled code
at the handler's return address. The compiled dispatch branches to the
matching catch clause's handler block.
### Caveats
- As described above, we are skipping 2 spec tests (will be addressed
when we implement typed references)
- Because of that, I am _not_ enabling by default the fuzzer: I would
have to special-case for typed references and/or TODOs I have left
behind; I did run the fuzzer with some special-casing while debugging
the issue with pdfium (#2488) and did not find issues (the fuzzer did go
frequently OOM though); in fact, the pdfium issue was actually unrelated
(!)
---------
Signed-off-by: Edoardo Vacchi <evacchi@users.noreply.github.com>
This pull request implements WebAssembly extended constant expressions
throughout the codebase.
Previously, constant expressions were represented as a single opcode,
with an accompanying data slice that was interpreted in a variety of
places according to the opcode. To implement extended constant
expressions, this representation needs to change to support multiple
opcodes with accompanying data. To accomplish this, constant expressions
are changed to reuse the binary representation of an expression as a
single data field. The constant expression was evaluated differently in
a number of contexts (i.e. at module instantiation time, and also at
validation time prior to a module instance existing), so to maximize
reusability, an Evaluate method was added to ConstantExpression to allow
for the various places a ConstantExpression is evaluated to call the
method, plugging in their own logic for resolving globals and function
references (since depending on where the constant expression is used
this logic can differ a good deal).
The most important changes are:
**Constant Expression Representation and Encoding:**
* Refactored `wasm.ConstantExpression` to use a single `Data` field
containing the opcode, operands, and `OpcodeEnd`, removing the separate
`Opcode` field throughout the codebase. This affects all construction
and usage of constant expressions.
**Test and Integration Code Updates:**
* Updated all test cases and integration tests to construct constant
expressions using the new format, including global, element, and data
sections in various modules and test helpers.
**Element and Global Section Handling:**
* Changed element section initialization to use arrays of constant
expressions (with full op/data/end encoding) instead of simple indices,
and updated related encoding logic to handle this new representation.
---------
Signed-off-by: Nuno Cruces <ncruces@users.noreply.github.com>
Co-authored-by: Nuno Cruces <ncruces@users.noreply.github.com>
Previously, only specification tests are run separately
from the rest of the tests just to run in a separate
GHA workflow (named spectest) to have a badge to brag.
This used to unnecessarily use CI resources, and no need to
do so anymore.
Signed-off-by: Takeshi Yoneda <t.y.mathetake@gmail.com>
vs was introduced to bench various things including
the tests against other runtimes. However, the
cases are not representative, and also some are
just unable to build plus some are simply failing.
This removes the entire vs directory, and reduces
the maintenance burden.
Signed-off-by: Takeshi Yoneda <t.y.mathetake@gmail.com>
Cross-platform msitools do not support the `Environment` directive,
which should be the proper way to update %PATH%, so we adapt
the Makefile to assume a Windows environment when building, addressing #1374.
`make dist` now runs on all platforms, but on Windows, assuming
`candle.exe`, `light.exe` and GNU `tar` are installed, it builds
all tarballs, zip and msi packages.
On other platforms it will only build the tarballs.
We also address some other issues with msitools
and change the GUID for UpgradeCode which caused
the wazero package to be mistaken for func-e
when installing/uninstalling both on the same
machine.
Signed-off-by: Edoardo Vacchi <evacchi@users.noreply.github.com>
This fixes a build problem on solaris and illumos. This also closes
issue #1106 because we can't make everything build with CGO (e.g. our
examples won't build unless CGO is available).
Signed-off-by: Adrian Cole <adrian@tetrate.io>
platform: Allows sysfs to implement utimesns natively
This moves away from `syscall.UtimesNano` as it has intentionally
avoided common features in POSIX, such as handling UTIME_NOW and
UTIME_OMIT. When we eventually expose this API, users will be free to
override `UTIME_NOW` with a fake clock, possibly the same that was
supplied to wazero's `ModuleConfig`.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
Co-authored-by: Edoardo Vacchi <evacchi@users.noreply.github.com>
Before, our README said gojs `GOOS=js compiled wasm` was experimental.
However, as we head to 1.0 we should be more explicit about that.
When we started gojs, there was no likely future where `GOOS=wasi` would
happen in the standard go compiler. This has changed, so we'll only keep
the gojs package around until wasi is usable for two Go releases. Being
in an experimental package helps others know to watch out for this.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
It looks like $(subst from,to,text) works differently on BSD make
vs GNU make. The behavior we expected for
$(subst,$(space),$(comma),$(var))
is instead the behavior of `$(patsubst ...)` on GNU Make.
The alternative syntax `$(var: =,)` seems portable across the two
and shows the same behavior.
Signed-off-by: Edoardo Vacchi <evacchi@users.noreply.github.com>
Edoardo Vacchi