Calling `mrb_gc_unregistor()` from `mrb_data_type::dfree` caused a use-after-free deep inside `mrb_close()`.
The impetus to investigate was <https://github.com/mruby/mruby/pull/6342#pullrequestreview-2292747530>.
Currently, when `mrb_close()` is called, all objects are destroyed first.
The process is done heap page by heap page, and when all objects belonging to a heap page are destroyed, the heap page is released.
If the next heap page contains `RData` objects, the `mrb_gc_unregistor()` function may be called from the `mrb_data_type::dfree` function.
At this time, the `mrb_gc_unregistor()` function gets an array object from a Ruby global variable.
If the array object belongs to a freed heap page, use-after-free is established by referencing this array object.
About the fixes.
First of all, there is the fact that the `mrb_gv_get()` function returns `nil` if `mrb->globals` is `NULL`.
Therefore, before destroying all objects, free `mrb->globals` and set `mrb->globals` to `NULL` at the same time.
Now the `mrb_gv_get()` function will return `nil` to the calling `mrb_gc_unregistor()` function and `mrb_gc_unregistor()` will do nothing more.
ref. https://github.com/mruby/mruby/issues/4618
As #6359 pointed out, calling const_missing hook from E_XXX_ERROR (that
calls mrb_exc_get_id()) can be an attack vector. Since E_XXX_ERROR is
supposed to be a defined error class, we think that the situation where
it is undefined and the const_missing hook is called should be detected
as an error; fix#6359
By adding a fast-path where we ignore boxed types we can gain a pretty substantial speedup of mrb_iv_get, making it about 25% faster during a standard optcarrot benchmark run.
NOTE: It is just mrb_iv_get that is that much faster, the whole benchmark seems to be about 3-5% faster with word boxing.
This will be a partial merge of #5317 with the following changes.
- Remove `iclass->iv_c` since `iclass->iv_c` is equivalent to `iclass->c`.
- `class_iv_ptr()` returns a single pointer instead of a double pointer.
Previously, for example, it was possible to retrieve the `String` class as follows:
```console
% bin/mruby -e 'p Comparable::Enumerable::Errno::GC::Kernel::Math::ObjectSpace::String'
String
```
Note that this patch affects the API function `mrb_const_get()`.
`mrb_class_find_path` resolves a `char*` pointer to a class name string
by calling `mrb_class_name`. It then allocates a new string with
capacity 40 to copy that `char*` into.
https://github.com/mruby/mruby/blob/e04184185ab43b94980550e850d8813a415fa438/src/variable.c#L1111-L1112
`mrb_class_name` resolves the class name via `class_name_str`, which
returns an `mrb_value` with type tag `MRB_TT_STRING` and backed by an
`RString*`. Then `mrb_class_name` extracts the `RSTRING_PTR`:
https://github.com/mruby/mruby/blob/e04184185ab43b94980550e850d8813a415fa438/src/class.c#L2133-L2134
That `RString*`-backed `mrb_value` ultimately comes from `mrb_class_path`
which resolves the string from the symbol table:
https://github.com/mruby/mruby/blob/e04184185ab43b94980550e850d8813a415fa438/src/class.c#L2111
The allocation of the target `str` after resolving the class name
`mrb_value` and extracting its pointer is fragile and assumes the
`RString*` is "static". If the `RString*` is not static, the
interleaving of extracting the `RSTRING_PTR` followed by a subsequent
allocation might result in the class name `mrb_value` being garbage
collected, which will leave the extracted pointer invalid.
Fix this bad interleaving by allocating the destination string first
before taking a raw pointer to an `RString*`.
Internal functions can only be called from within the library.
Functions listed in `mruby/internal.h` can be called from:
* core (src/*.c)
* gems (mrbgems/**/*.c)
But not from the application linked with `libmruby`.
Now `iv_get()` returns `pos+1` if it finds the entry, so you don't need
to call `iv_put()`. You can replace the entry value by assigning to
`t->ptr[pos-1]`.
This is a fundamentally simplified reimplementation of #5317
by @shuujii
Instead of having array of `struct iv_elem`, we have sequences of keys
and values packed in single chunk of malloc'ed memory. We don't have to
worry about gaps from alignment, especially on 64 bit architecture,
where `sizeof(struct iv_elem)` probably consumes 16 bytes, but
`sizeof(mrb_sym)+sizeof(mrb_value)` is 12 bytes.
In addition, this change could improve memory access locality.
close#5317
## Implementation Summary
* Only keys and only values of hash table are contiguous to eliminate
structure padding.
* Change upper limit of `iv_tbl` size to `UINT16_MAX` (it seems to be
acceptable in mruby because the total number of classes/modules
immediately after starting Redmine is 20,000 or less).
* `iv_tbl*` point hash buckets directly.
## Benchmark Summary
Only the results of typical situations on 64-bit Word-boxing are present
here. For more detailed information, including consideration, see below
report (although most of the body is written in Japanese).
* https://shuujii.github.io/mruby-iv-benchmark
### Memory Usage
Lower value is better.
| iv_tbl Size | Baseline | New | Factor |
|------------:|---------------:|---------------:|-----------:|
| 4 | 88B | 52B | 0.59091x |
| 30 | 536B | 388B | 0.72388x |
| 100 | 2072B | 1540B | 0.74324x |
| 200 | 4120B | 3076B | 0.74660x |
Although not mentioned in the above report, the memory usage of `mrbtest`
(full-core gembox) is as follows in the result by Valgrind.
* Baseline: 108,086 allocs, 16,313,122 bytes allocated
* New: 94,273 allocs, 15,875,214 bytes allocated
### Performance
Higher value is better.
#### `mrb_obj_iv_set`
| iv_tbl Size | Baseline | New | Factor |
|------------:|---------------:|---------------:|-----------:|
| 4 | 88.63003M i/s | 92.60611M i/s | 1.04486x |
| 30 | 32.97066M i/s | 25.25095M i/s | 0.76586x |
| 100 | 16.33224M i/s | 22.74998M i/s | 1.39295x |
| 200 | 5.64484M i/s | 6.79949M i/s | 1.20455x |
#### `mrb_obj_iv_get`
| iv_tbl Size | Baseline | New | Factor |
|------------:|---------------:|---------------:|-----------:|
| 4 | 217.58391M i/s | 237.59912M i/s | 1.09199x |
| 30 | 139.56195M i/s | 160.49470M i/s | 1.14999x |
| 100 | 143.09716M i/s | 190.95047M i/s | 1.33441x |
| 200 | 89.75291M i/s | 134.78717M i/s | 1.50176x |
### Binary Size
Lower value is better.
| File | Baseline | New | Factor |
|:------------|---------------:|---------------:|-----------:|
| mruby | 697,520B | 697,520B | 1.00000x |
| libmruby.a | 1,046,570B | 1,046,682B | 0.99989x |
## Note
The address in `struct RObject::iv` may change after initialization because
`iv_tbl*` points directly to hash buckets. Therefore, the address cannot be
copied and shared when include/prepend. So, when sharing `iv_tbl`, refer to
it via the sharing source class. As a result, the following bug have also
been fixed.
* [An `iv_tbl` is not shared when a class includes or prepends an empty module](https://gist.github.com/shuujii/0ac23fa24b0c55b2c602b534d81e4a95)
Instead of including `mruby/presym.h` everywhere, we provided the
fallback `mruby/presym.inc` under `include/mruby` directory, and specify
`-I<build-dir>/include` before `-I<top-dir>/include` in `presym.rake`.
So even when someone drops `-I<build-dir>/include` in compiler options,
it just compiles without failure.
dearblue