Commit Graph

12446 Commits

Author SHA1 Message Date
KOBAYASHI Shuji 98d091436d Reduce memory usage of instance variable table
## Implementation Summary

* Only keys and only values of hash table are contiguous to eliminate
  structure padding.
* Change upper limit of `iv_tbl` size to `UINT16_MAX` (it seems to be
  acceptable in mruby because the total number of classes/modules
  immediately after starting Redmine is 20,000 or less).
* `iv_tbl*` point hash buckets directly.

## Benchmark Summary

Only the results of typical situations on 64-bit Word-boxing are present
here. For more detailed information, including consideration, see below
report (although most of the body is written in Japanese).

* https://shuujii.github.io/mruby-iv-benchmark

### Memory Usage

Lower value is better.

| iv_tbl Size |    Baseline    |       New      |   Factor   |
|------------:|---------------:|---------------:|-----------:|
|           4 |            88B |            52B |   0.59091x |
|          30 |           536B |           388B |   0.72388x |
|         100 |          2072B |          1540B |   0.74324x |
|         200 |          4120B |          3076B |   0.74660x |

Although not mentioned in the above report, the memory usage of `mrbtest`
(full-core gembox) is as follows in the result by Valgrind.

* Baseline: 108,086 allocs, 16,313,122 bytes allocated
* New:       94,273 allocs, 15,875,214 bytes allocated

### Performance

Higher value is better.

#### `mrb_obj_iv_set`

| iv_tbl Size |    Baseline    |       New      |   Factor   |
|------------:|---------------:|---------------:|-----------:|
|           4 |  88.63003M i/s |  92.60611M i/s |   1.04486x |
|          30 |  32.97066M i/s |  25.25095M i/s |   0.76586x |
|         100 |  16.33224M i/s |  22.74998M i/s |   1.39295x |
|         200 |   5.64484M i/s |   6.79949M i/s |   1.20455x |

#### `mrb_obj_iv_get`

| iv_tbl Size |    Baseline    |      New       |   Factor   |
|------------:|---------------:|---------------:|-----------:|
|           4 | 217.58391M i/s | 237.59912M i/s |   1.09199x |
|          30 | 139.56195M i/s | 160.49470M i/s |   1.14999x |
|         100 | 143.09716M i/s | 190.95047M i/s |   1.33441x |
|         200 |  89.75291M i/s | 134.78717M i/s |   1.50176x |

### Binary Size

Lower value is better.

|    File     |    Baseline    |      New       |   Factor   |
|:------------|---------------:|---------------:|-----------:|
| mruby       |       697,520B |       697,520B |   1.00000x |
| libmruby.a  |     1,046,570B |     1,046,682B |   0.99989x |

## Note

The address in `struct RObject::iv` may change after initialization because
`iv_tbl*` points directly to hash buckets. Therefore, the address cannot be
copied and shared when include/prepend. So, when sharing `iv_tbl`, refer to
it via the sharing source class. As a result, the following bug have also
been fixed.

* [An `iv_tbl` is not shared when a class includes or prepends an empty module](https://gist.github.com/shuujii/0ac23fa24b0c55b2c602b534d81e4a95)
2021-02-03 23:16:09 +09:00
Yukihiro "Matz" Matsumoto d759a73525 Allow more than 256 child irep; fix #5310
We have introduced following new instructions.

 * `OP_LAMBDA16`
 * `OP_BLOCK16`
 * `OP_METHOD16`
 * `OP_EXEC16`

Each instruction uses 16 bits operand for `reps` index. Since new
instructions are added, `mruby/c` VM should be updated.

Due to new instructions, dump format compatibility is lost, we have
increased `RITE_BINARY_MAJOR_VER`.

In addition, we have decreased the size of `refcnt` in `mrb_irep` from
`uint32_t` to `uint16_t`, which is reasonably big enough.
2021-02-01 16:20:58 +09:00
Yukihiro "Matz" Matsumoto bd6b48fa3b Copy iv table with #prepend; fix #5309
In addition, stop eager allocation of `mt` table.
2021-02-01 14:33:28 +09:00
Yukihiro "Matz" Matsumoto d0ba7b95a9 Merge pull request #5314 from dearblue/presym_init
Introduced `MRB_PRESYM_INIT_SYMBOLS()`
2021-02-01 11:19:13 +09:00
Yukihiro "Matz" Matsumoto 5cce4ad433 Merge pull request #5315 from shuujii/use-mrb_open_core-instead-of-mrb_open--in-mrbc
Use `mrb_open_core()` instead of `mrb_open()`  in `mrbc`
2021-02-01 10:12:29 +09:00
Yukihiro "Matz" Matsumoto 874f828a7a Merge pull request #5313 from dearblue/diet-5272
Remove unnecessary `ci0` variables; ref #5272
2021-02-01 09:36:11 +09:00
KOBAYASHI Shuji 1fc8e00e4a Use mrb_open_core() instead of mrb_open() in mrbc 2021-01-31 22:37:27 +09:00
Yukihiro "Matz" Matsumoto 406159d412 Merge pull request #5312 from dearblue/diet-5060
Remove unnecessary configuration macros; ref #5060
2021-01-31 19:57:31 +09:00
dearblue a9949f5da4 Introduced MRB_PRESYM_INIT_SYMBOLS()
The `init_SYMBOLS()` function implicitly defined in `MRB_PRESYM_DEFINE_VAR_AND_INITER()` requires some familiarity when trying to find it from the caller.
By introducing `MRB_PRESYM_INIT_SYMBOLS()`, it is possible to find directly from the identifier.
2021-01-31 18:13:28 +09:00
dearblue d824488915 Remove unnecessary ci0 variables; ref #5272 2021-01-31 17:24:16 +09:00
dearblue 5ede25c40f Remove unnecessary configuration macros; ref #5060
The following macros will be removed:
- `ENSURE_STACK_INIT_SIZE`
- `RESCUE_STACK_INIT_SIZE`
- `MRB_ECALL_DEPTH_MAX`
2021-01-31 17:22:27 +09:00
Yukihiro "Matz" Matsumoto cee916ab0f Merge pull request #5311 from shuujii/change-.o.d-back-to-.d-because-.pi.d-is-no-longer-created
Change `.o.d` back to `.d` because `.pi.d` is no longer created
2021-01-31 15:25:58 +09:00
KOBAYASHI Shuji d90abc648e Change .o.d back to .d because .pi.d is no longer created 2021-01-31 15:08:37 +09:00
Yukihiro "Matz" Matsumoto 02e336ee27 Merge pull request #5308 from dearblue/internal-mrbc
Build internal mrbc in an internal directory
2021-01-30 15:43:50 +09:00
Yukihiro "Matz" Matsumoto dd603d494d Merge pull request #5306 from dearblue/unpack
Fixed `String#unpack` to handle the highest range of integer values
2021-01-29 23:51:26 +09:00
dearblue 14e07f721a Build internal mrbc in an internal directory 2021-01-29 23:02:17 +09:00
dearblue b500e82950 Remove unnecessary range confirmation
This is a correction based on the review by @matz.
https://github.com/mruby/mruby/pull/5306#pullrequestreview-578378401
2021-01-29 22:33:03 +09:00
dearblue 9c36499e31 Fixed String#unpack to handle the highest range of integer values
Previously, problems occurred when the `fixnum` was exceeded.

- 32-bit cpu mode with `MRB_WORD_BOXING` and `MRB_INT32`:

  ```console
  % bin/mruby -e 'p [0x7fffffff].pack("N").unpack("N")'
  trace (most recent call last):
  -e:1: cannot unpack to Integer: 2147483647 (RangeError)
  ```

- 64-bit cpu mode with `MRB_WORD_BOXING` and `MRB_INT64`:

  ```console
  % bin/mruby -e 'p [0x7fffffff_ffffffff].pack("q").unpack("q")'
  trace (most recent call last):
  -e:1: cannot unpack to Integer: 9223372036854775807 (RangeError)
  ```
2021-01-29 22:27:36 +09:00
Yukihiro "Matz" Matsumoto 825241205f Replace mrb_intern_cstr to mrb_intern_lit. 2021-01-29 18:32:26 +09:00
Yukihiro "Matz" Matsumoto 4a7309d327 Merge pull request #5307 from dearblue/stackerr
Raise SystemStackError if mruby VM stack expansion fails
2021-01-28 23:01:40 +09:00
Yukihiro "Matz" Matsumoto 0030417ace Merge pull request #5305 from dearblue/symbol0
Check first `0` when converting symbols into strings
2021-01-28 22:56:27 +09:00
dearblue 0dbb9e6e41 Raise SystemStackError if mruby VM stack expansion fails 2021-01-28 22:04:47 +09:00
dearblue 3d8a8fb661 Check first 0 when converting symbols into strings
This was because it caused `SIGSEGV` when `mruby -v` displayed an unnamed variable.

```console
% bin/mruby -ve 'call { |(a, b)| }'

  ...SNIP...

irep 0x8007d0050 nregs=3 nlocals=1 pools=0 syms=1 reps=1 iseq=12
file: -e
    1 000 OP_LOADSELF   R1
    1 002 OP_BLOCK      R2      I(0:0x8007d00a0)
    1 005 OP_SENDB      R1      :call   0
    1 009 OP_RETURN     R1
    1 011 OP_STOP

irep 0x8007d00a0 nregs=6 nlocals=5 pools=0 syms=0 reps=0 iseq=29
local variable names:
zsh: segmentation fault (core dumped)  bin/mruby -ve 'call { |(a, b)| }'
```
2021-01-28 21:15:31 +09:00
Yukihiro "Matz" Matsumoto bb42c1b286 Merge pull request #5304 from shuujii/refine-preprocess_options
Refine `preprocess_options`; ref d95ffb036
2021-01-28 19:57:16 +09:00
Yukihiro "Matz" Matsumoto d42d308620 Remove duplicated --cflags; [ci skip] 2021-01-28 19:55:05 +09:00
KOBAYASHI Shuji c858cba3e9 Refine preprocess_options; ref d95ffb036
If we modify an option that may have been specified by users, we may
make unintended changes, so it is better not to modify it as much as
possible, IMO.
2021-01-28 18:42:19 +09:00
Yukihiro "Matz" Matsumoto 1c05764ba3 Update limitation.md for integer division change in 3.0. 2021-01-28 17:08:30 +09:00
Yukihiro "Matz" Matsumoto 56842628b9 Stop mentioning MRB_NO_PRESYM in the doc.
It's for internal use. Please use `conf.disable_presym`.
2021-01-28 16:47:02 +09:00
Yukihiro "Matz" Matsumoto 07d4498613 Silence VC warning. 2021-01-28 16:35:22 +09:00
Yukihiro "Matz" Matsumoto d95ffb0362 Stop generating build/<build>/src/*.pi.d. 2021-01-28 15:58:35 +09:00
Yukihiro "Matz" Matsumoto 1e009061f8 Describe disabling preallocated symbols in the doc. 2021-01-28 15:47:51 +09:00
Yukihiro "Matz" Matsumoto 4063af90e2 Update TODO after mruby3.0. 2021-01-28 15:06:22 +09:00
Yukihiro "Matz" Matsumoto 2e33c2d120 Remove unnecessary bit shift in rational_new_f. 2021-01-28 08:47:16 +09:00
Yukihiro "Matz" Matsumoto f5e10e294c Add symbol names as comments to presym_length_table; ref #5303 2021-01-27 23:47:32 +09:00
Yukihiro "Matz" Matsumoto 69f9460075 Add static modifiers to preallocated symbols tables; ref #5303 2021-01-27 23:44:04 +09:00
Yukihiro "Matz" Matsumoto f0eb50a55d Merge pull request #5303 from shuujii/split-presym_table-for-reduced-program-size
Split `presym_table` for reduced program size
2021-01-27 23:41:03 +09:00
KOBAYASHI Shuji 3104aed8c6 Split presym_table for reduced program size
Because a structure that is an element of `presym_table` has padding, split
it into individual arrays for name and length.

#### Result (64-bit CPU with full-core gembox)

|        |   mruby    | libmruby.a |
|--------|------------|------------|
| Before | 1,087,444B | 1,476,872B |
| After  | 1,079,340B | 1,469,784B |
2021-01-27 20:47:10 +09:00
Yukihiro "Matz" Matsumoto 251fd74315 Merge pull request #5302 from shuujii/rename-.i-created-for-presym-scan-to-.pi
Rename `.i` created for presym scan to `.pi`
2021-01-27 18:08:58 +09:00
KOBAYASHI Shuji d6a9ffa2b5 Rename .i created for presym scan to .pi
This is because compiler's `-save-temps=obj` option creates `.i` with the
same name.
2021-01-27 16:09:12 +09:00
Yukihiro "Matz" Matsumoto bf56ecd2bf Merge pull request #5301 from shuujii/fix-typos-in-mruby-presym-disableenable.h
Fix typos in `mruby/presym/{disable,enable}.h` [ci skip]
2021-01-27 12:32:42 +09:00
KOBAYASHI Shuji 826e0b108f Fix typos in mruby/presym/{disable,enable}.h [ci skip] 2021-01-27 12:18:41 +09:00
Yukihiro "Matz" Matsumoto 5e479c66d5 Merge pull request #5300 from shuujii/avoid-possible-loss-of-data-casting-in-binary-search
Avoid 'possible loss of data' casting in binary search
2021-01-27 12:15:58 +09:00
Yukihiro "Matz" Matsumoto 89cd742506 Merge pull request #5299 from shuujii/fix-Use-MRB_SYM-for-error-class-retrieval
Fix "Use `MRB_SYM()` for error class retrieval"; ref 2ddfd50df
2021-01-27 12:02:34 +09:00
KOBAYASHI Shuji 504788bf89 Avoid 'possible loss of data' casting in binary search
Because it may not be expected result.
example: https://wandbox.org/permlink/F5Mp7IEJ1VY3CFLp
2021-01-27 11:42:18 +09:00
KOBAYASHI Shuji dd07067067 Fix "Use MRB_SYM() for error class retrieval"; ref 2ddfd50df
The new macro (`MRB_E_SYM`) was not being used, so it is being used. Also
`MRB_E_SYM` is confusing with `MRB_SYM_E`, so change it to `MRB_ERROR_SYM`.
2021-01-27 11:23:08 +09:00
Yukihiro "Matz" Matsumoto efa0314731 Adjust spaces after list markers. 2021-01-26 21:21:26 +09:00
Yukihiro "Matz" Matsumoto d4101d0a08 Revert "Fix the wrong directory host-bin to host/bin."
This reverts commit 34f82f24e9.
Ref #5282
2021-01-26 21:17:28 +09:00
Yukihiro "Matz" Matsumoto f3d5deb981 Clarify that contributors agree with MIT license in CONTRIBUTING.md. 2021-01-26 18:21:49 +09:00
Yukihiro "Matz" Matsumoto 04d5e7da39 Update doc/mruby3.md. 2021-01-26 17:16:22 +09:00
Yukihiro "Matz" Matsumoto 73418a90fa Add doc/symbols.md. [ci skip]
To describe how to use symbols, especially preallocated symbols from C.
2021-01-26 12:42:36 +09:00