admin/mruby-mruby
1
0
Fork 0
mirror of https://github.com/mruby/mruby synced 2026-06-08 16:11:16 +00:00
Code Issues Packages Projects Releases Wiki Activity
12,029 Commits 6 Branches 38 Tags
bed04414d7d394a74bdbbf3377afb5d246aa9fb8
Commit Graph

12029 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
KOBAYASHI Shuji bed04414d7 Fix deprecated set-env command error on GitHub Actions [ci skip] 
I tried `$GITHUB_PATH` and `$GITHUB_ENV` instead of `set-env`, but
for some reason path was not recognized, so I changed to using
`ACTIONS_ALLOW_UNSECURE_COMMANDS`.
 2020-11-17 18:40:59 +09:00
Yukihiro "Matz" Matsumoto aae8237335 Scan all active C source files; ref 600e333 
In addition, `include/mruby.h` will be scanned for error class symbols.
 2020-11-17 09:00:06 +09:00
Yukihiro "Matz" Matsumoto 354934d398 Refactor integer division. 2020-11-17 07:10:31 +09:00
Yukihiro "Matz" Matsumoto 37a1884749 Simplify mrb_num_div_int(). 2020-11-17 00:24:43 +09:00
Yukihiro "Matz" Matsumoto 3107d1caae Allow int size combinations for MRB_WORD_BOXING. 
- `MRB_64BIT` and `MRB_INT32`
- `MRB_32BIT` and `MRB_INT64`
 2020-11-16 22:49:55 +09:00
Yukihiro "Matz" Matsumoto 3bb419aeab Merge pull request #5137 from shuujii/use-uintptr_t-instead-of-unsigned-long-in-include-mruby-boxing_word.h 
Use `uintptr_t` instead of `unsigned long` in `include/mruby/boxing_word.h`
 2020-11-16 22:49:42 +09:00
Yukihiro "Matz" Matsumoto 2609f0a2b3 Merge pull request #5139 from shuujii/avoid-undefined-behavior 
Avoid undefined behavior
 2020-11-16 22:49:03 +09:00
Yukihiro "Matz" Matsumoto 5156ea997f Merge pull request #5138 from shuujii/simplify-obj_free 
Simplify `obj_free`
 2020-11-16 22:48:05 +09:00
KOBAYASHI Shuji 7c470e25b8 Avoid undefined behavior 
### ASAN report (`MRB_INT32`)

  ```console
  $ bin/mruby -ve '-0x40000000'
  mruby 3.0.0preview (2020-10-16)
  00001 NODE_SCOPE:
  00001   NODE_BEGIN:
  00001     NODE_NEGATE:
  00001       NODE_INT 40000000 base 16
  irep 0x6070000001e0 nregs=2 nlocals=1 pools=0 syms=0 reps=0 iseq=9
  file: -e
  /mruby/src/codedump.c:173:49: runtime error: left shift of 49152 by 16 places cannot be represented in type 'int'
  SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /mruby/src/codedump.c:173:49 in
      1 000 OP_LOADI32	R1	-1073741824
      1 006 OP_RETURN	R1
      1 008 OP_STOP

  /mruby/src/vm.c:1138:7: runtime error: left shift of 49152 by 16 places cannot be represented in type 'mrb_int' (aka 'int')
  SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /mruby/src/vm.c:1138:7 in
  ```
 2020-11-16 20:20:30 +09:00
KOBAYASHI Shuji cbc3dbedb4 Simplify obj_free 2020-11-16 16:40:37 +09:00
KOBAYASHI Shuji 8a019106c0 Use uintptr_t instead of unsigned long in include/mruby/boxing_word.h 
`mrb_bool`  on LLP64 environment seems to become false if, for example,
`mrb_value` is a pointer whose lower 32 bits are 0.
 2020-11-16 16:04:05 +09:00
Yukihiro "Matz" Matsumoto 8a6614d30e Merge pull request #5135 from shuujii/fix-paths-of-gem-files-to-be-scanned-for-presym 
Fix paths of gem files to be scanned for presym
 2020-11-15 23:15:34 +09:00
Yukihiro "Matz" Matsumoto 144cf6cd58 Merge pull request #5134 from shuujii/fix-that-large-integer-is-not-GCed-with-Word-boxing 
Fix that large integer is not GCed with Word-boxing
 2020-11-15 23:14:59 +09:00
KOBAYASHI Shuji e334c02a50 Fix paths of gem files to be scanned for presym 
When building from outside `MRUBY_ROOT` (e.g. `rake -f`), gem files were not
scanned.
 2020-11-15 20:16:12 +09:00
KOBAYASHI Shuji acc114003b Fix that large integer is not GCed with Word-boxing 
### Example (32-bit Word-boxing)

  ```ruby
  # example.rb
  int_count = ObjectSpace.count_objects[:T_INTEGER]||0
  int = 1<<30
  p (ObjectSpace.count_objects[:T_INTEGER]||0) - int_count
  int = nil
  GC.start
  p (ObjectSpace.count_objects[:T_INTEGER]||0) - int_count
  ```

#### Before this patch:

  ```console
  $ bin/mruby example.rb
  1
  1
  ```

#### After this patch:

  ```console
  $ bin/mruby example.rb
  1
  0
  ```
 2020-11-15 16:45:03 +09:00
Yukihiro "Matz" Matsumoto a6119b0077 Fixed further wrong condition in overflow detection. 2020-11-15 08:08:55 +09:00
Yukihiro "Matz" Matsumoto bb1fb2f43d Fixed wrong condition in mrb_int_mul_overflow(). 2020-11-14 23:34:43 +09:00
Yukihiro "Matz" Matsumoto 2c6a5d3a77 Change the fallback mrb_int_mul_overflow(). 2020-11-14 23:28:05 +09:00
Yukihiro "Matz" Matsumoto 10c1c9d6f8 Change the type of rlen to uint8_t in mrb_irep. 2020-11-14 23:27:28 +09:00
Yukihiro "Matz" Matsumoto 8a5dfa92ab Integer overflow in operations should cause RangeError. 2020-11-14 20:44:07 +09:00
Yukihiro "Matz" Matsumoto dd3ed8d99c Allow full mrb_int operations in overflow detection. 
Fix overflow detection in integer operations with `MRB_WORD_BOXING`.
This bug made `1073741824 == 1073741824+0` to be `false` on 32bit
platforms.
 2020-11-14 20:40:06 +09:00
Yukihiro "Matz" Matsumoto 19450df41e Integer operation should result in Integer. 
Should raise `RangeError` if the operation overflows.
 2020-11-14 19:38:27 +09:00
Yukihiro "Matz" Matsumoto 600e3330b3 Scan source files only from cdump enabled gems. 2020-11-14 18:52:57 +09:00
Yukihiro "Matz" Matsumoto 4dcd8dd7ce Merge pull request #5132 from shuujii/remove-redundant-command-execution-in-header_search_paths 
Remove redundant command execution in `header_search_paths`
 2020-11-14 18:52:48 +09:00
KOBAYASHI Shuji e941fb7866 Remove redundant command execution in header_search_paths 
There is no need to execute command in duplicate for the same
reason as #5128.
 2020-11-14 17:19:38 +09:00
Yukihiro "Matz" Matsumoto 8398020d0a Merge pull request #5131 from shuujii/remove-unused-constant-in-lib-mruby-build-command.rb 
Remove unused constant in `lib/mruby/build/command.rb`
 2020-11-14 17:01:37 +09:00
Yukihiro "Matz" Matsumoto 69dbe8cfa5 Merge pull request #5130 from shuujii/workaround-to-build-mirb-with-Homebrews-readline-on-Mac 
Workaround to build `mirb` with Homebrew's readline on Mac; fix #4537
 2020-11-14 16:53:23 +09:00
KOBAYASHI Shuji 08df172e91 Remove unused constant in lib/mruby/build/command.rb 2020-11-14 16:50:00 +09:00
KOBAYASHI Shuji 06a8e502fa Workaround to build mirb with Homebrew's readline on Mac; fix #4537 2020-11-14 16:30:45 +09:00
Yukihiro "Matz" Matsumoto fdbb1731ce Merge pull request #5128 from shuujii/simplify-MRubyCommand-_run-to-avoid-duplicated-compilation 
Simplify `MRuby::Command#_run` to avoid duplicated compilation
 2020-11-14 16:08:10 +09:00
Yukihiro "Matz" Matsumoto 5edac8b885 Merge pull request #5129 from dearblue/mrb_mt_foreach 
Don't use private structs with `mrb_mt_foreach()`
 2020-11-14 16:06:47 +09:00
dearblue 368f8a5da6 Don't use private structs with mrb_mt_foreach() 
The `MRB_API` function `mrb_mt_foreach()` previously used the private structure `struct mt_elem`.
Therefore, use `mrb_method_t` instead.
 2020-11-14 11:30:34 +09:00
KOBAYASHI Shuji 0c55b85fb0 Simplify MRuby::Command#_run to avoid duplicated compilation 
ref: https://github.com/mruby/mruby/pull/4959#discussion_r402086196

Compiles twice because it falls back to `build.filename(command)` when
`command` fails. This process was added at 9968af4 to support `ccache gcc`
etc. At that time, it seems that it was necessary because
`build.filename(command)` quoted the whole `command`, but now it does not
quote, so we can just run `build.filename(command)`.

### Example

  ```console
  $ echo 1 > src/a.c
  $ rake -v
  ```

#### Before this patch:

  ```console
  (snip)
  gcc -std=gnu99 -g -O3 -Wall -Wundef -Wdeclaration-after-statement -Werror-implicit-function-declaration -Wwrite-strings -I"/mruby/mruby/include" -MMD -o "/mruby/mruby/build/host/src/a.o" -c "/mruby/mruby/src/a.c"
  /mruby/mruby/src/a.c:1:1: error: expected identifier or '('
  1
  ^
  1 error generated.
  gcc -std=gnu99 -g -O3 -Wall -Wundef -Wdeclaration-after-statement -Werror-implicit-function-declaration -Wwrite-strings -I"/mruby/mruby/include" -MMD -o "/mruby/mruby/build/host/src/a.o" -c "/mruby/mruby/src/a.c"
  /mruby/mruby/src/a.c:1:1: error: expected identifier or '('
  1
  ^
  1 error generated.
  rake aborted!
  (snip)
  ```

#### After this patch:

  ```console
  (snip)
  gcc -std=gnu99 -g -O3 -Wall -Wundef -Wdeclaration-after-statement -Werror-implicit-function-declaration -Wwrite-strings -I"/mruby/mruby/include" -MMD -o "/mruby/mruby/build/host/src/a.o" -c "/mruby/mruby/src/a.c"
  /mruby/mruby/src/a.c:1:1: error: expected identifier or '('
  1
  ^
  1 error generated.
  rake aborted!
  (snip)
  ```
 2020-11-13 21:49:12 +09:00
Yukihiro "Matz" Matsumoto 7fb62670a9 Merge pull request #5127 from shuujii/include-size-of-iv-table-in-ObjectSpace.memsize_of-to-Hash-object 
Include size of iv table in `ObjectSpace.memsize_of` to `Hash` object
 2020-11-13 18:13:46 +09:00
KOBAYASHI Shuji 292a096118 Include size of iv table in ObjectSpace.memsize_of to Hash object 2020-11-13 16:59:45 +09:00
Yukihiro "Matz" Matsumoto 75beb55c2a Merge pull request #5126 from shuujii/use-enable_sanitizer-to-simplify-build_config-clang-asan.rb 
Use `enable_sanitizer` to simplify `build_config/clang-asan.rb`
 2020-11-13 16:47:20 +09:00
Yukihiro "Matz" Matsumoto f8ce3bec13 Merge pull request #5125 from shuujii/rename-mrb_os_memsize_of_hash_table-to-mrb_hash_memsize 
Rename `mrb_os_memsize_of_hash_table` to `mrb_hash_memsize`
 2020-11-13 16:45:52 +09:00
KOBAYASHI Shuji bad6223914 Use enable_sanitizer to simplify build_config/clang-asan.rb 2020-11-13 16:30:18 +09:00
KOBAYASHI Shuji ac4eb99629 Rename mrb_os_memsize_of_hash_table to mrb_hash_memsize 
* The term `hash_table` can be misleading because the return value of this
  function includes memory usage of entire `Hash` object, including not only
  hash table part but also entry list part, etc.
* This function takes a `Hash` object as a receiver and is defined in
  `src/hash.c`, so it is natural to have a `mrb_hash_` prefix.
 2020-11-13 16:18:32 +09:00
Yukihiro "Matz" Matsumoto af13cfb7d5 Merge pull request #5122 from shuujii/change-name-and-usage-of-presym-macros 
Change name and usage of presym macros
 2020-11-13 14:05:26 +09:00
KOBAYASHI Shuji 89f591485b Change name and usage of presym macros 
To be also able to build mruby without presym in the future. However,
`MRB_QSYM` has been removed and changed as follows:

### Example

|       Type                | Symbol |  Previous Style  |   New Style    |
|---------------------------|--------|------------------|----------------|
| Operator                  | &      | MRB_QSYM(and)    | MRB_OPSYM(and) |
| Class Variable            | @@foo  | MRB_QSYM(00_foo) | MRB_CVSYM(foo) |
| Instance Variable         | @foo   | MRB_QSYM(0_foo)  | MRB_IVSYM(foo) |
| Method with Bang          | foo!   | MRB_QSYM(foo_b)  | MRB_SYM_B(foo) |
| Method with Question mark | foo?   | MRB_QSYM(foo_p)  | MRB_SYM_Q(foo) |
| Mmethod with Equal        | foo=   | MRB_QSYM(foo_e)  | MRB_SYM_E(foo) |

This change makes it possible to define, for example, `MRB_IVSYM(foo)` as
`mrb_intern_lit(mrb, "@" "foo")`, which is useful if we support building
without presym in the future.
 2020-11-13 13:41:20 +09:00
Yukihiro "Matz" Matsumoto 280cb80a0d Merge pull request #5124 from shuujii/remove-unused-argument-of-mrb_obj_iv_tbl_memsize 
Remove unused argument of `mrb_obj_iv_tbl_memsize`
 2020-11-12 22:18:57 +09:00
KOBAYASHI Shuji ff2ef44c0e Remove unused argument of mrb_obj_iv_tbl_memsize 2020-11-12 20:19:41 +09:00
Yukihiro "Matz" Matsumoto 6a5e97b448 Merge pull request #5121 from shuujii/reduce-memory-usage-of-Hash-object 
Reduce memory usage of Hash object
 2020-11-10 21:56:42 +09:00
Yukihiro "Matz" Matsumoto cc20c5ca3a Merge pull request #5120 from kou/mrb-exe-new-str-lit-typo 
Fix typo in backward compatible mrb_exc_new_str_lit() macro
 2020-11-10 16:07:36 +09:00
Sutou Kouhei a00eabbf5d Fix typo in backward compatible mrb_exc_new_str_lit() macro 2020-11-10 16:03:05 +09:00
KOBAYASHI Shuji f2d8db39be Reduce memory usage of Hash object 
## Implementation Summary

* Change entry list from segmented list to flat array.
* Change value of hash bucket from pointer to entry to index of entry list,
  and represent it by variable length bits according to capacity of hash
  buckets.
* Store management information about entry list and hash table to `struct
  RHash` as much as possible.

## Benchmark Summary

Only the results of typical situations on 64-bit Word-boxing are present
here. For more detailed information, including consideration, see below
(although most of the body is written in Japanese).

* https://shuujii.github.io/mruby-hash-benchmark

### Memory Usage

Lower value is better.

| Hash Size |   Baseline    |      New      |   Factor   |
|----------:|--------------:|--------------:|-----------:|
|        16 |          344B |          256B |   0.74419x |
|        40 |        1,464B |          840B |   0.57377x |
|       200 |        8,056B |        3,784B |   0.46971x |
|       500 |       17,169B |        9,944B |   0.57949x |

### Performance

Higher value is better.

#### `mrb_hash_set`

| Hash Size |   Baseline    |      New      |   Factor   |
|----------:|--------------:|--------------:|-----------:|
|        16 |  1.41847M i/s |  1.36004M i/s |   0.95881x |
|        40 |  0.39224M i/s |  0.31888M i/s |   0.81296x |
|       200 |  0.03780M i/s |  0.04290M i/s |   1.13494x |
|       500 |  0.01225M i/s |  0.01314M i/s |   1.07275x |

#### `mrb_hash_get`

| Hash Size |   Baseline    |      New      |   Factor   |
|----------:|--------------:|--------------:|-----------:|
|        16 | 26.05920M i/s | 30.19543M i/s |   1.15872x |
|        40 | 44.26420M i/s | 32.75781M i/s |   0.74005x |
|       200 | 44.55171M i/s | 31.56926M i/s |   0.70860x |
|       500 | 39.19250M i/s | 29.73806M i/s |   0.75877x |

#### `mrb_hash_each`

| Hash Size |   Baseline    |      New      |   Factor   |
|----------:|--------------:|--------------:|-----------:|
|        16 | 25.11964M i/s | 30.34167M i/s |   1.20789x |
|        40 | 11.74253M i/s | 13.25539M i/s |   1.12884x |
|       200 |  2.01133M i/s |  2.97214M i/s |   1.47770x |
|       500 |  0.87411M i/s |  1.21178M i/s |   1.38631x |

#### `Hash#[]=`

| Hash Size |   Baseline    |      New      |   Factor   |
|----------:|--------------:|--------------:|-----------:|
|        16 |  0.50095M i/s |  0.56490M i/s |   1.12764x |
|        40 |  0.19132M i/s |  0.18392M i/s |   0.96129x |
|       200 |  0.03624M i/s |  0.03256M i/s |   0.89860x |
|       500 |  0.01527M i/s |  0.01236M i/s |   0.80935x |
#### `Hash#[]`

| Hash Size |   Baseline    |      New      |   Factor   |
|----------:|--------------:|--------------:|-----------:|
|        16 | 11.53211M i/s | 12.78806M i/s |   1.10891x |
|        40 | 15.26920M i/s | 13.37529M i/s |   0.87596x |
|       200 | 15.28550M i/s | 13.36410M i/s |   0.87430x |
|       500 | 14.57695M i/s | 12.75388M i/s |   0.87494x |

#### `Hash#each`

| Hash Size |   Baseline    |      New      |   Factor   |
|----------:|--------------:|--------------:|-----------:|
|        16 |  0.30462M i/s |  0.27080M i/s |   0.88898x |
|        40 |  0.12912M i/s |  0.11704M i/s |   0.90642x |
|       200 |  0.02638M i/s |  0.02402M i/s |   0.91071x |
|       500 |  0.01066M i/s |  0.00959M i/s |   0.89953x |

#### `Hash#delete`

| Hash Size |   Baseline    |      New      |   Factor   |
|----------:|--------------:|--------------:|-----------:|
|        16 |  7.84167M i/s |  6.96419M i/s |   0.88810x |
|        40 |  6.91292M i/s |  7.41427M i/s |   1.07252x |
|       200 |  3.75952M i/s |  7.32080M i/s |   1.94727x |
|       500 |  2.10754M i/s |  7.05963M i/s |   3.34970x |

#### `Hash#shift`

| Hash Size |   Baseline    |      New      |   Factor   |
|----------:|--------------:|--------------:|-----------:|
|        16 | 14.66444M i/s | 13.18876M i/s |   0.89937x |
|        40 | 11.95124M i/s | 11.10420M i/s |   0.92913x |
|       200 |  5.53681M i/s |  7.88155M i/s |   1.42348x |
|       500 |  2.96728M i/s |  5.40405M i/s |   1.82121x |

#### `Hash#dup`

| Hash Size |   Baseline    |      New      |   Factor   |
|----------:|--------------:|--------------:|-----------:|
|        16 |  0.15063M i/s |  5.37889M i/s |  35.71024x |
|        40 |  0.06515M i/s |  3.38196M i/s |  51.91279x |
|       200 |  0.01359M i/s |  1.46538M i/s | 107.84056x |
|       500 |  0.00559M i/s |  0.75411M i/s | 134.88057x |

### Binary Size

Lower value is better.

|    File    |   Baseline    |      New      |  Factor   |
|:-----------|--------------:|--------------:|----------:|
| mruby      |      730,408B |      734,176B |  1.00519x |
| libmruby.a |    1,068,134B |    1,072,846B |  1.00441x |

## Other Fixes

The following issues have also been fixed in the parts where there was some
change this time.

* [Heap use-after-free in `Hash#value?`](https://gist.github.com/shuujii/30e4fcd5844a4112a0ecd4a5b3483101#file-heap-use-after-free-in-hash-value-md)
* [Heap use-after-free in `ht_hash_equal`](https://gist.github.com/shuujii/30e4fcd5844a4112a0ecd4a5b3483101#file-heap-use-after-free-in-ht_hash_equal-md)
* [Heap use-after-free in `ht_hash_func`](https://gist.github.com/shuujii/30e4fcd5844a4112a0ecd4a5b3483101#file-heap-use-after-free-in-ht_hash_func-md)
* [Heap use-after-free in `mrb_hash_merge`](https://gist.github.com/shuujii/30e4fcd5844a4112a0ecd4a5b3483101#file-heap-use-after-free-in-mrb_hash_merge-md)
* [Self-replacement does not work for `Hash#replace`](https://gist.github.com/shuujii/30e4fcd5844a4112a0ecd4a5b3483101#file-self-replacement-does-not-work-for-hash-replace-md)
* [Repeated deletes and inserts increase memory usage of `Hash`](https://gist.github.com/shuujii/30e4fcd5844a4112a0ecd4a5b3483101#file-repeated-deletes-and-inserts-increase-memory-usage-of-hash-md)
* [`Hash#rehash` does not reindex completely](https://gist.github.com/shuujii/30e4fcd5844a4112a0ecd4a5b3483101#file-hash-rehash-does-not-reindex-completely-md)
* `mrb_hash_delete_key` does not cause an error for frozen object
* `mrb_hash_new_capa` does not allocate required space first
* [`mrb_os_memsize_of_hash_table` result is incorrect](https://github.com/mruby/mruby/pull/5032#discussion_r457994075)
 2020-11-10 15:21:49 +09:00
Yukihiro "Matz" Matsumoto 352c4755b2 Merge pull request #5118 from dearblue/mrb_value 
Define `mrb_value` in struct for Word/NaN boxing
 2020-11-10 11:50:59 +09:00
Yukihiro "Matz" Matsumoto 0df460892c Add range check before casting float to integer. 2020-11-09 11:35:54 +09:00
Yukihiro "Matz" Matsumoto 82249bb973 Merge pull request #5119 from kou/mruby-io-standard-io 
Don't check FD for STDIN/STDOUT/STDERR; #5114
 2020-11-09 11:19:21 +09:00