Commit Graph

768 Commits

Author SHA1 Message Date
mirichi a997e41d17 add block argument test #2144 2021-06-04 22:25:33 +09:00
Yukihiro "Matz" Matsumoto 01a21085f6 add a few regressions test from #2313
The code was contributed from Carson McDonald (@carsonmcdonald)
2021-06-03 16:47:04 +09:00
Yukihiro "Matz" Matsumoto 03c4e114a9 test/syntax.rb: fix endless def warning (no assertion). 2021-05-21 08:13:37 +09:00
Yukihiro "Matz" Matsumoto 07c8470278 parse.y: allow "command" syntax in endless method definition.
This change allows `def hello = puts "Hello"` without parentheses.
This syntax has been introduced since Ruby3.1.
2021-05-18 16:26:36 +09:00
Yukihiro "Matz" Matsumoto ce054bd078 symbol.c: remove Symbol#id2sym.
In the ancient Ruby, symbols are represented by integers. In that era,
to get string representation from integers, we used `Integer#id2sym`
method. Later, `Symbol` was introduced, and `id2sym` was used for
compatibility. Today, no one uses `id2sym` any longer. It is described
in ISO 30170:2012 standard but I consider it as a mistake.
2021-05-03 15:30:59 +09:00
Yukihiro "Matz" Matsumoto 9f77232b71 array.c: update Array#shift to take optional argument; close #5428 2021-04-27 16:37:24 +09:00
Yukihiro "Matz" Matsumoto 4d249c28d9 Skip tests that use Float inside; ref #5421 2021-04-24 12:04:08 +09:00
John Bampton 9d32d440eb feat(CI): add the GitHub Super Linter
The GitHub Super Linter is a more robust and better supported
tool than the current GitHub Actions we are using.

Running these checks:

ERROR_ON_MISSING_EXEC_BIT: true
VALIDATE_BASH: true
VALIDATE_BASH_EXEC: true
VALIDATE_EDITORCONFIG: true
VALIDATE_MARKDOWN: true
VALIDATE_SHELL_SHFMT: true
VALIDATE_YAML: true

https://github.com/marketplace/actions/super-linter
https://github.com/github/super-linter

Added the GitHub Super Linter badge to the README.

Also updated the pre-commit framework and added
more documentation on pre-commit.

Added one more pre-commit check: check-executables-have-shebangs

Added one extra check for merge conflicts to our
GitHub Actions.

EditorConfig and Markdown linting.

Minor grammar and spelling fixes.

Update linter.yml
2021-04-16 16:37:52 +09:00
Yukihiro "Matz" Matsumoto bda242a135 vm.c: change the default error message for undefined super method.
- (old) `undefined method 'foo'`
- (new) `no superclass method 'foo'`
2021-04-01 08:42:30 +09:00
Yukihiro "Matz" Matsumoto 9f85c5e483 codegen.c: yield outside of method is now SyntaxError. 2021-03-31 15:57:43 +09:00
fundamental c146e81c4a Disable tests on backtraces w/ unknown line numbers 2021-03-30 20:26:10 -04:00
KOBAYASHI Shuji 6731f935cc Float::NAN/0 should be Float::NAN; ref a0b3378b3
#### Before this patch:

```console
$ bin/mruby -e 'p(Float::NAN/0)'
Infinity
```

#### After this patch (same as Ruby):

```console
$ bin/mruby -e 'p(Float::NAN/0)'
NaN
```
2021-03-18 17:10:51 +09:00
Yukihiro "Matz" Matsumoto c0d63ea09f hash.c: Hash#shift to return nil when a hash is empty.
It used to be return the default value if available, but it should
ignore the default value for behavior consistency. CRuby will adopt
this behavior too in the future. [ruby-bugs:16908]
2021-03-17 15:14:23 +09:00
Yukihiro "Matz" Matsumoto 75ae3d3e23 range.c: fixed a begin-less ranges issue. 2021-03-17 09:48:27 +09:00
John Bampton 1c9b1bfeb7 feat: add pre-commit framework 2021-03-01 10:06:17 +10:00
KOBAYASHI Shuji 28eb6271ba Fix heap-buffer-overflow for small Hash (HT) in Hash#rehash
### Example

##### example.rb

```ruby
h = {}
(1..17).each{h[_1] = _1}
(1..16).each{h.delete(_1)}
h.rehash
```

##### ASAN report

```console
$ bin/mruby example.rb
==52587==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x602000006998 at pc 0x55a29cddf96b bp 0x7fff7b1b1720 sp 0x7fff7b1b1710
READ of size 4 at 0x602000006998 thread T0
    #0 0x55a29cddf96a in ib_it_next /mruby/src/hash.c:639
    #1 0x55a29cde2ca2 in ht_rehash /mruby/src/hash.c:900
    #2 0x55a29cde379f in h_rehash /mruby/src/hash.c:996
    #3 0x55a29cde7f3d in mrb_hash_rehash /mruby/src/hash.c:1735
    #4 0x55a29ce77b62 in mrb_vm_exec /mruby/src/vm.c:1451
    #5 0x55a29ce5fa88 in mrb_vm_run /mruby/src/vm.c:981
    #6 0x55a29ceb87e1 in mrb_top_run /mruby/src/vm.c:2874
    #7 0x55a29cf36bdf in mrb_load_exec mrbgems/mruby-compiler/core/parse.y:6805
    #8 0x55a29cf36f25 in mrb_load_detect_file_cxt mrbgems/mruby-compiler/core/parse.y:6848
    #9 0x55a29cdba0a2 in main /mruby/mrbgems/mruby-bin-mruby/tools/mruby/mruby.c:347
    #10 0x7f24ef43b0b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)
    #11 0x55a29cdb4a6d in _start (/mruby/bin/mruby+0x2a3a6d)

0x602000006998 is located 0 bytes to the right of 8-byte region [0x602000006990,0x602000006998)
allocated by thread T0 here:
    #0 0x7f24f01cfffe in __interceptor_realloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10dffe)
    #1 0x55a29ceb9440 in mrb_default_allocf /mruby/src/state.c:68
    #2 0x55a29cdba747 in mrb_realloc_simple /mruby/src/gc.c:228
    #3 0x55a29cdba928 in mrb_realloc /mruby/src/gc.c:242
    #4 0x55a29cde12e5 in ht_init /mruby/src/hash.c:749
    #5 0x55a29cde2b8e in ht_rehash /mruby/src/hash.c:897
    #6 0x55a29cde379f in h_rehash /mruby/src/hash.c:996
    #7 0x55a29cde7f3d in mrb_hash_rehash /mruby/src/hash.c:1735
    #8 0x55a29ce77b62 in mrb_vm_exec /mruby/src/vm.c:1451
    #9 0x55a29ce5fa88 in mrb_vm_run /mruby/src/vm.c:981
    #10 0x55a29ceb87e1 in mrb_top_run /mruby/src/vm.c:2874
    #11 0x55a29cf36bdf in mrb_load_exec mrbgems/mruby-compiler/core/parse.y:6805
    #12 0x55a29cf36f25 in mrb_load_detect_file_cxt mrbgems/mruby-compiler/core/parse.y:6848
    #13 0x55a29cdba0a2 in main /mruby/mrbgems/mruby-bin-mruby/tools/mruby/mruby.c:347
    #14 0x7f24ef43b0b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)
```
2021-02-10 20:03:34 +09:00
Yukihiro "Matz" Matsumoto f4a3dc4f07 Add tests for Integer#quo and Float#quo; #5268 2021-01-17 23:02:36 +09:00
Seeker 8286e6052a Fix for escaped newlines in squiggly heredocs 2021-01-03 00:48:10 -08:00
Yukihiro "Matz" Matsumoto 15e7b3e8fe Merge pull request #5251 from SeekingMeaning/squiggly-fix-3
Fix regression for squiggly heredocs
2021-01-02 20:28:13 +09:00
KOBAYASHI Shuji 1a9ac02d8d Refine "wrong number of arguments" message in mrb_get_args
#### Before this patch:

```ruby
__send__         #=> wrong number of arguments
{}.default(1,2)  #=> wrong number of arguments
```

#### After this patch:

```ruby
__send__         #=> wrong number of arguments (given 0, expected 1+)
{}.default(1,2)  #=> wrong number of arguments (given 2, expected 0..1)
```
2021-01-02 19:15:34 +09:00
Seeker 4683a196d9 Fix regression for squiggly heredocs 2020-12-31 19:08:44 -08:00
Seeker 49b01f23a4 Fix mixed indentation and escaped tabs in squiggly heredocs 2020-12-28 18:22:13 -08:00
Seeker 3a2ff9824e Fix for empty lines in squiggly heredocs 2020-12-27 23:10:08 -08:00
Seeker 54bfcaf1ff Add support for squiggly heredocs 2020-12-26 23:48:20 -08:00
John Bampton 4fa3359d44 refactor: remove trailing whitespace from C, Header, Ruby and YAML files
Lint
2020-12-15 19:44:02 +10:00
John Bampton 940dec5e7d Fix spelling 2020-12-13 18:38:22 +10:00
Yukihiro "Matz" Matsumoto 092dd4a667 Argument forwarding with ... now supports leading arguments.
```ruby
def method_missing(meth, ...)
  send(:"do_#{meth}", ...)
end
```
2020-12-09 11:56:54 +09:00
Yukihiro "Matz" Matsumoto 1974aa0f72 Update Float#to_s to keep trailing zero as CRuby does; ref 68cebb6 2020-11-21 14:38:13 +09:00
Yukihiro "Matz" Matsumoto 19450df41e Integer operation should result in Integer.
Should raise `RangeError` if the operation overflows.
2020-11-14 19:38:27 +09:00
KOBAYASHI Shuji f2d8db39be Reduce memory usage of Hash object
## Implementation Summary

* Change entry list from segmented list to flat array.
* Change value of hash bucket from pointer to entry to index of entry list,
  and represent it by variable length bits according to capacity of hash
  buckets.
* Store management information about entry list and hash table to `struct
  RHash` as much as possible.

## Benchmark Summary

Only the results of typical situations on 64-bit Word-boxing are present
here. For more detailed information, including consideration, see below
(although most of the body is written in Japanese).

* https://shuujii.github.io/mruby-hash-benchmark

### Memory Usage

Lower value is better.

| Hash Size |   Baseline    |      New      |   Factor   |
|----------:|--------------:|--------------:|-----------:|
|        16 |          344B |          256B |   0.74419x |
|        40 |        1,464B |          840B |   0.57377x |
|       200 |        8,056B |        3,784B |   0.46971x |
|       500 |       17,169B |        9,944B |   0.57949x |

### Performance

Higher value is better.

#### `mrb_hash_set`

| Hash Size |   Baseline    |      New      |   Factor   |
|----------:|--------------:|--------------:|-----------:|
|        16 |  1.41847M i/s |  1.36004M i/s |   0.95881x |
|        40 |  0.39224M i/s |  0.31888M i/s |   0.81296x |
|       200 |  0.03780M i/s |  0.04290M i/s |   1.13494x |
|       500 |  0.01225M i/s |  0.01314M i/s |   1.07275x |

#### `mrb_hash_get`

| Hash Size |   Baseline    |      New      |   Factor   |
|----------:|--------------:|--------------:|-----------:|
|        16 | 26.05920M i/s | 30.19543M i/s |   1.15872x |
|        40 | 44.26420M i/s | 32.75781M i/s |   0.74005x |
|       200 | 44.55171M i/s | 31.56926M i/s |   0.70860x |
|       500 | 39.19250M i/s | 29.73806M i/s |   0.75877x |

#### `mrb_hash_each`

| Hash Size |   Baseline    |      New      |   Factor   |
|----------:|--------------:|--------------:|-----------:|
|        16 | 25.11964M i/s | 30.34167M i/s |   1.20789x |
|        40 | 11.74253M i/s | 13.25539M i/s |   1.12884x |
|       200 |  2.01133M i/s |  2.97214M i/s |   1.47770x |
|       500 |  0.87411M i/s |  1.21178M i/s |   1.38631x |

#### `Hash#[]=`

| Hash Size |   Baseline    |      New      |   Factor   |
|----------:|--------------:|--------------:|-----------:|
|        16 |  0.50095M i/s |  0.56490M i/s |   1.12764x |
|        40 |  0.19132M i/s |  0.18392M i/s |   0.96129x |
|       200 |  0.03624M i/s |  0.03256M i/s |   0.89860x |
|       500 |  0.01527M i/s |  0.01236M i/s |   0.80935x |
#### `Hash#[]`

| Hash Size |   Baseline    |      New      |   Factor   |
|----------:|--------------:|--------------:|-----------:|
|        16 | 11.53211M i/s | 12.78806M i/s |   1.10891x |
|        40 | 15.26920M i/s | 13.37529M i/s |   0.87596x |
|       200 | 15.28550M i/s | 13.36410M i/s |   0.87430x |
|       500 | 14.57695M i/s | 12.75388M i/s |   0.87494x |

#### `Hash#each`

| Hash Size |   Baseline    |      New      |   Factor   |
|----------:|--------------:|--------------:|-----------:|
|        16 |  0.30462M i/s |  0.27080M i/s |   0.88898x |
|        40 |  0.12912M i/s |  0.11704M i/s |   0.90642x |
|       200 |  0.02638M i/s |  0.02402M i/s |   0.91071x |
|       500 |  0.01066M i/s |  0.00959M i/s |   0.89953x |

#### `Hash#delete`

| Hash Size |   Baseline    |      New      |   Factor   |
|----------:|--------------:|--------------:|-----------:|
|        16 |  7.84167M i/s |  6.96419M i/s |   0.88810x |
|        40 |  6.91292M i/s |  7.41427M i/s |   1.07252x |
|       200 |  3.75952M i/s |  7.32080M i/s |   1.94727x |
|       500 |  2.10754M i/s |  7.05963M i/s |   3.34970x |

#### `Hash#shift`

| Hash Size |   Baseline    |      New      |   Factor   |
|----------:|--------------:|--------------:|-----------:|
|        16 | 14.66444M i/s | 13.18876M i/s |   0.89937x |
|        40 | 11.95124M i/s | 11.10420M i/s |   0.92913x |
|       200 |  5.53681M i/s |  7.88155M i/s |   1.42348x |
|       500 |  2.96728M i/s |  5.40405M i/s |   1.82121x |

#### `Hash#dup`

| Hash Size |   Baseline    |      New      |   Factor   |
|----------:|--------------:|--------------:|-----------:|
|        16 |  0.15063M i/s |  5.37889M i/s |  35.71024x |
|        40 |  0.06515M i/s |  3.38196M i/s |  51.91279x |
|       200 |  0.01359M i/s |  1.46538M i/s | 107.84056x |
|       500 |  0.00559M i/s |  0.75411M i/s | 134.88057x |

### Binary Size

Lower value is better.

|    File    |   Baseline    |      New      |  Factor   |
|:-----------|--------------:|--------------:|----------:|
| mruby      |      730,408B |      734,176B |  1.00519x |
| libmruby.a |    1,068,134B |    1,072,846B |  1.00441x |

## Other Fixes

The following issues have also been fixed in the parts where there was some
change this time.

* [Heap use-after-free in `Hash#value?`](https://gist.github.com/shuujii/30e4fcd5844a4112a0ecd4a5b3483101#file-heap-use-after-free-in-hash-value-md)
* [Heap use-after-free in `ht_hash_equal`](https://gist.github.com/shuujii/30e4fcd5844a4112a0ecd4a5b3483101#file-heap-use-after-free-in-ht_hash_equal-md)
* [Heap use-after-free in `ht_hash_func`](https://gist.github.com/shuujii/30e4fcd5844a4112a0ecd4a5b3483101#file-heap-use-after-free-in-ht_hash_func-md)
* [Heap use-after-free in `mrb_hash_merge`](https://gist.github.com/shuujii/30e4fcd5844a4112a0ecd4a5b3483101#file-heap-use-after-free-in-mrb_hash_merge-md)
* [Self-replacement does not work for `Hash#replace`](https://gist.github.com/shuujii/30e4fcd5844a4112a0ecd4a5b3483101#file-self-replacement-does-not-work-for-hash-replace-md)
* [Repeated deletes and inserts increase memory usage of `Hash`](https://gist.github.com/shuujii/30e4fcd5844a4112a0ecd4a5b3483101#file-repeated-deletes-and-inserts-increase-memory-usage-of-hash-md)
* [`Hash#rehash` does not reindex completely](https://gist.github.com/shuujii/30e4fcd5844a4112a0ecd4a5b3483101#file-hash-rehash-does-not-reindex-completely-md)
* `mrb_hash_delete_key` does not cause an error for frozen object
* `mrb_hash_new_capa` does not allocate required space first
* [`mrb_os_memsize_of_hash_table` result is incorrect](https://github.com/mruby/mruby/pull/5032#discussion_r457994075)
2020-11-10 15:21:49 +09:00
Yukihiro "Matz" Matsumoto dc886873bc Replace R-assignment by single-line pattern matching.
Since `R-assignment` in CRuby is abandoned.  Single-line pattern matching
in `mruby` only matches single local variable at the moment. Currently
it works as a right assignment to a local variable. It will be enhanced
in the future.
2020-11-02 15:14:36 +09:00
Yukihiro "Matz" Matsumoto 32ffd81011 Add beginless/endless ranges tests. 2020-10-29 10:43:09 +09:00
Yukihiro "Matz" Matsumoto 6b457d2c00 Merge branch 'work_for_merge' of https://github.com/zubycz/mruby into zubycz-work_for_merge 2020-10-15 18:35:13 +09:00
taiyoslime bec4d05340 Introduce endless range (a part of #5085)
Co-Authored-By: n4o847 <22975590+n4o847@users.noreply.github.com>
Co-Authored-By: smallkirby <ssmallkirby@gmail.com>
2020-10-13 14:09:36 +09:00
Yukihiro "Matz" Matsumoto 2b188ed8a1 Reorganize Integer system.
- Integrate `Fixnum` and `Integer`
- Remove `Integral`
- `int / int -> int`
- Replace `mrb_fixnum()` to `mrb_int()`
- Replace `mrb_fixnum_value()` to `mrb_int_value()`.
- Use `mrb_integer_p()` instead of `mrb_fixnum_p()`
2020-10-12 18:19:54 +09:00
Yukihiro "Matz" Matsumoto 04fd7af23c Fixed ISO section numbers in test/t/superclass.rb. 2020-10-12 16:21:48 +09:00
Yukihiro "Matz" Matsumoto 2a92fb2516 Make division by zero cause ZeroDivisionError.
As described in ISO 15.2.30.
2020-10-12 16:21:48 +09:00
KOBAYASHI Shuji 8bd1bc47b3 Always add trailing .0 in Float#inspect; ref #4225
Trailing `.0` is removed from `Float#to_s` and `Float#inspect` at
9d08025b. However, I think the more human-readable format is better
for `Float#inspect`.

For example, in the `Float#to_s` format, the failure message is not
well understood when testing values including types by `eql?` (e.g.
`Numeric#step` test).

  ```ruby
  assert "example" do
    exp = 1.0
    act = 1
    assert_operator(exp, :eql?, act)  #=> Expected 1 to be eql? 1.
  end
  ```
2020-10-12 16:21:45 +09:00
dearblue 80fe9838d2 Integrate Fixnum class into Integer class
* The `Fixnum` constant is now an alias for the `Integer` class.
* Remove `struct mrb_state::fixnum_class` member.
  If necessary, use `struct mrb_state::integer_class` instead.
2020-10-12 16:21:44 +09:00
Yukihiro "Matz" Matsumoto ed32b6fcdc Update tests for integer division. 2020-10-12 16:21:43 +09:00
Yukihiro "Matz" Matsumoto 8a87549315 Rename float configuration option names.
- `MRB_WITHOUT_FLOAT` => `MRB_NO_FLOAT`
- `MRB_USE_FLOAT` => `MRB_USE_FLOAT32`

The former is to use `USE_XXX` naming convention. The latter is to make
sure `float` is 32bit float and not floating point number in general.
2020-10-12 16:21:40 +09:00
Yukihiro "Matz" Matsumoto 6e86e290e4 Rightward-assign by ASSOC.
[ruby-bugs:15921]
2020-10-12 16:21:18 +09:00
dearblue 5dc87f77d4 Fixed shift width for MRB_ENV_SET_BIDX()
ref c07f24cd1 and close #5035
2020-07-16 23:11:30 +09:00
dearblue 4cd858f93d Update test/t/kernel.rb
- Remove unnecessoury assert
- Fix typo
2020-06-09 20:33:56 +09:00
Yusuke Endoh 6f4c585bd7 Do not destruct rest arguments for __send__
Formerly, `__send__(*args)` modified `args` with `Array#shift`.
This bug affects optcarrot.

This changeset avoids the array destruction by using
`args = args[1, len-1]`.
2020-05-24 01:25:03 +09:00
dearblue 81c8352179 Integrate class EnsureYieldBreak under test 2020-04-26 21:16:44 +09:00
Yukihiro "Matz" Matsumoto c181d89c9d Kernel#clone and Kernel#dup no longer raise TypeError; fix #4974
In ISO, those methods should raise `TypeError`, but the spec has been
changed.  The change was discussed in [Feature#12979].
2020-04-21 13:21:12 +09:00
Yukihiro "Matz" Matsumoto 364c47610f Fixed backtrace message for top-level blocks; fix #4936
In top-level, `mid` is `NULL`. We used to ignore 'mid` update for `NULL`.
2020-01-27 14:34:54 +09:00
Yukihiro "Matz" Matsumoto 06e09fc7a9 Numbered parameters should not be available in the lambda bodies.
`mruby` does not warn like `CRuby` for cases like #4893.
Fix #4890, fix #4891, fix #4893.
2019-12-21 22:41:32 +09:00
KOBAYASHI Shuji 74f564b0bb _0 is not numbered parameter
#### Before this patch:

  ```console
  $ bin/mruby rb -e '_0=:l; p ->{_0}.()'
  -e:1:13: _0 is not available
  -e:1:13: syntax error, unexpected $end, expecting '}'
  ```

#### After this patch (same as Ruby):

  ```console
  $ bin/mruby rb -e '_0=:l; p ->{_0}.()'
  :l
  ```
2019-12-21 18:54:15 +09:00