mruby/mruby#4511 demonstrated an infinite loop in `String#each_line` when
given an empty string separator. In MRI, an empty separator places
String#each_line in paragraph mode, where the String is separated on
successive runs of newlines. In paragraph mode, the String
`"abc\n\n\ndef\nxyz"` is split into `["abc\n\n\n", "def\nxyz"]`.
This commit makes the String#each_line implementation as close to
ruby/spec compliant as possible given the limitations of mruby core.
With this patch, the following specs fail for `String#each_line`:
- uses `$/` as the separator when none is given (can be fixed by
aliasing and redefining the method to use $/ as the default value
of separator in mruby-io)
- when no block is given returned Enumerator size should return nil
(`Enumerator#size` is not supported on mruby)
- tries to convert the separator to a string using to_str (`String#to_str`
is not implemented on mruby)
This patch has similar memory consumption compared to the prior
implementation and is takes 4x the time the prior implementation takes to
execute:
```console
/usr/bin/time -l ./bin/mruby -e '("aaa\n\nbbbbb\n\n\n\n\ncccc" * 100000).each_line("\n") { }';
```
We have added internal convenience method `__to_str` which
does string type check.
The issue #3854 was fixed but fundamental flaw of lack of stack
depth check along with fibers still remains. Use `MRB_GC_FIXED_ARENA`
for workaround.
The ISO standard does not include implicit type conversion using
`to_int`. This implicit conversion often causes vulnerability.
There will be no more attacks like #4120.
In addition, we have added internal convenience method `__to_int` which
does type check and conversion (from floats).
Even though `Symbol#to_proc` is not included in ISO standard, the
`some_method(&:method_name)` is used very widely and convenient.
So we moved it to the core.
Example benchmark:
$ time build/bench/bin/mruby -e "Array.new(2_000_000){ |i| i }.index{ |i| i == 1_999_999 }"
Before:
real 0m0.934s
user 0m0.922s
sys 0m0.003s
After:
real 0m0.590s
user 0m0.583s
sys 0m0.007s
It's incompatible with CRuby, but it is required for mruby because
it cannot detect `break` outside of loops in the parsing time, but
in the code generation time where it cannot raise `SyntaxError`.
Ryan Lopopolo