Make `Integer#chr("UTF-8")` reject UTF-16 surrogate code points
(U+D800..U+DFFF), and make `String#ord` reject ill-formed UTF-8 byte
sequences (overlong encodings, surrogates encoded as UTF-8, and code
points above U+10FFFF), matching CRuby and RFC 3629.
The `utf8code()` helper now decodes the code point first and then
validates the range per byte length:
len=2: cp >= 0x80 (rejects overlong)
len=3: cp >= 0x800 and not D800..DFFF
len=4: 0x10000 <= cp <= 0x10FFFF
close#2708
Co-authored-by: Claude <noreply@anthropic.com>
Currently, `Integral#chr` in mruby changes behavior by `MRB_UTF8_STRING`
setting.
before this patch:
$ bin/mruby -e 'p 171.chr' #=> "\xab" (`MRB_UTF8_STRING` is disabled)
$ bin/mruby -e 'p 171.chr' #=> "«" (`MRB_UTF8_STRING` is enabled)
This behavior is incompatible with Ruby, and a little inconvenient because
it can't be interpreted as ASCII-8BIT with `MRB_UTF8_STRING`, I think.
So add encoding argument according to Ruby.
after this patch:
$ bin/mruby -e 'p 171.chr' #=> "\xab"
$ bin/mruby -e 'p 171.chr("ASCII-8BIT")' #=> "\xab"
$ bin/mruby -e 'p 171.chr("UTF-8")' #=> "«"
Allow only `String` for encoding because mruby doesn't have `Encoding`
class, and `"ASCII-8BIT"` (`"BINARY"`) and `"UTF-8"` (only with
`MRB_UTF8_STRING`) are valid value (default is `"ASCII-8BIT"`).
Because they're defined in both `mruby-string-ext` and `mruby-numeric-ext`
(they seem more natural to define in N, but `mruby-string-ext` depends on
`Integral#chr`).
Yukihiro "Matz" Matsumoto