The Pike VM and pattern compiler were exporting bare names like
`re_compile`, `re_exec`, `re_free`, `re_is_word_char`, `re_utf8_charlen`,
`re_utf8_decode`. `re_exec` in particular collides with the obsolete
BSD libc function of the same name (still present on FreeBSD/NetBSD
base), so embedding mruby alongside platform regex could surface a
link-time symbol clash.
Rename all six entry points to `mrb_re_*` to keep the gem's external
symbols inside mruby's namespace. Source file names and the public
header path are unchanged.
Refs #6858.
Co-authored-by: Claude <noreply@anthropic.com>
first_set_walk returned TRUE when it reached RE_MATCH via epsilon
transitions, but that's exactly the case where the optimization is
wrong: an empty-matchable pattern can start matching at any position,
including bytes that aren't in the computed first-byte set. The
skip-ahead loop in pike_vm then advanced past valid empty-match
positions, producing a match at the wrong offset (e.g. /a?/.match("b")
reported the empty match at index 1 instead of 0).
Co-authored-by: Claude <noreply@anthropic.com>
insert_inst was incrementing every offset >= pos, but an offset equal to
pos already points to the new instruction's slot -- bumping it shifts
the target onto whatever code got displaced (typically the body of the
quantified atom). For patterns like /a?b?/ the SPLIT for `a?` then
landed on `CHAR 'b'` instead of the new SPLIT for `b?`, so the "skip a"
thread tried to consume 'b' and died, and both atoms failed to match
zero characters at once.
Fixes#6853.
Co-authored-by: Claude <noreply@anthropic.com>
regexp_init() called re_compile() before setting @source / @flags
IVs, so a Regexp that survived a compile-time exception (e.g. picked
up via ObjectSpace.each_object after `Regexp.new("(")` raised) was
left with no @source. obj.hash then dereferenced nil through
mrb_str_hash() and crashed.
Set the IVs before re_compile(), and make regexp_hash / regexp_eql
defensive against a non-String @source so Regexp.allocate.hash also
behaves.
Co-authored-by: Claude <noreply@anthropic.com>
Inside `[...]`, `\b` denotes U+0008 (backspace) -- the same as
MRI/Onigmo and PCRE. parse_escape() was missing the case, so
the backslash was dropped and the bare letter `b` was inserted
into the class. `[\b]` therefore matched every `b` instead of
backspace.
Add `case 'b': return '\b';` to parse_escape(). The function
is only reached from the character-class body and range
endpoints; the top-level dispatcher emits RE_WBOUND for `\b`
before falling through, so the word-boundary semantics outside
`[...]` are unchanged.
Reported by Sam Ruby in matz/spinel#632; same engine bug
affects both spinel and mruby.
Co-authored-by: Claude <noreply@anthropic.com>
re_compile stored raw pointers into the pattern source in
pat->named_captures[i].name. Two ways this could dangle:
- With /x, the source was c.stripped, freed at end of compile.
Later reads (regexp construction, MatchData[:name] lookup) hit
freed memory.
- Without /x, the pointer aliased the input string's RSTRING_PTR.
Mutating that string after Regexp.new could re-buffer it, leaving
name dangling.
Allocate one arena buffer per regexp (only when num_named > 0) and
copy all names in. Common-case regexps without named captures pay
zero bytes.
Reported by OSS-Fuzz (testcase 5695283416858624).
Co-authored-by: Claude <noreply@anthropic.com>
Add (?<=...) positive and (?<!...) negative lookbehind support.
The sub-pattern must have a fixed byte length (no quantifiers or
alternation), computed at compile time and stored in the instruction.
At execution time, the engine backs up by that many bytes and runs
the sub-pattern forward. Maximum lookbehind length is 255 bytes.
Co-authored-by: Claude <noreply@anthropic.com>
Internal flags (MULTILINE=2, DOTALL=4, EXTENDED=8) differ from
Ruby constants (EXTENDED=2, MULTILINE=4). Convert in C instead
of returning raw internal flags. Also add Regexp#casefold?.
Co-authored-by: Claude <noreply@anthropic.com>
The x flag ignores unescaped whitespace and #comments in patterns,
making complex regexps more readable. Whitespace inside character
classes [...] remains literal. Implemented as a preprocessing step
that strips whitespace/comments before compilation.
Co-authored-by: Claude <noreply@anthropic.com>
Two regexps are equal when they have the same source and flags.
Hash is computed from source string hash mixed with flags.
Co-authored-by: Claude <noreply@anthropic.com>
Regexp#to_s now returns (?flags:source) format (e.g. "(?i:abc)")
instead of the /source/flags format used by Regexp#inspect.
Co-authored-by: Claude <noreply@anthropic.com>
- $1-$9 global variables set by Regexp#match and Regexp#=~
- $1-$9 cleared to nil on match failure
- add mruby-regexp to stdlib.gembox (auto-included in standard builds)
- remove duplicate gem entry from host-debug.rb
Co-authored-by: Claude <noreply@anthropic.com>
implement positive and negative lookahead in the backtracking engine:
- (?=pattern): succeeds if pattern matches at current position
without consuming characters
- (?!pattern): succeeds if pattern does NOT match at current position
new bytecodes RE_LOOKAHEAD and RE_NEG_LOOKAHEAD implemented in
the backtracking engine via nested bt_match calls.
Co-authored-by: Claude <noreply@anthropic.com>
support named capture groups in patterns:
- compiler parses (?<name>...) syntax and builds name table
- MatchData#[:name] and MatchData#["name"] access by name
- MatchData#named_captures returns {name => value} hash
- Regexp#named_captures returns {name => group_number} hash
- named captures stored in mrb_regexp_pattern for GC safety
Co-authored-by: Claude <noreply@anthropic.com>
non-greedy patterns now correctly match the shortest possible
string. patterns with non-greedy quantifiers are dispatched to
the backtracking engine which naturally handles non-greedy
semantics.
the Pike VM continues to be used for purely greedy patterns
(O(n*m) guarantee).
Co-authored-by: Claude <noreply@anthropic.com>
add a recursive backtracking engine that handles \1-\9
backreferences. the Pike VM (NFA) is used for patterns without
backreferences; patterns with backreferences automatically
fall back to the backtracking engine.
the backtracking engine has a step limit (MRB_REGEXP_STEP_LIMIT,
default 1M) to prevent ReDoS on pathological patterns.
also adds SAVE backtracking (save/restore capture positions on
failed branches) for correct submatch tracking.
Co-authored-by: Claude <noreply@anthropic.com>
add tests for: empty pattern, nested captures, word boundary \b,
non-capturing groups (?:), sub/gsub with block, scan with captures,
split with regexp, case/when with regexp, date reformatting.
known limitation: non-greedy quantifiers (*?, +?) currently behave
as greedy. needs match priority tracking (TODO for Phase 2).
Co-authored-by: Claude <noreply@anthropic.com>
- /regex/ literal syntax now works (compiler generates Regexp.compile)
- Regexp#match and Regexp#=~ set $~ global variable
- Regexp.last_match(n) for accessing capture groups
- Regexp.compile as alias for Regexp.new
- Regexp#options
Co-authored-by: Claude <noreply@anthropic.com>
Yukihiro "Matz" Matsumoto