The Pike VM and pattern compiler were exporting bare names like
`re_compile`, `re_exec`, `re_free`, `re_is_word_char`, `re_utf8_charlen`,
`re_utf8_decode`. `re_exec` in particular collides with the obsolete
BSD libc function of the same name (still present on FreeBSD/NetBSD
base), so embedding mruby alongside platform regex could surface a
link-time symbol clash.
Rename all six entry points to `mrb_re_*` to keep the gem's external
symbols inside mruby's namespace. Source file names and the public
header path are unchanged.
Refs #6858.
Co-authored-by: Claude <noreply@anthropic.com>
The backtracking engine recurses via C function calls at RE_SPLIT,
RE_SPLITNG, RE_SAVE, RE_LOOKAHEAD, RE_NEG_LOOKAHEAD, RE_LOOKBEHIND,
and RE_NEG_LOOKBEHIND. Patterns like `(?=)+` make the engine
recurse without consuming input, exhausting the C stack and
triggering SIGSEGV long before MRB_REGEXP_STEP_LIMIT is reached
(each recursion charges only ~1 step, but each frame costs ~150
bytes of stack).
Reported by ClusterFuzz testcase
clusterfuzz-testcase-minimized-mruby_fuzzer-4653331195953152.
Add an integer recursion-depth counter passed alongside the step
counter, and abort the current branch with FALSE when it exceeds
MRB_REGEXP_RECURSION_LIMIT (default 1000, configurable like
STEP_LIMIT). Legitimate patterns nest only a few levels;
pathological inputs bail without crashing the VM.
Co-authored-by: Claude <noreply@anthropic.com>
re_compile stored raw pointers into the pattern source in
pat->named_captures[i].name. Two ways this could dangle:
- With /x, the source was c.stripped, freed at end of compile.
Later reads (regexp construction, MatchData[:name] lookup) hit
freed memory.
- Without /x, the pointer aliased the input string's RSTRING_PTR.
Mutating that string after Regexp.new could re-buffer it, leaving
name dangling.
Allocate one arena buffer per regexp (only when num_named > 0) and
copy all names in. Common-case regexps without named captures pay
zero bytes.
Reported by OSS-Fuzz (testcase 5695283416858624).
Co-authored-by: Claude <noreply@anthropic.com>
Pre-allocate visited array and thread lists at compile time and
reuse them across re_exec calls. A cache_in_use flag detects
re-entrancy and falls back to malloc when needed.
Eliminates 3 malloc + 3 free per NFA execution for the common
non-re-entrant case. Combined with the literal fast path, brings
literal match? from 4.7x to 2.5x vs CRuby.
Co-authored-by: Claude <noreply@anthropic.com>
Pure literal patterns (/hello/, /abc/) are detected at compile
time and matched using memchr+memcmp directly, completely
bypassing Pike VM setup (no malloc, no visited array, no thread
lists). Engine-only time drops from ~400ns to ~80ns.
Co-authored-by: Claude <noreply@anthropic.com>
Compute a 128-bit bitmap of bytes that could start a match.
For patterns like /cat|dog|fox/ the bitmap contains only {b,c,d,f},
skipping positions where no alternative can match. For /\d+/ only
{'0'-'9'} are set.
Used when no literal prefix is available (alternation, character
class patterns). Falls back gracefully when too many bytes match.
Key improvements vs CRuby ratio:
alternation miss: 9.5x -> 3.6x
\d+ medium string: 3.0x -> 1.4x
Co-authored-by: Claude <noreply@anthropic.com>
Extract the leading literal bytes from compiled bytecode and use
memchr + memcmp to skip positions where the prefix cannot match.
Both Pike VM and backtracking engine benefit.
For /needle/ in a 2006-char string: 29x faster (1.97s -> 0.07s),
now on par with CRuby/Oniguruma.
Co-authored-by: Claude <noreply@anthropic.com>
The flag is set not only for non-greedy quantifiers but also for
lookahead, lookbehind, and backreferences. The new name accurately
reflects its purpose: indicating that the backtracking engine is
required instead of the Pike VM.
Co-authored-by: Claude <noreply@anthropic.com>
Add (?<=...) positive and (?<!...) negative lookbehind support.
The sub-pattern must have a fixed byte length (no quantifiers or
alternation), computed at compile time and stored in the instruction.
At execution time, the engine backs up by that many bytes and runs
the sub-pattern forward. Maximum lookbehind length is 255 bytes.
Co-authored-by: Claude <noreply@anthropic.com>
The x flag ignores unescaped whitespace and #comments in patterns,
making complex regexps more readable. Whitespace inside character
classes [...] remains literal. Implemented as a preprocessing step
that strips whitespace/comments before compilation.
Co-authored-by: Claude <noreply@anthropic.com>
implement positive and negative lookahead in the backtracking engine:
- (?=pattern): succeeds if pattern matches at current position
without consuming characters
- (?!pattern): succeeds if pattern does NOT match at current position
new bytecodes RE_LOOKAHEAD and RE_NEG_LOOKAHEAD implemented in
the backtracking engine via nested bt_match calls.
Co-authored-by: Claude <noreply@anthropic.com>
support named capture groups in patterns:
- compiler parses (?<name>...) syntax and builds name table
- MatchData#[:name] and MatchData#["name"] access by name
- MatchData#named_captures returns {name => value} hash
- Regexp#named_captures returns {name => group_number} hash
- named captures stored in mrb_regexp_pattern for GC safety
Co-authored-by: Claude <noreply@anthropic.com>
non-greedy patterns now correctly match the shortest possible
string. patterns with non-greedy quantifiers are dispatched to
the backtracking engine which naturally handles non-greedy
semantics.
the Pike VM continues to be used for purely greedy patterns
(O(n*m) guarantee).
Co-authored-by: Claude <noreply@anthropic.com>
Yukihiro "Matz" Matsumoto