Major changes to the NFA execution engine:
- Thread captures stored in a flat pool sized to actual ncap
(e.g. 4 ints for 1 capture group vs 64 fixed), dramatically
reducing per-thread copy cost
- Generation counter for visited[] eliminates per-step memset
of the entire bytecode-length array
- Pool compaction between steps reclaims dead thread slots
- Backtracking engine also uses dynamic ncap-sized captures
Co-authored-by: Claude <noreply@anthropic.com>
Internal flags (MULTILINE=2, DOTALL=4, EXTENDED=8) differ from
Ruby constants (EXTENDED=2, MULTILINE=4). Convert in C instead
of returning raw internal flags. Also add Regexp#casefold?.
Co-authored-by: Claude <noreply@anthropic.com>
The x flag ignores unescaped whitespace and #comments in patterns,
making complex regexps more readable. Whitespace inside character
classes [...] remains literal. Implemented as a preprocessing step
that strips whitespace/comments before compilation.
Co-authored-by: Claude <noreply@anthropic.com>
Two regexps are equal when they have the same source and flags.
Hash is computed from source string hash mixed with flags.
Co-authored-by: Claude <noreply@anthropic.com>
Regexp#to_s now returns (?flags:source) format (e.g. "(?i:abc)")
instead of the /source/flags format used by Regexp#inspect.
Co-authored-by: Claude <noreply@anthropic.com>
- $1-$9 global variables set by Regexp#match and Regexp#=~
- $1-$9 cleared to nil on match failure
- add mruby-regexp to stdlib.gembox (auto-included in standard builds)
- remove duplicate gem entry from host-debug.rb
Co-authored-by: Claude <noreply@anthropic.com>
implement positive and negative lookahead in the backtracking engine:
- (?=pattern): succeeds if pattern matches at current position
without consuming characters
- (?!pattern): succeeds if pattern does NOT match at current position
new bytecodes RE_LOOKAHEAD and RE_NEG_LOOKAHEAD implemented in
the backtracking engine via nested bt_match calls.
Co-authored-by: Claude <noreply@anthropic.com>
support named capture groups in patterns:
- compiler parses (?<name>...) syntax and builds name table
- MatchData#[:name] and MatchData#["name"] access by name
- MatchData#named_captures returns {name => value} hash
- Regexp#named_captures returns {name => group_number} hash
- named captures stored in mrb_regexp_pattern for GC safety
Co-authored-by: Claude <noreply@anthropic.com>
non-greedy patterns now correctly match the shortest possible
string. patterns with non-greedy quantifiers are dispatched to
the backtracking engine which naturally handles non-greedy
semantics.
the Pike VM continues to be used for purely greedy patterns
(O(n*m) guarantee).
Co-authored-by: Claude <noreply@anthropic.com>
add a recursive backtracking engine that handles \1-\9
backreferences. the Pike VM (NFA) is used for patterns without
backreferences; patterns with backreferences automatically
fall back to the backtracking engine.
the backtracking engine has a step limit (MRB_REGEXP_STEP_LIMIT,
default 1M) to prevent ReDoS on pathological patterns.
also adds SAVE backtracking (save/restore capture positions on
failed branches) for correct submatch tracking.
Co-authored-by: Claude <noreply@anthropic.com>
add tests for: empty pattern, nested captures, word boundary \b,
non-capturing groups (?:), sub/gsub with block, scan with captures,
split with regexp, case/when with regexp, date reformatting.
known limitation: non-greedy quantifiers (*?, +?) currently behave
as greedy. needs match priority tracking (TODO for Phase 2).
Co-authored-by: Claude <noreply@anthropic.com>
- /regex/ literal syntax now works (compiler generates Regexp.compile)
- Regexp#match and Regexp#=~ set $~ global variable
- Regexp.last_match(n) for accessing capture groups
- Regexp.compile as alias for Regexp.new
- Regexp#options
Co-authored-by: Claude <noreply@anthropic.com>
Yukihiro "Matz" Matsumoto