Replace fixed RE_MAX_CAPTURES*2 (256 bytes) stack array with
malloc sized to actual pat->num_captures*2. Consistent with
the Pike VM's dynamic ncap-sized pool.
Co-authored-by: Claude <noreply@anthropic.com>
Both methods had identical loop bodies, differing only in the
starting group index (1 vs 0). Extracted matchdata_to_ary()
with a from parameter.
Co-authored-by: Claude <noreply@anthropic.com>
Regexp#match and Regexp#=~ shared most of their logic (get pattern,
execute, create MatchData, set globals). Extracted into exec_match()
internal function. Regexp#=~ now calls exec_match() and reads the
match position from the returned MatchData.
Co-authored-by: Claude <noreply@anthropic.com>
The pattern of reading @flags IV and converting to uint32_t was
repeated in 6 methods. Consolidated into a single helper function.
Co-authored-by: Claude <noreply@anthropic.com>
CI_PROC_SET: split NULL/non-NULL proc paths so the compiler can
eliminate the CFUNC/ALIAS checks when proc is a compile-time NULL
(8 of 11 cipush call sites).
cipop: add fast path for the common case where no env and no blk
are set. skips ci_env_set, orphan check, and env_unshare entirely.
most simple method calls (no blocks, no closures) take this path.
Co-authored-by: Claude <noreply@anthropic.com>
Add (?<=...) positive and (?<!...) negative lookbehind support.
The sub-pattern must have a fixed byte length (no quantifiers or
alternation), computed at compile time and stored in the instruction.
At execution time, the engine backs up by that many bytes and runs
the sub-pattern forward. Maximum lookbehind length is 255 bytes.
Co-authored-by: Claude <noreply@anthropic.com>
Collect replacement parts in an array and join at the end instead
of repeated string += which creates intermediate string objects.
Co-authored-by: Claude <noreply@anthropic.com>
When match? calls re_exec with captures=NULL, the Pike VM now
skips all capture pool operations: no pool_copy, no RE_SAVE
writes, no pool compaction. Only a single dummy pool slot is
allocated. This significantly reduces work for boolean matching.
Co-authored-by: Claude <noreply@anthropic.com>
Pre-intern the $1-$9 symbols on first use instead of calling
mrb_intern_cstr (which computes strlen + hash) on every match.
Co-authored-by: Claude <noreply@anthropic.com>
Major changes to the NFA execution engine:
- Thread captures stored in a flat pool sized to actual ncap
(e.g. 4 ints for 1 capture group vs 64 fixed), dramatically
reducing per-thread copy cost
- Generation counter for visited[] eliminates per-step memset
of the entire bytecode-length array
- Pool compaction between steps reclaims dead thread slots
- Backtracking engine also uses dynamic ncap-sized captures
Co-authored-by: Claude <noreply@anthropic.com>
Internal flags (MULTILINE=2, DOTALL=4, EXTENDED=8) differ from
Ruby constants (EXTENDED=2, MULTILINE=4). Convert in C instead
of returning raw internal flags. Also add Regexp#casefold?.
Co-authored-by: Claude <noreply@anthropic.com>
The x flag ignores unescaped whitespace and #comments in patterns,
making complex regexps more readable. Whitespace inside character
classes [...] remains literal. Implemented as a preprocessing step
that strips whitespace/comments before compilation.
Co-authored-by: Claude <noreply@anthropic.com>
Two regexps are equal when they have the same source and flags.
Hash is computed from source string hash mixed with flags.
Co-authored-by: Claude <noreply@anthropic.com>
Regexp#to_s now returns (?flags:source) format (e.g. "(?i:abc)")
instead of the /source/flags format used by Regexp#inspect.
Co-authored-by: Claude <noreply@anthropic.com>
- $1-$9 global variables set by Regexp#match and Regexp#=~
- $1-$9 cleared to nil on match failure
- add mruby-regexp to stdlib.gembox (auto-included in standard builds)
- remove duplicate gem entry from host-debug.rb
Co-authored-by: Claude <noreply@anthropic.com>
implement positive and negative lookahead in the backtracking engine:
- (?=pattern): succeeds if pattern matches at current position
without consuming characters
- (?!pattern): succeeds if pattern does NOT match at current position
new bytecodes RE_LOOKAHEAD and RE_NEG_LOOKAHEAD implemented in
the backtracking engine via nested bt_match calls.
Co-authored-by: Claude <noreply@anthropic.com>
support named capture groups in patterns:
- compiler parses (?<name>...) syntax and builds name table
- MatchData#[:name] and MatchData#["name"] access by name
- MatchData#named_captures returns {name => value} hash
- Regexp#named_captures returns {name => group_number} hash
- named captures stored in mrb_regexp_pattern for GC safety
Co-authored-by: Claude <noreply@anthropic.com>
non-greedy patterns now correctly match the shortest possible
string. patterns with non-greedy quantifiers are dispatched to
the backtracking engine which naturally handles non-greedy
semantics.
the Pike VM continues to be used for purely greedy patterns
(O(n*m) guarantee).
Co-authored-by: Claude <noreply@anthropic.com>
add a recursive backtracking engine that handles \1-\9
backreferences. the Pike VM (NFA) is used for patterns without
backreferences; patterns with backreferences automatically
fall back to the backtracking engine.
the backtracking engine has a step limit (MRB_REGEXP_STEP_LIMIT,
default 1M) to prevent ReDoS on pathological patterns.
also adds SAVE backtracking (save/restore capture positions on
failed branches) for correct submatch tracking.
Co-authored-by: Claude <noreply@anthropic.com>
add tests for: empty pattern, nested captures, word boundary \b,
non-capturing groups (?:), sub/gsub with block, scan with captures,
split with regexp, case/when with regexp, date reformatting.
known limitation: non-greedy quantifiers (*?, +?) currently behave
as greedy. needs match priority tracking (TODO for Phase 2).
Co-authored-by: Claude <noreply@anthropic.com>
- /regex/ literal syntax now works (compiler generates Regexp.compile)
- Regexp#match and Regexp#=~ set $~ global variable
- Regexp.last_match(n) for accessing capture groups
- Regexp.compile as alias for Regexp.new
- Regexp#options
Co-authored-by: Claude <noreply@anthropic.com>
add workload-specific GC tuning advice based on benchmark data:
- allocation-heavy: interval_ratio 400 for ~12% improvement
- real-time: step_limit for bounded pause times
- large buffers: malloc_threshold
- diagnosing GC overhead with GC.stat
Co-authored-by: Claude <noreply@anthropic.com>
split mrb_obj_alloc() into type-validation wrapper and allocation
core (mrb_obj_alloc_core). internal callers (mrb_proc_new,
mrb_env_new) use the core directly, skipping 15+ lines of type
validation per allocation.
most impactful for workloads with heavy Proc/Env allocation
(lambda calculus, block-intensive code).
Co-authored-by: Claude <noreply@anthropic.com>
when all elements are plain String (not subclass) and no block is
given, use specialized sort that calls mrb_str_cmp() directly,
bypassing sort_cmp overhead (GC arena, type dispatch, array
modification check).
includes subclass check to ensure String#<=> is not overridden.
Co-authored-by: Claude <noreply@anthropic.com>
when all elements are integers and no block is given, use
specialized heapify/insertion_sort that compare mrb_int values
directly, bypassing sort_cmp entirely. this eliminates per-comparison
overhead of GC arena save/restore, type checking, and array
modification checks.
the pre-scan to detect all-integer arrays is O(n), negligible
compared to O(n log n) sort. non-integer and block sorts are
unaffected.
Co-authored-by: Claude <noreply@anthropic.com>
two improvements to Array#sort!'s heap sort:
1. hole-style sift-down: save root value, move larger children up
one at a time, write saved value once at the end. reduces
assignments from 3 per level (swap) to 1 per level (move).
2. Floyd's bottom-up heap deletion: during extraction phase, sift
the hole down to a leaf using only child-child comparisons
(~1 comparison per level), then sift up to find the correct
position. this reduces average comparisons from ~2 log n to
~log n per extraction, nearly halving the total comparison
count for the sort.
both changes preserve O(n log n) worst case and O(1) extra space.
Co-authored-by: Claude <noreply@anthropic.com>
when dynamic symbol count reaches MRB_SYMBOL_MAX, run a mark-sweep
pass over all live objects to identify referenced symbols. sweep
unreferenced dynamic symbols, freeing their individually-allocated
string data and marking symtbl slots as tombstones.
mark phase traverses:
- all heap objects (method tables, IV tables, arrays, hashes, envs)
- VM stack values (MRB_TT_SYMBOL)
- call stack method IDs (ci->mid)
- root and current context
after sweep, rebuild hash table to maintain valid collision chains.
this completes the A+ symbol GC plan: the limit acts as a GC
trigger rather than a hard cap. unreferenced DoS symbols are
reclaimed, allowing legitimate code to continue.
Co-authored-by: Claude <noreply@anthropic.com>
dynamic symbols (created via to_sym, send, etc.) now use
mrb_malloc() instead of sym_pool_alloc(). this makes them
individually freeable by future symbol GC.
static symbols (presym, mrb_intern_static, literals) continue
to use the pool allocator for compact storage.
Co-authored-by: Claude <noreply@anthropic.com>
track dynamic (runtime-created) symbols separately from presyms,
inline symbols, and static C API symbols. raise RuntimeError when
the dynamic symbol count exceeds MRB_SYMBOL_MAX (default 4096).
this prevents DoS attacks via unbounded symbol creation (e.g.
"str".to_sym in a loop). presyms and inline symbols are not
counted toward the limit.
infrastructure for future symbol GC: sym_flags array tracks
per-symbol metadata (SYM_FL_DYNAMIC flag).
Co-authored-by: Claude <noreply@anthropic.com>
merge codegen_while/codegen_until into codegen_loop, and
codegen_while_mod/codegen_until_mod into codegen_loop_mod.
each pair differed only in swapped constant-condition checks
(true_always/false_always) and jump opcode (OP_JMPNOT/OP_JMPIF).
Co-authored-by: Claude <noreply@anthropic.com>
replace four repeated `mrb_free(mrb, bin); return MRB_DUMP_WRITE_FAULT`
sequences with a single goto-based cleanup path.
Co-authored-by: Claude <noreply@anthropic.com>
mrb_include_module() and mrb_prepend_module() did not invalidate
the constant cache. stale cache entries caused incorrect constant
resolution after include changed the ancestor chain.
Co-authored-by: Claude <noreply@anthropic.com>
mrb_vm_define_module() and mrb_vm_define_class() incorrectly
reopened modules/classes accessible through include rather than
creating new ones. CRuby only reopens modules directly defined
on the outer scope.
the internal define_module()/define_class() use
mrb_const_defined_at() which walks ancestors for Object class.
bypass them and create modules/classes directly in the VM path.
Co-authored-by: Claude <noreply@anthropic.com>
three functions differed only in the trailing character check ('=',
'?', '!'). replace with a single parameterized function.
Co-authored-by: Claude <noreply@anthropic.com>
both opcodes share identical proc dispatch logic (alias resolution,
callinfo setup, cfunc/irep branching). the only difference is how
nargs is computed (ci_bidx vs operand b).
Co-authored-by: Claude <noreply@anthropic.com>
Decrement gc_debt by the actual number of objects processed
instead of the fixed GC_STEP_SIZE. This makes step_ratio
directly affect debt repayment: larger steps repay more debt,
naturally reducing GC invocation frequency.
Co-authored-by: Claude <noreply@anthropic.com>
Expose gc_debt directly as :debt in GC.stat without sign negation.
The debt model has no threshold ceiling, so :threshold was a
misleading name. Negative debt means credit, positive means GC
is behind on collection work.
Co-authored-by: Claude <noreply@anthropic.com>
Yukihiro "Matz" Matsumoto