Major changes to the NFA execution engine:
- Thread captures stored in a flat pool sized to actual ncap
(e.g. 4 ints for 1 capture group vs 64 fixed), dramatically
reducing per-thread copy cost
- Generation counter for visited[] eliminates per-step memset
of the entire bytecode-length array
- Pool compaction between steps reclaims dead thread slots
- Backtracking engine also uses dynamic ncap-sized captures
Co-authored-by: Claude <noreply@anthropic.com>
Internal flags (MULTILINE=2, DOTALL=4, EXTENDED=8) differ from
Ruby constants (EXTENDED=2, MULTILINE=4). Convert in C instead
of returning raw internal flags. Also add Regexp#casefold?.
Co-authored-by: Claude <noreply@anthropic.com>
The x flag ignores unescaped whitespace and #comments in patterns,
making complex regexps more readable. Whitespace inside character
classes [...] remains literal. Implemented as a preprocessing step
that strips whitespace/comments before compilation.
Co-authored-by: Claude <noreply@anthropic.com>
Two regexps are equal when they have the same source and flags.
Hash is computed from source string hash mixed with flags.
Co-authored-by: Claude <noreply@anthropic.com>
Regexp#to_s now returns (?flags:source) format (e.g. "(?i:abc)")
instead of the /source/flags format used by Regexp#inspect.
Co-authored-by: Claude <noreply@anthropic.com>
- $1-$9 global variables set by Regexp#match and Regexp#=~
- $1-$9 cleared to nil on match failure
- add mruby-regexp to stdlib.gembox (auto-included in standard builds)
- remove duplicate gem entry from host-debug.rb
Co-authored-by: Claude <noreply@anthropic.com>
implement positive and negative lookahead in the backtracking engine:
- (?=pattern): succeeds if pattern matches at current position
without consuming characters
- (?!pattern): succeeds if pattern does NOT match at current position
new bytecodes RE_LOOKAHEAD and RE_NEG_LOOKAHEAD implemented in
the backtracking engine via nested bt_match calls.
Co-authored-by: Claude <noreply@anthropic.com>
support named capture groups in patterns:
- compiler parses (?<name>...) syntax and builds name table
- MatchData#[:name] and MatchData#["name"] access by name
- MatchData#named_captures returns {name => value} hash
- Regexp#named_captures returns {name => group_number} hash
- named captures stored in mrb_regexp_pattern for GC safety
Co-authored-by: Claude <noreply@anthropic.com>
non-greedy patterns now correctly match the shortest possible
string. patterns with non-greedy quantifiers are dispatched to
the backtracking engine which naturally handles non-greedy
semantics.
the Pike VM continues to be used for purely greedy patterns
(O(n*m) guarantee).
Co-authored-by: Claude <noreply@anthropic.com>
add a recursive backtracking engine that handles \1-\9
backreferences. the Pike VM (NFA) is used for patterns without
backreferences; patterns with backreferences automatically
fall back to the backtracking engine.
the backtracking engine has a step limit (MRB_REGEXP_STEP_LIMIT,
default 1M) to prevent ReDoS on pathological patterns.
also adds SAVE backtracking (save/restore capture positions on
failed branches) for correct submatch tracking.
Co-authored-by: Claude <noreply@anthropic.com>
add tests for: empty pattern, nested captures, word boundary \b,
non-capturing groups (?:), sub/gsub with block, scan with captures,
split with regexp, case/when with regexp, date reformatting.
known limitation: non-greedy quantifiers (*?, +?) currently behave
as greedy. needs match priority tracking (TODO for Phase 2).
Co-authored-by: Claude <noreply@anthropic.com>
- /regex/ literal syntax now works (compiler generates Regexp.compile)
- Regexp#match and Regexp#=~ set $~ global variable
- Regexp.last_match(n) for accessing capture groups
- Regexp.compile as alias for Regexp.new
- Regexp#options
Co-authored-by: Claude <noreply@anthropic.com>
add workload-specific GC tuning advice based on benchmark data:
- allocation-heavy: interval_ratio 400 for ~12% improvement
- real-time: step_limit for bounded pause times
- large buffers: malloc_threshold
- diagnosing GC overhead with GC.stat
Co-authored-by: Claude <noreply@anthropic.com>
split mrb_obj_alloc() into type-validation wrapper and allocation
core (mrb_obj_alloc_core). internal callers (mrb_proc_new,
mrb_env_new) use the core directly, skipping 15+ lines of type
validation per allocation.
most impactful for workloads with heavy Proc/Env allocation
(lambda calculus, block-intensive code).
Co-authored-by: Claude <noreply@anthropic.com>
when all elements are plain String (not subclass) and no block is
given, use specialized sort that calls mrb_str_cmp() directly,
bypassing sort_cmp overhead (GC arena, type dispatch, array
modification check).
includes subclass check to ensure String#<=> is not overridden.
Co-authored-by: Claude <noreply@anthropic.com>
when all elements are integers and no block is given, use
specialized heapify/insertion_sort that compare mrb_int values
directly, bypassing sort_cmp entirely. this eliminates per-comparison
overhead of GC arena save/restore, type checking, and array
modification checks.
the pre-scan to detect all-integer arrays is O(n), negligible
compared to O(n log n) sort. non-integer and block sorts are
unaffected.
Co-authored-by: Claude <noreply@anthropic.com>
two improvements to Array#sort!'s heap sort:
1. hole-style sift-down: save root value, move larger children up
one at a time, write saved value once at the end. reduces
assignments from 3 per level (swap) to 1 per level (move).
2. Floyd's bottom-up heap deletion: during extraction phase, sift
the hole down to a leaf using only child-child comparisons
(~1 comparison per level), then sift up to find the correct
position. this reduces average comparisons from ~2 log n to
~log n per extraction, nearly halving the total comparison
count for the sort.
both changes preserve O(n log n) worst case and O(1) extra space.
Co-authored-by: Claude <noreply@anthropic.com>
when dynamic symbol count reaches MRB_SYMBOL_MAX, run a mark-sweep
pass over all live objects to identify referenced symbols. sweep
unreferenced dynamic symbols, freeing their individually-allocated
string data and marking symtbl slots as tombstones.
mark phase traverses:
- all heap objects (method tables, IV tables, arrays, hashes, envs)
- VM stack values (MRB_TT_SYMBOL)
- call stack method IDs (ci->mid)
- root and current context
after sweep, rebuild hash table to maintain valid collision chains.
this completes the A+ symbol GC plan: the limit acts as a GC
trigger rather than a hard cap. unreferenced DoS symbols are
reclaimed, allowing legitimate code to continue.
Co-authored-by: Claude <noreply@anthropic.com>
dynamic symbols (created via to_sym, send, etc.) now use
mrb_malloc() instead of sym_pool_alloc(). this makes them
individually freeable by future symbol GC.
static symbols (presym, mrb_intern_static, literals) continue
to use the pool allocator for compact storage.
Co-authored-by: Claude <noreply@anthropic.com>
track dynamic (runtime-created) symbols separately from presyms,
inline symbols, and static C API symbols. raise RuntimeError when
the dynamic symbol count exceeds MRB_SYMBOL_MAX (default 4096).
this prevents DoS attacks via unbounded symbol creation (e.g.
"str".to_sym in a loop). presyms and inline symbols are not
counted toward the limit.
infrastructure for future symbol GC: sym_flags array tracks
per-symbol metadata (SYM_FL_DYNAMIC flag).
Co-authored-by: Claude <noreply@anthropic.com>
merge codegen_while/codegen_until into codegen_loop, and
codegen_while_mod/codegen_until_mod into codegen_loop_mod.
each pair differed only in swapped constant-condition checks
(true_always/false_always) and jump opcode (OP_JMPNOT/OP_JMPIF).
Co-authored-by: Claude <noreply@anthropic.com>
replace four repeated `mrb_free(mrb, bin); return MRB_DUMP_WRITE_FAULT`
sequences with a single goto-based cleanup path.
Co-authored-by: Claude <noreply@anthropic.com>
mrb_include_module() and mrb_prepend_module() did not invalidate
the constant cache. stale cache entries caused incorrect constant
resolution after include changed the ancestor chain.
Co-authored-by: Claude <noreply@anthropic.com>
mrb_vm_define_module() and mrb_vm_define_class() incorrectly
reopened modules/classes accessible through include rather than
creating new ones. CRuby only reopens modules directly defined
on the outer scope.
the internal define_module()/define_class() use
mrb_const_defined_at() which walks ancestors for Object class.
bypass them and create modules/classes directly in the VM path.
Co-authored-by: Claude <noreply@anthropic.com>
three functions differed only in the trailing character check ('=',
'?', '!'). replace with a single parameterized function.
Co-authored-by: Claude <noreply@anthropic.com>
both opcodes share identical proc dispatch logic (alias resolution,
callinfo setup, cfunc/irep branching). the only difference is how
nargs is computed (ci_bidx vs operand b).
Co-authored-by: Claude <noreply@anthropic.com>
Decrement gc_debt by the actual number of objects processed
instead of the fixed GC_STEP_SIZE. This makes step_ratio
directly affect debt repayment: larger steps repay more debt,
naturally reducing GC invocation frequency.
Co-authored-by: Claude <noreply@anthropic.com>
Expose gc_debt directly as :debt in GC.stat without sign negation.
The debt model has no threshold ceiling, so :threshold was a
misleading name. Negative debt means credit, positive means GC
is behind on collection work.
Co-authored-by: Claude <noreply@anthropic.com>
Use -gc_debt as the :threshold key in GC.stat for familiarity.
Positive means credit remaining, negative means GC is overdue.
Co-authored-by: Claude <noreply@anthropic.com>
Replace the threshold-based GC trigger (gc->threshold vs gc->live)
with a debt model (gc->gc_debt). Each allocation increments debt;
each GC step decrements by GC_STEP_SIZE. When a cycle completes,
credit is proportional to live_after_mark * interval_ratio, giving
a natural feedback loop that adapts to allocation rate.
Co-authored-by: Claude <noreply@anthropic.com>
GC.step_limit caps the per-step work in incremental GC,
enabling more predictable pause times for real-time use.
GC.malloc_threshold triggers GC based on allocation bytes,
addressing memory pressure from large buffers.
Both default to 0 (disabled), preserving existing behavior.
Co-authored-by: Claude <noreply@anthropic.com>
mrb_ccontext functions are implemented in mruby-compiler (y.tab.c),
causing linker errors when the compiler is excluded from the build.
Replace with mrb_load_irep_file() which is in core (src/load.c).
Also fix incorrect *argv in error message and remove dead fname field.
Co-authored-by: Claude <noreply@anthropic.com>
The `mrb` command executes only precompiled RiteBinary (.mrb) files
without depending on mruby-compiler. This enables smaller binaries
for embedded deployments where scripts are precompiled on a
development machine.
Co-authored-by: Claude <noreply@anthropic.com>
Leaf types (String, Integer, BigInt, Complex, CPTR) have no children
besides their class pointer. Mark them black immediately in
mrb_gc_mark() instead of pushing to the gray stack, reducing gray
stack pressure and overflow frequency.
Co-authored-by: Claude <noreply@anthropic.com>
Return a Hash with GC statistics: live, threshold, state,
generational, full. When MRB_GC_STATS is defined, also includes
total, minor, major counters.
Co-authored-by: Claude <noreply@anthropic.com>
Add gc_total_count, minor_gc_count, major_gc_count (uint32_t) to
mrb_gc, guarded by MRB_GC_STATS. Zero cost when disabled.
Co-authored-by: Claude <noreply@anthropic.com>
Phase 2 of opcode handler extraction. these opcodes use
L_SEND_SYM/L_SENDB_SYM fallback for generic method dispatch when
the fast path (Array/Hash/String/Integer/Float) does not apply.
add VM_SEND_SYM and VM_SENDB_SYM return codes. move TYPES2 macro
to file scope for use by vm_op_div.
Co-authored-by: Claude <noreply@anthropic.com>
Yukihiro "Matz" Matsumoto