Major changes to the NFA execution engine:
- Thread captures stored in a flat pool sized to actual ncap
(e.g. 4 ints for 1 capture group vs 64 fixed), dramatically
reducing per-thread copy cost
- Generation counter for visited[] eliminates per-step memset
of the entire bytecode-length array
- Pool compaction between steps reclaims dead thread slots
- Backtracking engine also uses dynamic ncap-sized captures
Co-authored-by: Claude <noreply@anthropic.com>
Internal flags (MULTILINE=2, DOTALL=4, EXTENDED=8) differ from
Ruby constants (EXTENDED=2, MULTILINE=4). Convert in C instead
of returning raw internal flags. Also add Regexp#casefold?.
Co-authored-by: Claude <noreply@anthropic.com>
The x flag ignores unescaped whitespace and #comments in patterns,
making complex regexps more readable. Whitespace inside character
classes [...] remains literal. Implemented as a preprocessing step
that strips whitespace/comments before compilation.
Co-authored-by: Claude <noreply@anthropic.com>
Two regexps are equal when they have the same source and flags.
Hash is computed from source string hash mixed with flags.
Co-authored-by: Claude <noreply@anthropic.com>
Regexp#to_s now returns (?flags:source) format (e.g. "(?i:abc)")
instead of the /source/flags format used by Regexp#inspect.
Co-authored-by: Claude <noreply@anthropic.com>
- $1-$9 global variables set by Regexp#match and Regexp#=~
- $1-$9 cleared to nil on match failure
- add mruby-regexp to stdlib.gembox (auto-included in standard builds)
- remove duplicate gem entry from host-debug.rb
Co-authored-by: Claude <noreply@anthropic.com>
implement positive and negative lookahead in the backtracking engine:
- (?=pattern): succeeds if pattern matches at current position
without consuming characters
- (?!pattern): succeeds if pattern does NOT match at current position
new bytecodes RE_LOOKAHEAD and RE_NEG_LOOKAHEAD implemented in
the backtracking engine via nested bt_match calls.
Co-authored-by: Claude <noreply@anthropic.com>
support named capture groups in patterns:
- compiler parses (?<name>...) syntax and builds name table
- MatchData#[:name] and MatchData#["name"] access by name
- MatchData#named_captures returns {name => value} hash
- Regexp#named_captures returns {name => group_number} hash
- named captures stored in mrb_regexp_pattern for GC safety
Co-authored-by: Claude <noreply@anthropic.com>
non-greedy patterns now correctly match the shortest possible
string. patterns with non-greedy quantifiers are dispatched to
the backtracking engine which naturally handles non-greedy
semantics.
the Pike VM continues to be used for purely greedy patterns
(O(n*m) guarantee).
Co-authored-by: Claude <noreply@anthropic.com>
add a recursive backtracking engine that handles \1-\9
backreferences. the Pike VM (NFA) is used for patterns without
backreferences; patterns with backreferences automatically
fall back to the backtracking engine.
the backtracking engine has a step limit (MRB_REGEXP_STEP_LIMIT,
default 1M) to prevent ReDoS on pathological patterns.
also adds SAVE backtracking (save/restore capture positions on
failed branches) for correct submatch tracking.
Co-authored-by: Claude <noreply@anthropic.com>
add tests for: empty pattern, nested captures, word boundary \b,
non-capturing groups (?:), sub/gsub with block, scan with captures,
split with regexp, case/when with regexp, date reformatting.
known limitation: non-greedy quantifiers (*?, +?) currently behave
as greedy. needs match priority tracking (TODO for Phase 2).
Co-authored-by: Claude <noreply@anthropic.com>
- /regex/ literal syntax now works (compiler generates Regexp.compile)
- Regexp#match and Regexp#=~ set $~ global variable
- Regexp.last_match(n) for accessing capture groups
- Regexp.compile as alias for Regexp.new
- Regexp#options
Co-authored-by: Claude <noreply@anthropic.com>
merge codegen_while/codegen_until into codegen_loop, and
codegen_while_mod/codegen_until_mod into codegen_loop_mod.
each pair differed only in swapped constant-condition checks
(true_always/false_always) and jump opcode (OP_JMPNOT/OP_JMPIF).
Co-authored-by: Claude <noreply@anthropic.com>
mrb_ccontext functions are implemented in mruby-compiler (y.tab.c),
causing linker errors when the compiler is excluded from the build.
Replace with mrb_load_irep_file() which is in core (src/load.c).
Also fix incorrect *argv in error message and remove dead fname field.
Co-authored-by: Claude <noreply@anthropic.com>
The `mrb` command executes only precompiled RiteBinary (.mrb) files
without depending on mruby-compiler. This enables smaller binaries
for embedded deployments where scripts are precompiled on a
development machine.
Co-authored-by: Claude <noreply@anthropic.com>
ENV is a plain Object with singleton methods and Enumerable,
matching CRuby's behavior. C methods wrap getenv/setenv/unsetenv
with platform support for POSIX, macOS, and Windows.
Co-authored-by: Claude <noreply@anthropic.com>
MPZ_CTX_INIT used a compound literal with designated initializers:
mpz_ctx_t ctx##_struct = ((mpz_ctx_t){.mrb = ..., .pool = ...});
Both features are C99-only, and are not accepted by legacy C++
compilers (notably gcc 4.x) when mruby is pulled into a C++
translation unit via the -cxx.cxx wrapper.
Replace with plain member assignment so the macro expands to code
that is valid under C89/C++98 as well.
Co-authored-by: Claude <noreply@anthropic.com>
Compress the 24-bit aspec into 13 free flag bits on RProc (bits 0-6
and 14-19) when wrapping cfunc methods. Field widths: req/opt 3 bits
(max 7), post/key 2 bits (max 3), rest/kdict/block 1 bit each. Values
exceeding the compressed range are clamped and rest is forced to 1.
This enables Proc#arity and Proc#parameters to return correct results
for cfunc-backed Procs (e.g. from Method#to_proc) with zero memory
overhead -- no struct change needed.
Closes#6764
mrb_str_format captured raw C pointers (p, end) into the format
string's buffer before the main loop. The %s and %p specifiers call
to_s and inspect, which can invoke Ruby code that mutates the format
string via String#replace, freeing or reallocating its buffer. The
loop then continued iterating with dangling pointers, reading freed
memory and potentially leaking adjacent heap contents into the result.
Duplicate the format string with mrb_str_dup() before the loop. This
is O(1) because mrb_str_dup shares the underlying buffer; if the
original is later mutated via String#replace, str_replace decrements
the shared refcount, leaving our duplicate's buffer intact.
Co-authored-by: Claude <noreply@anthropic.com>
String#prepend(s, s) read RSTRING_LEN(argv[i]) in the copy loop after
mrb_str_resize had already updated the receiver's length, causing the
memcpy to write past the allocated buffer.
Detect self-references with mrb_obj_eq() and read from the memmoved
original data at p + total_prepend_len using the captured self_len.
This also handles mixed cases like s.prepend("X", s) where earlier
writes would otherwise corrupt the source of later reads.
Co-authored-by: Claude <noreply@anthropic.com>
The "d" directive in `mrb_get_args()` can be used as an alternative.
Furthermore, NULL checking is unnecessary for the following reasons:
- Incomplete objects from `ary_combination_init()` are not passed to the caller and are garbage collected when `ObjectSpace.each_object` is called, so they are never retrieved
- Even if `state.clone` is called, the `RData::type` of the cloned object is set to NULL, so it is rejected by `mrb_get_args()`
The s = self workaround and XXX comment in recvfrom_nonblock date back
to the initial import of mruby-socket. The underlying bug where self
became a SystemcallException inside ensure blocks has since been fixed.
Verified that self correctly refers to the socket object in ensure
blocks after exceptions from recvfrom.
Change the grammar rule for tLPAREN_ARG from accepting only a
single stmt to accepting compstmt. This allows compound
statements with semicolons inside parenthesized arguments when
the parenthesis is preceded by a space, e.g., `p (f1; f2)`.
This matches the behavior of CRuby 3.3+.
Fixes#6766.
Co-authored-by: Claude <noreply@anthropic.com>
When the block passed to Lazy#flat_map returns a non-enumerable value
(e.g. an Integer), mruby raised NoMethodError because it unconditionally
called #each on the result. CRuby yields non-enumerable values directly.
Use respond_to?(:each) to match CRuby behavior: iterate enumerable
results, yield non-enumerable results as-is.
With `rake -m`, the C compiler can start reading a partially-written
gem_test.c before generation completes. Write to a .tmp file first,
then rename to the final path.
Co-authored-by: Claude <noreply@anthropic.com>
Several methods defined in mruby-array-ext are written in C and may call `mrb_vm_exec()`.
If array objects are modified on the Ruby side, problems may arise in subsequent processing.
- Using objects that have been removed from the array and garbage collected
- Using pointers or array lengths that have become invalid due to changes to the array object
- Modifying the contents of a shared array object directly
ref: https://github.com/mruby/mruby/issues/6662
Replace `__product_group` method with `__product_generate` and `__product_next`.
This change eliminates the need for Ruby to perform internal state calculations, allowing it to simply receive the results.
Yukihiro "Matz" Matsumoto