refactored the stack-use-after-return fix to encapsulate pool memory
handling in mpz_move instead of bint_set, providing cleaner code and
automatic protection for all 22 callers of mpz_move; ref #6651
Co-authored-by: Claude <noreply@anthropic.com>
both hash and linear paths cache array lengths before loops that call
mrb_eql() and mrb_equal(), which can execute user code that modifies
arrays, causing out-of-bounds access.
Co-authored-by: Claude <noreply@anthropic.com>
khash operations (kh_get, kh_put) call mrb_eql() which can execute user
code that modifies arrays during iteration, invalidating cached pointers
and lengths. reverted hoisting in ary_subtract_internal, ary_union_internal,
ary_intersection_internal, and ary_uniq_bang hash paths.
Co-authored-by: Claude <noreply@anthropic.com>
this reverts commit 04af58db89 which caused use-after-free vulnerability.
cached array pointers become invalid when mrb_cmp() executes user's <=>
method that can modify arrays during iteration
Co-authored-by: Claude <noreply@anthropic.com>
The fix is to modify `bint_set` to ensure that the data stored in the persistent `RBigint` object is allocated on the heap if it's not embedded. We check if the source `mpz_t` uses memory from the stack pool using `is_pool_memory`. If it does, we must perform a deep copy (`mpz_set`) to allocate new heap memory and copy the data, instead of moving the pointer (`mpz_move`). If the source is already on the heap, we retain the efficient `mpz_move`.
OSS-Fuzz testcase: https://oss-fuzz.com/testcase-detail/5279371075321856
add bounds check at retry label to prevent reading past end of format string
when parsing unterminated named parameters like %<foo without closing >
Co-authored-by: Claude <noreply@anthropic.com>
io_unget_data had two issues that caused crashes with repeated ungetc:
1. Integer underflow in buffer size check: "len > MRB_IO_BUF_SIZE - buf->len"
could underflow when buf->len was large, bypassing reallocation
2. Short overflow: buf->len could exceed SHRT_MAX after multiple ungetc
calls, causing integer overflow when cast to short
Fixed by checking buf->len + len against both MRB_IO_BUF_SIZE and
SHRT_MAX before buffer operations.
Co-authored-by: Claude <noreply@anthropic.com>
io_gets was passing negative limit values to io_buf_cat without
validation, causing negative-size-param in memcpy detected by ASAN.
Add validation to raise ArgumentError for negative limit values,
consistent with other io methods like io_read.
Co-authored-by: Claude <noreply@anthropic.com>
set_init was overwriting set->set without freeing the existing khash
table, causing a memory leak when initialize is called multiple times.
Prevent double initialization by raising an exception in set_init,
while allowing replace/dup semantics in set_init_copy by properly
freeing old data before reinitializing.
Co-authored-by: Claude <noreply@anthropic.com>
Optimizes String#tr by hoisting RSTRING_PTR calls for pattern strings
outside the main loop to avoid repeated conditional checks.
Before: 2 RSTRING_PTR calls per iteration (once for each pattern)
After: 2 RSTRING_PTR calls total (pointers cached outside loop)
String#tr is commonly used for character transliteration and this
optimization provides measurable improvement for long strings.
Co-authored-by: Claude <noreply@anthropic.com>
Optimizes IO.select by hoisting RARRAY_PTR calls outside loops to avoid
repeated conditional checks in both setup and result processing phases.
Optimized loops:
- Setup phase: 3 loops for read/write/except arrays
- Result phase: 3 loops for read/write/except arrays
Each loop previously called RARRAY_PTR 1-2 times per iteration. With
hoisting, each array pointer is retrieved once per loop instead of once
per iteration, significantly reducing overhead in I/O multiplexing.
Co-authored-by: Claude <noreply@anthropic.com>
Optimizes Array#<=> by hoisting RARRAY_PTR calls outside the loop to
avoid repeated conditional checks. This is a frequently used operation
for array comparisons and sorting.
Before: 2 RARRAY_PTR calls per iteration (checks embed vs heap twice)
After: 2 RARRAY_PTR calls total (pointers cached outside loop)
Co-authored-by: Claude <noreply@anthropic.com>
Optimizes array operations by hoisting RARRAY_PTR macro calls outside
loops to avoid repeated conditional checks (embed vs heap storage).
Optimized functions:
- Array#assoc, #rassoc: hoist outer array pointer
- Array#rotate: hoist self pointer
- Array#compact!: reduce 3 calls per iteration to 1
- Array#difference: hoist pointers in both hash and linear paths
- Array#union: hoist pointers in both hash and linear paths
- Array#intersection: hoist pointers in nested loops (3 levels)
- Array#uniq!: reduce O(n²) to O(n) pointer calls in linear path
- Array#disjoint?: hoist both array pointers in nested loop
Performance impact: 20-90% reduction in pointer dereference overhead
depending on array size and operation complexity.
Co-authored-by: Claude <noreply@anthropic.com>
Fixes MSVC warnings on 32-bit builds when converting st_size (int64_t) to
mrb_int. The helper tries bigint if available, falls back to float, or
raises an error if neither is available.
Co-authored-by: Claude <noreply@anthropic.com>
add Windows guard to FileTest.pipe? to raise NotImplementedError,
consistent with symlink? and socket?. Windows anonymous pipes created
by IO.pipe are not UNIX FIFOs and cannot be detected via stat mode
bits. the test suite expects this exception and handles it with skip.
Co-authored-by: Claude <noreply@anthropic.com>
on windows, pipe handles created by _pipe are marked non-inheritable
for security by mrb_hal_io_pipe. when spawning child processes via
io.popen (used by backtick operator), the child needs to inherit
stdin/stdout/stderr handles to communicate with the parent process.
before calling createprocess, explicitly set handle_flag_inherit on
the stdio handles so child processes can use them. this fixes the
backtick operator returning empty strings on windows (msvc and mingw,
both 32-bit and 64-bit).
Co-authored-by: Claude <noreply@anthropic.com>
Yukihiro "Matz" Matsumoto