This commit refactors the str_succ_bang function in mrbgems/mruby-string-ext/src/string.c to perform string succession in-place.
The previous implementation used an intermediate temporary string, which incurred memory and performance overhead.
This change aims to reduce memory usage and improve performance by modifying the string buffer directly, using mrb_str_resize and memmove when prepending characters is necessary (e.g., '9' -> '10').
Formatting changes (newlines before 'else') have also been applied as per user request.
- Remove circular reference check in favor of max depth check only
- Fix memory leak by properly handling errors in set_flatten_bang
- Simplify code by reducing variables and unifying error handling
Add two helper functions to improve code clarity and maintainability:
- set_check_type: Checks if a value is a Set and raises an error if not
- set_is_set: Checks if a value is a Set and returns a boolean result
Update all relevant methods to use these helper functions, reducing
code duplication and centralizing type checking logic.
Reimplemented the following methods in C for improved efficiency:
- superset? and proper_superset? (>= and >)
- subset? and proper_subset? (<= and <)
- intersect? and disjoint?
- <=> comparison operator
Renaming all internal C methods from __set_* to a cleaner __* convention
(e.g., __set_merge is now __merge), and updating their call sites in the
Ruby code accordingly.
We refactor out loop by set_khash_foreach() function, so that we don't
need to repeat for loop. it makes the code simpler. The code is written
by Atlassian Rovodev.
Define generic operation method in Ruby, then prepare fast-path function
in C as a general structure. Some methods will follow this pattern.
The code is generated by Atlassian Rovodev.
This commit adds Doxygen-style comments to several MRB_API functions
in src/vm.c to improve code readability and documentation.
The following functions were commented:
- mrb_stack_extend
- mrb_protect_error
- mrb_funcall
- mrb_funcall_id
- mrb_funcall_with_block
- mrb_funcall_argv
- mrb_yield_with_class
- mrb_yield_argv
- mrb_yield
- mrb_vm_run
- mrb_vm_exec
- mrb_top_run
Additionally, the parameter name 'self' in mrb_yield_with_class was
renamed to 'self_obj' for better clarity and consistency with the new comment.
This commit corrects the placement of C-style block comments
for MRB_API functions in src/variable.c. Comments are now
placed directly before each function definition as per standard
documentation practices.
This commit adds descriptive comments to the following MRB_API functions in src/state.c, clarifying their purpose and functionality:
- mrb_open_core
- mrb_open
- mrb_free_context
- mrb_close
- mrb_add_irep
- mrb_top_self
- mrb_state_atexit
This commit adds Doxygen-style comments to the following MRB_API functions in src/proc.c:
- mrb_proc_new_cfunc
- mrb_proc_new_cfunc_with_env
- mrb_closure_new_cfunc
- mrb_proc_cfunc_env_get
The comments describe the purpose, parameters, and return value of each function.
This commit adds descriptive C-style comments to the following functions
in `src/kernel.c`:
- `mrb_func_basic_p`: Explains that the function checks if an object's method is implemented by a specific C function.
- `mrb_obj_freeze`: Explains that the function freezes an object, preventing further modifications.
- `mrb_obj_is_instance_of`: Explains that the function checks if an object is an instance of a given class.
This commit adds Doxygen-style comments to several MRB_API functions
in the `src/etc.c` file. These comments explain the purpose,
parameters, and return values of these functions, improving code
readability and maintainability.
The following functions were commented:
- mrb_data_object_alloc
- mrb_data_check_type
- mrb_data_check_get_ptr
- mrb_data_get_ptr
- mrb_obj_to_sym
- mrb_obj_id
- mrb_word_boxing_float_value
- mrb_word_boxing_value_float
- mrb_word_boxing_cptr_value
- mrb_boxing_int_value
This commit adds descriptive comments to the following functions:
- mrb_range_ptr
- mrb_range_new
- mrb_range_beg_len
The comments were written by Google Jules.
This commit adds C-style block comments to all MRB_API functions
defined in the src/class.c file. The comments explain the purpose
of each function, its parameters, and its return value, aiming to
improve code readability and maintainability.
This commit adds Doxygen-style comments to the following functions
in src/dump.c, ensuring "@brief" is not used:
Non-static functions (comments reviewed/updated):
- mrb_dump_irep
- mrb_dump_irep_binary
- mrb_dump_irep_cfunc
Static functions (new comments added):
- write_irep_header
- write_iseq_block
- dump_float (if compiled)
- get_pool_block_size
- write_pool_block
- get_syms_block_size
- write_syms_block
- get_irep_record_size
This work is part of a larger effort to document all non-trivial
functions in this file as per your feedback. The remaining functions
will be documented in subsequent commits.
This commit adds descriptive comments to all functions marked with MRB_API
in the `src/symbol.c` file. The comments explain the purpose, parameters,
and return values of these functions, improving code readability and
maintainability.
The following functions were commented:
- mrb_intern
- mrb_intern_static
- mrb_intern_cstr
- mrb_intern_str
- mrb_intern_check
- mrb_check_intern
- mrb_intern_check_cstr
- mrb_check_intern_cstr
- mrb_intern_check_str
- mrb_check_intern_str
- mrb_sym_name_len
- mrb_sym_str
- mrb_sym_name
- mrb_sym_dump
This change adds C-style multiline comments to all functions
marked with MRB_API in the src/array.c file.
The comments explain each function's purpose, its parameters,
and what it returns, where applicable. This improves the
readability and maintainability of the C API for mruby arrays.
The `@brief` markup was intentionally avoided as per your
requirements.
This commit adds C-style descriptive comments to MRB_API functions
in `src/hash.c` that are intended for use as part of mruby's C API.
The comments are targeted at C developers using these functions directly.
Comments were added or updated for the following functions:
- mrb_hash_new: Added comment.
- mrb_hash_new_capa: Updated existing comment to be more C API user-centric.
- mrb_hash_dup: Added comment.
- mrb_hash_get: Added comment.
- mrb_hash_fetch: Added comment.
- mrb_hash_set: Added comment.
- mrb_hash_delete_key: Added comment.
- mrb_hash_size: Added comment.
- mrb_hash_merge: Added comment.
- mrb_hash_foreach: Comment updated in a previous phase of work.
This work aligns with the guideline to comment C-facing MRB_API functions,
while avoiding adding new C-API comments to those MRB_API functions
that solely implement Ruby methods and already have extensive Ruby
method documentation (call-seq) in the source. The @brief markup
was avoided as per the original issue request.
This commit adds and updates C-style block comments for all functions marked with MRB_API in src/object.c.
The comments explain the purpose, parameters, and return values of these functions, adhering to the project's documentation style and avoiding the use of '@brief' markup.
Existing comments were also reviewed and updated for clarity and consistency.
Add descriptive comments for MRB_API functions in src/string.c
This commit adds descriptive comments to various MRB_API functions
within the src/string.c file. These comments aim to improve code
readability and maintainability by explaining the purpose,
parameters, and return values of these functions.
This change adds Doxygen-style comments to the public API functions
in `src/debug.c`, including mrb_packed_int_len, mrb_packed_int_encode,
and mrb_packed_int_decode. The comments explain the purpose of each function,
its parameters, and its return value. This improves the readability
and maintainability of the code.
This commit adds Doxygen-style comments to the public functions,
internal static functions, and structs within the `src/mempool.c` file.
These comments clarify the purpose, parameters, and return values (where applicable)
of these code elements, improving code readability and maintainability.
The following elements were commented:
- struct mempool_page
- struct mempool
- ALIGN_PADDING macro
- mempool_open()
- mempool_close()
- page_alloc()
- mempool_alloc()
- mempool_realloc()
This commit adds a descriptive comment at the beginning of the `mrb_read_float` function in `src/readfloat.c`.
The comment explains the function's purpose, its parameters (`str`, `endp`, `fp`), and its return value (`TRUE` or `FALSE`). This improves code readability and understanding.
The sorted array binary search implementation no longer uses the inline
cache array, so remove all MRB_INLINE_METHOD_CACHE definitions and
related code.
By replacing the open‐addressing hash with a sorted array and binary
search, we eliminate tombstone management and improve cache locality of
method entries. This preserves the original contiguous values+keys
layout and peak memory usage, while simplifying growth logic and
delivering lookup performance gains.
Preserve the original iv_tbl heap layout and allocation pattern, but
switch iv_put to maintain sorted keys and iv_get/iv_del to perform
binary search. This eliminates extra probing overhead, improves
read-heavy lookup performance for small tables, and incurs zero
additional allocations.
My previous attempts to improve method table (mt_tbl) performance by introducing a load factor and adjusting the initial allocation size unfortunately led to undesirable increases in memory consumption.
Given that memory usage is a primary concern, I've reverted the following changes:
- Removed the load factor based rehashing logic.
- Removed the related definitions for the load factor.
- Ensured the initial allocation size is 8.
This restores the method table to its original behavior, where it rehashes only when the table is completely full or during initial allocation. This should bring memory usage back to its baseline level prior to these optimization attempts.
The method table (mt_tbl) in src/class.c previously only rehashed when it became completely full. With linear probing, this could lead to significant performance degradation for lookups and insertions as the table approached full capacity.
This change introduces a load factor (MT_LOAD_FACTOR_NUM/MT_LOAD_FACTOR_DEN, set to 3/4 or 0.75). The mt_put function now checks if adding a new element would cause the table's size to meet or exceed this load factor relative to its allocated capacity. If so, it triggers a rehash before inserting the new element.
This helps maintain more empty slots in the hash table, improving the average-case performance of linear probing and reducing the likelihood of worst-case scenarios. The existing initial allocation size and doubling strategy for rehashing are retained.
ADDI/SUBI may fall back to method call that may clear block argument
place holder, which may be a live register. So we cannot directly call
ADDI/SUBI over local variables.
Since we have introduced lrama, everyone can generate same `y.tab.c`
on any platform, without installing Bison. That was the reason we have
removed `y.tab.c` from the repository. But this change cause #6515 and
bothered out-of-tree builds. So we (reluctantly) added `y.tab.c` again.
mempool.h provide compatibility layer so that existing programs does not
need to update (but update recommended anyway, since compatibility layer
takes mrb_state that is not used at all).
Since mrb_calloc() (along with mrb_calloc) takes two arguments: nmemb
which is number of array elements, and size which is size of the array.
Of course, revsersing does not change the behavior, but we'd like to
respect the original design intention of calloc(3).
Along with removing mrb_state first argument from the function. From
mruby 3.2, this function is *not* the default function, but the entry
point that can be redefined for the application. The function in
`src/allocf.c` is the default *implementation* (using malloc / realloc /
free) of the function.
Since the function returns either callinfo or env, thus the name does
not describe the current behavior. In addition, we did some refactoring
on the function.
Prettier uses an ignore file to exclude files from formatting when
running with pre-commit and also when running standalone from the command line.
This is detailed on the CLI docs page:
https://prettier.io/docs/cli
The technique is called "double dispatch" (that was popular in
Smalltalk), but it does not work well with mruby. It's slower and
consumes more memory. Even thought `#append_features` defined in ISO
standard (15.2.2.4.11), we decided to remove it. Strictly speaking, it
is mruby limitation. And it should be documented clearly.
There was a problem with visibility state from proc that straddles a fiber or is independent.
Therefore, it has been changed to give priority to env objects, if any.
Also, added "separate module" flag to block traversal to a higher level env object.
Note that the "separate module" flag is now set when calling blocks with the `mrb_yield_with_class()` function.
fixed https://github.com/mruby/mruby/issues/6494
For big numbers (especially integers close to MRB_INT_MAX), float
conversion may be lose precisions. So we try not to convert numbers, but
just compare.
Ideally, the default visibility should be stored in the scope. But for
the time being mruby stores the visibility in the class/module. As a
result, nesting class/module reopening or class_eval/module_eval could
cause incompatibility. We will try to fix them in the future.
The reason for this is to fix the following problems
- Even on non-Windows, drive letter recognition was still being handled.
- On Windows, paths starting with a slash were not expanded correctly.
- On Windows, relative paths containing drive letters behaved differently than in CRuby.
Due to reimplementation, the internal methods `File._concat_path`, `File._gethome` and `File._getwd` methods have been removed.
The difference from CRuby's Encoding:
- Encoding is a module, instead of a class
- each Encoding (e.g. `Encoding::UTF_8) is a string instead of Encoding object
- only supports `UTF-8` and `ASCII-8BIT` (and its alias `BINARY`)
Using this gem automatically turn on `MRB_UTF8_STRING` support.
The old name `str_modify_keep_ascii` does not describe the function
behavior. It was named so since it was separated from the function
`mrb_str_modify_keep_ascii`.
The purpose is as follows:
- Stop using `mrb_locale_from_utf8()`.
- Because there is no corresponding `mrb_utf8_from_locale()`.
- Because on Windows, for example, if the code page is 932 (CP932, likely ShiftJIS), it cannot be distinguished from the second byte 0x5c (\), and returns wrong results.
- Stop using `dirname(3)`.
- Because leading consecutive slashes are not truncated.
For example, if `/////a/b` is given, CRuby returns `/a`, but mruby so far returns `/////a`.
- Because the `path` argument cannot be passed in an immutable form.
- Stop using `_splitpath()` in the Windows implementation.
- Because there is no support for UNC paths with up to 32767 characters.
ref. https://learn.microsoft.com/ja-jp/dotnet/standard/io/file-path-formats#unc-paths
- Because modifying the result of paths terminated by a directory separator.
Previously, for example, `C:/` would return `C:.` instead of `C:/`, and `a/b/` would return `a/b` instead of `a`.
Change to `MRB_USE_IO_PREAD_PWRITE` for consistency with mruby configuration macros.
Similarly, `MRB_WITHOUT_IO_PREAD_PWRITE` is changed to `MRB_NO_IO_PREAD_PWRITE`.
The previous names are available for compatibility but are deprecated.
- The inner method `File._gethome` could be read as intending to use the `USERPROFILE` environment variable instead on Windows when the `HOME` environment variable is not available.
In reality, however, this was not the case.
- The result of `File.expand_path` should unify path separators with `/`, but it did not.
Since `mruby-io` does not depend on `mruby-env` even for test builds, it is impossible that `ENV` constants are defined.
Therefore, define `MRubyIOTestUtil::ENV_HOME` for alternative use.
OP_LOADI stores an 8 bit integer to a register, so we renamed the
instruction name to describe the behavior more precisely, like
OP_LOADI16 and OP_LOADI32.
Add `OP_NOP` to distinguish `retry` and jump targets while maintaining instruction compatibility.
Ideally, it might be preferable to separate them into `OP_REDO`.
fixed#6439
Recent changes make mrb_irep_remove_lv() used no longer. Removing this
function would not make any compatibility issue, since it's an internal
function.
In `src/hash.c`, there are code blocks that are passed as macro arguments.
These code blocks are interpreted as part of the macro function, so breakpoints cannot be set in the debugger.
Also, the gcov command will aggregate them to the caller, and the code in the block will not be counted.
This patch will prevent them from being interpreted as part of a macro, and thus the aforementioned problems will no longer occur.
If a tombstone (a deleted entry slot) is found in searching the entry,
it should be skipped, but we had added the new entry even if the entry
to be replaced might be found in the further search. #6414 and #6421
tried to rehash the table to remove tombstone. But rehashing consumes
memory. So for the time being, we just skip tombstones in the search.
Maybe we will add some heuristics to rehash when the table has too many
tombstones. close#6414
Some functions called by `mrb_vm_exec()` involve re-entry into the mruby VM.
If the `ci` variable is not updated after re-entry, use-after-free is caused.
This patch makes the following after-call fixes.
| called | might call methods
| ----------------------- | ----------------
| `mrb_ary_splat()` | `#to_a`
| `hash_new_from_regs()` | `#eql?` `#hash`
| `mrb_hash_delete_key()` | `#eql?` `#hash`
| `mrb_hash_get()` | `#eql?` `#hash` `#default`
| `mrb_hash_key_p()` | `#eql?` `#hash`
| `mrb_hash_merge()` | `#eql?` `#hash`
| `mrb_hash_set()` | `#eql?` `#hash`
| `mrb_range_new()` | `#<=>`
Under normal circumstances, `rake -m test` will parallelize not only the build task, but also the test task.
With this patch, `rake -m test:run:serial` will parallelize the build tasks, but the test tasks will be done one by one in sequence.
The name of the task to be added is as follows:
| tasks to be added | corresponding exist tasks
| --------------------- | ----------------
| `test:run:serial` | `test:run`
| `test:run:serial:bin` | `test:run:bin`
| `test:run:serial:lib` | `test:run:lib`
CRuby 3.4 puts spaces around `=>` since for example `{:a!=>2}` can be
confusing where to separate tokens. mruby should follow the behavior.
Many tests in `test/t` directory assumed no spaces around `=>`, so we
needed to fix them too.
The documentation table is generated by the `rake doc:update-index` command.
The following conditions must be met for links to be added to the documentation table.
- The file must be placed under the `doc/` directory
- The file must have the extension `.md`
- The file must be written at the top of the file with `<! -- summary: ANY-TEXT -->`
When multiple identical proc objects are placed on the call stack, it is not possible to distinguish where to `return`.
Therefore, use env object comparisons to do this.
fixed#6411
Introduce the `MRuby::Build#install_excludes` attribute.
This attribute is an array to which you add strings, regular expressions, and proc objects that will serve as filters to exclude.
This feature was inspired by @hoshiumiarata's comment.
https://github.com/mruby/mruby/issues/6352#issuecomment-2426721517
To avoid confusion with pools in irep, we renamed region-based memory
manager from pool to mempool.
- rename pool.c to mempool.c
- separate mempool.h
- rename all mrb_pool to mrb_mempool
So if someone is using pool.c functions (I suppose no one does though),
they need to rename all `mrb_pool` to `mrb_mempool` and include
`mruby/mempool.h` header at the top.
mrb_pool_value is a structure that represents a value in the irep
literal pool and is unrelated to mrb_pool, which performs region-based
memory management. It has been renamed mrb_irep_pool to avoid confusion.
When called in combination with a method like `*_eval` or `*_exec` that switches self, `__send__` was passed an object that was not necessarily a symbol as the method name.
This problem was discovered during the #6389 correction process.
C to Ruby calls using `mrb_exec_irep()` were not forwarding arguments.
There was also a problem in setting the target class and method ID, which is also fixed.
This issue was discovered during the work to fix#6389.
However, on C, there is no easy way to pass keyword arguments.
Therefore, when called `Kernel#instance_exec` on C, keyword arguments are converted to positional arguments.
This is a limitation of current mruby.
fixed#6389
Calling `mrb_gc_unregistor()` from `mrb_data_type::dfree` caused a use-after-free deep inside `mrb_close()`.
The impetus to investigate was <https://github.com/mruby/mruby/pull/6342#pullrequestreview-2292747530>.
Currently, when `mrb_close()` is called, all objects are destroyed first.
The process is done heap page by heap page, and when all objects belonging to a heap page are destroyed, the heap page is released.
If the next heap page contains `RData` objects, the `mrb_gc_unregistor()` function may be called from the `mrb_data_type::dfree` function.
At this time, the `mrb_gc_unregistor()` function gets an array object from a Ruby global variable.
If the array object belongs to a freed heap page, use-after-free is established by referencing this array object.
About the fixes.
First of all, there is the fact that the `mrb_gv_get()` function returns `nil` if `mrb->globals` is `NULL`.
Therefore, before destroying all objects, free `mrb->globals` and set `mrb->globals` to `NULL` at the same time.
Now the `mrb_gv_get()` function will return `nil` to the calling `mrb_gc_unregistor()` function and `mrb_gc_unregistor()` will do nothing more.
ref. https://github.com/mruby/mruby/issues/4618
If the operand is a small integer, those functions tried to reduce
bigint allocations, but we had some bugs in them. We removed those
imperfect optimization altogether.
Once the class is set, objects can be referenced and manipulated from the Ruby side by using `ObjectSpace.each_object`.
Also, currently `mrb_gc_unregister()` assumes that the element is a non-immediate object.
However, `mrb_gc_unregister()` does not read or write to the address, so there was no problem.
https://github.com/gitleaks/gitleaks
Adding another check/test to our pre-commit framework.
gitleaks is a popular tool that helps with security.
Removes the gitleaks check from the Super-Linter.
So now we can run gitleaks with pre-commit on `git commit`
As #6359 pointed out, calling const_missing hook from E_XXX_ERROR (that
calls mrb_exc_get_id()) can be an attack vector. Since E_XXX_ERROR is
supposed to be a defined error class, we think that the situation where
it is undefined and the const_missing hook is called should be detected
as an error; fix#6359
The pull-request #6371 was tight integration of mpz and bint functions.
The mpz functions take `struct RBigint*` instead of `mpz_t*`. It
decrease maintainability, in my opinion. This commit initializes `mpz_t`
from `struct RBigint*` in bint functions, so that we can keep separation
of function roles.
Both functions are only called from mpz_init_set_str(). We can assume
- mpz_t is modifiable
- mpz_t is positive
- n is positive and small (n <= 36)
Those new definitions consume less memory and are slightly faster.
We tried many times to implement the Karatsuba method to improve the
performance of multiplication of large multi-precision integers. But it
did not speed up in all cases due to the cost of memory allocation. We
decided to go back to the basic multiplication method.
If anyone wants to take on the challenge of improving the performance of
multiplication, we welcome it.
To mark `MRB_PROC_ORPHAN` we need to keep track of passed block, even
after the assignment to the block argument. And `yield` should use the
original block; #5786, #5791, #6369
The callinfo refers blk since #5786 but not marked at the time. Later we
added reclamation check by #5791 but its repeated heap scans decrease
the performance drastically in some cases. So the original @dearblue's
solution should be taken
Probably we need to always keep the original block at the bottom of
arguments. And the explicit block argument should be a normal local
variable. We will investigate it later.
This reverts commit e76bebe836 (#6309).
Because it crashes limited to gcc13 -O3.
ref. #6358.
Also, benchmark tests have shown that revert tends to be preferable in this time.
Validate GitHub Actions with pre-commit.
Remove the Super-Linter GitHub Actions check.
It is more useful to run "actionlint" with pre-commit
since the hooks run on our local machines on git commit.
We also run pre-commit on GitHub.
Whereas the Super-Linter tests only run on GitHub.
https://github.com/rhysd/actionlint/blob/main/docs/usage.md#pre-commit
The new version gives more accurate values for decimal number
representation that are not divisible in binary representations, for
example `0.3`.
The function uses `long double` for precision. Please report if `long
double` causes problems on any platform (especially microcontrollers).
Ref #6182
The `mrb_get_argv()` function and the `*` specifier of `mrb_get_args()` get the address of the argument.
At this time, if it is passed in the form of a splat argument, it will be an address to an element of an array object.
After getting the pointer to the array object, the caller may call `mrb_vm_exec()` directly or indirectly.
At this time, a splat argument with the class set can be retrieved as an array object by searching with `ObjectSpace.each_object`.
If changes are made as array objects, addresses on the heap as arrays may become invalid, or objects in the array may be recycled by the GC.
When the caller references the changed address in a subsequent operation, use-after-free is established.
This patch assigns `NULL` as the class of the array object so that it cannot be detected by `ObjectSpace.each_object` from the Ruby side.
`mrb_equal()` may call `obj.==` method internally.
Therefore, using an unupdated pointer and length after `mrb_equal()` could result in a read/write to an invalid address.
Fresh properties must always be obtained regardless of the result of `mrb_equal()`.
Also, `ary_modify()` must be called each time before writing.
ref. #6339
In some environments, the test will fail because the directory in use cannot be deleted.
This problem was encountered when building 32-bit binary with mingw32 on FreeBSD and running on wine.
Suppress warnings for 0-length sequences is not required.
By commit f1a02dff58, it was introduced.
By commit 24939723d7, pseudo-variable length arrays are now used and the warning suppression is no longer needed.
By commit e8841fbf58, moved the intervening code.
The C local variable is not protected from GC, so we use the function
mrb_gc_protect() to keep the value. We also keep the arena position by
mrb_gc_arena_save(), then restoring the position for every new return
value, to minimize arena size.
Small cosmetic changes (pre-increment to post-increment) are also made
in this commit.
When calling `mrb_equal()` or `mrb_funcall()` family functions, the GC arena should be restored if the loop is repeated by a non-immediate return value.
In my opinion, restoring the GC arena is unnecessary when a non-immediate (true) value causes the function to return (e.g. the `mrb_ary_index_m()` function).
The patch does not take into account the case of recursive calls and may be incomplete.
The `mrb_ary_cmp()` function calls `mrb_cmp()` for comparison, but `mrb_cmp()` may call the `obj.<=>` method internally.
If a user-defined `<=>` method is called and the array object under comparison is expanded or reduced, a reference to an invalid address may subsequently be made.
We assumed there's no need for gc_arena_keep() when MRB_GC_FIXED_ARENA
is set. But it turned out that gc_protect() still can cause use-after-free
with fixed arena.
Revert "gc.c (gc_protect): should not call gc_arena_keep twice from allocation"
This reverts commit 28ece4ed8b.
Revert "gc.c (gc_arena_keep): reorganized for MRB_GC_FIXED_ARENA; ref #6329"
This reverts commit 33dd623a02.
Static proc objects defined as methods may be placed in 4-byte alignments in 32-bit environments.
This may be misinterpreted as an immediate value depending on the address.
Since C11 and C++11 have additional language features for byte alignment, corresponding compilers use them to define the `mrb_alignas()` macro.
For earlier compilers, they use their own extensions to define the `mrb_alignas()` macro.
GCC supports `__attribute__((aligned(alignment)))` since at least version 2.95.3 (1999).
https://gcc.gnu.org/onlinedocs/gcc-2.95.3/gcc_4.html#IDX305
According to GPT-4, support was added in version 2.7 (1995).
It is not known which version of Visual C++ added support for `__declspec(align(n))`.
According to GPT-4, at least Visual C++ 6.0 (1998) seems to support it.
Also, the documentation of past Intel C/C++ compilers that support `__declspec(align(n))` makes reference to support with Visual C++ 4.2 (1996).
https://www.intel.com/content/dam/www/public/ijkk/jp/ja/documents/developer/ccomp40j.pdf
“mruby-compiler” should be able to generate `y.tab.c` files through a separate build configuration if it is not added to the ‘host’ build.
In the example above, the “host/mrbc” build should generate the `y.tab.c` file.
When GC occurs during the expansion of the GC arena by `gc_protect()` in `mrb_obj_alloc()`, the object page just allocated by `add_heap()` is released.
Therefore, as soon as control returns from `gc_protect()`, there is a possibility of illegal writing or reading to the address just released.
This issue was discovered during the investigation of #6326.
The following assertions can be added to omit the `if` block
- env object must be non-null
- env object must be in a shared state with the stack
The current caller is believed to satisfy the condition.
A reference to an invalid address might occur in `is_dead()` of `obj_free()` called from `incremental_sweep_phase()`.
This would happen if the heap page was freed ahead of time in the same `incremental_sweep_phase()`.
fixed#6326
If comparing function (block or `<=>`) modifies the sorting array and GC
happens after the modification, objects passed to comparison may be
freed by GC.
The build configuration for `mruby` assumes only the `ncurses` library
needs to be linked because `tinfo` is implicitly pulled in.
In environments where ncurses is available only as a static library,
`tinfo` needs to be linked explicitly (needed for functions like
`tputs`.
This patch fixes that by linking `tinfo` if available.
It also fixes the build for environments where only the `ncursesw`
version of the library (including wide character support) is present,
while still giving preference to the `ncurses` version (without wide
character support).
```console
% find -s lib -type f -name '*.rb' -exec ruby -cw {} \;
lib/mruby/build/command.rb:320: warning: `+' after local variable or literal is interpreted as binary operator
lib/mruby/build/command.rb:320: warning: even though it seems like unary operator
Syntax OK
Syntax OK
Syntax OK
Syntax OK
Syntax OK
lib/mruby/gem.rb:469: warning: `&' interpreted as argument prefix
Syntax OK
Syntax OK
Syntax OK
Syntax OK
```
- Can refer directly to `proc->e.env` after `MRB_PROC_ENV_P()`.
- Can omit `MRB_ENV_ONSTACK_P()` since `mrb->c` is never NULL and can be directly compared to `env->cxt`.
- Can avoid `goto` by putting the code block that raises the `LocalJumpError` at the end.
It used to check all `start`, `end` and `step`. If either of them are
float number, `#step` iterated over float number. Now we don't check the
type of `end` argument.
The latter is a check for integers that have been set up (and disclosed to the
outer world), while the former is a check for integers that are being worked on.
Since uzero() is a predicate, and zero() is a function to assign zero to
mpz_t, it's confusion. Rename predicate uzero() to uzero_p() to follow
mruby naming convention.
Negative integers are virtually considered as 2's compliment of the
absolute value of the corresponding number. It means `-1` is considered
as infinite sequence of `1` toward msb side. ref #6314
Clearing errors at the beginning of `mrb_vm_exec()` essentially keeps the mruby VM in a non-error state.
For consistency, functions such as `mrb_funcall()` check for errors when control returns from a C function as a method.
In the case of a tail call, it should return to `mrb_vm_exec()` afterwards, so error checking is performed there.
Instructions issued while `mrb->exc` is non-null should be limited to `OP_EXCEPT`, the jump target of the catch handler table.
`string[]=(idx, replace)` should return `replace`.
## Actual (wrong)
```
string.[]=(idx, replace) → string
string.[]=(idx, len, replace) → string
```
## Expected
```
string.[]=(idx, replace) → replace
string.[]=(idx, len, replace) → replace
```
## Sidenote
As of the current mruby-compiler, `(string[idx] = 'X')` creates not only "CALL_NODE" but also "ASGN_NODE" and "OP_MOVE", overriding the wrong return value.
On the other hand, `string.[]=(idx, 'X')` creates only "CALL_NODE", exposing the wrong return value.
If my new mruby-compiler2, leveraging Prism, took the place of official compiler, `(string[idx] = 'X')` and `string.[]=(idx, 'X')` would be going to generate the same VM code without "OP_MOVE".
So I paranoidly added tests.
FYI: You can find how the new mruby-compiler2's AST and VM code look like in mruby/c's issue (mruby/c had the same bug): https://github.com/mrubyc/mrubyc/pull/210
This reverts commit ad2e626e7a.
Because of the changes made by #6282, the following code caused a problem.
```ruby
b = proc { break "BAD!" }
p self.tap { b.call }
# (expected) => break from proc-closure (LocalJumpError)
# (after #6282) => "BAD!"
```
I revived the `mrb_callinfo::blk` field to fix this, but it did not overcome the following problem.
```ruby
def m(&b); b = b.clone; GC.start; b.call; end
p m { break "OK!" }
# (expected) => "OK!"
# (revived blk) => break from proc-closure (LocalJumpError)
```
By adding a fast-path where we ignore boxed types we can gain a pretty substantial speedup of mrb_iv_get, making it about 25% faster during a standard optcarrot benchmark run.
NOTE: It is just mrb_iv_get that is that much faster, the whole benchmark seems to be about 3-5% faster with word boxing.
This allows the compiler to optimise the case in obj_iv_p into a range check. There does not seem to be any other very hot uses of this index and it grants a pretty big gain on optcarrot.
- use heap sort (O(1)) instead of merge sort (O(n)) for better space
complexity.
- method implemented in C for better performance
As a result, simple sorting now consumes far less memory and is faster.
Since it's implemented in C, fiber context switching is not allowed from
comparison, but we consider the risk is minimal (no one switches context
in the comparison, right?)
- Added the index number corresponding to the instruction code.
- Omitted trailing `|` from table elements.
The table elements in GitHub Flavored Markdown can't wrap wherever wanted.
And trying to align the end of it tends to make the whole thing longer.
This change itself does nothing good, but it is a preparation for the
future Bison to Lrama migration. As of 0.6.9, Lrama has a compatibility
issue for grammar files without `@n`.
- There was some unnecessary complexity in `OP_BREAK` introduced in commit ad2e626 (#6282).
- Since `mrb->c` is never NULL, there is no need to check it with `MRB_ENV_ONSTACK_P()` beforehand.
Supplement to commit 177debacc5 (#6276).
If the ci is incomplete, the previous method may cause the application to crash because `env->stack` points to an invalid address when expanding the data stack.
Since the ci is in an abnormal state, control it by putting `NULL` in `env->stack`.
If the ci is fine and top-level, detach `env` as usual with `mrb_env_unshare()`.
The part removed in this patch was introduced by commit c7c9543bed.
The current mechanism should be able to trace from block objects created by `eval` to higher level blocks without any problems.
There are two issues to be fixed:
- `mrb_irep` could leak if `mrb_calloc()` encountered an out-of-memory exception
- `mrb_proc_merge_lvar()` allocated one extra variable name.
`irep->lv` can always refer to only one less range than `irep->nlocals`.
Also, when `mrb_proc_merge_lvar()` extends `irep->lv`, `mrb_realloc()` with `NULL` has the same behavior as `mrb_malloc()`.
Set `env->cxt` to `NULL` when it is detached from the call frame.
In other words, we can determine if `env->cxt` is `NULL` or not.
Also, `mruby-binding` had been setting `env->cxt` unnecessarily, so this has been fixed.
We need to include stdlib.h and malloc.h to use malloc()/free() but
they aren't included in src/string.c with WIN32_LEAN_AND_MEAN. It
generates build time warnings.
We can solve this by including stdlib.h and malloc.h explicitly.
In the following example, the fiber context and call stack may be in an incomplete state.
- In case another thread running mruby is terminated abnormally
- In case of a global jump that is out of management by mruby
In `Binding#eval`, two times parsing is executed.
The problem found in this case was caused by not passing an upper block to hold variables of the binding object during the first parsing.
The problem was uncovered by <https://github.com/mruby/mruby/discussions/6274>.
The equal (`==`) method of the comparison target might be redefined
(the root cause of #6262), and not supposed to be compared with NONE.
To reduce chance for the problem, we use `NONE.equal?()` for comparison.
Only go to exception handling if `mrb->exc` is non-null.
This may cause some compatibility problems, but I doubt that it is necessary to maintain that compatibility.
Here is how I see the incompatibility with the change at this time:
- If `mrb->exc` is non-null and `mrb_vm_exec()` is called, an exception will be thrown immediately.
- If `MRB_THROW()` is used while `mrb->exc` is `NULL`, it will not go to exception handling.
Previous SWAR version assumes valid UTF-8 to count number of code points
in the string, but we need to handle invalid sequence as well. We now
use `search_nonascii` to skip counting single byte characters for
performance. The new version is even faster than SWAR version (probably
because `search_nonascii` uses SSE2 on Intel compatible CPU (which I use).
The patch assumes that `struct REnv::cxt` only performs checks with the `OP_BREAK` and `OP_RETURN_BLK` instructions, and does not reference the entity.
Therefore, by changing to a weak reference, it is possible to collect fibers that are no longer directly referenced while in the suspended state.
However, we need to detach the living env objects that remain in the call stack of the fiber.
So, in effect, it involves a revert of following commits.
- commit a3365d8b3f
- commit 57ffa1c150
Examples of the effects of change are shown below.
Note that it was built with `rake MRUBY_CONFIG=host-debug`.
```ruby
f = Fiber.new { (x, y, z) = "X", "Y", "Z"; Fiber.yield -> { [x, y, z] } }
g = f.resume
GC.start
p ObjectSpace.memsize_of_all
# => 59532
g.call
# => ["X", "Y", "Z"]
f = nil
GC.start
ObjectSpace.memsize_of_all
# BEFORE => 59532
# AFTER => 58044
g.call
# => ["X", "Y", "Z"]
```
`mrb_env_unshare()` calls `mrb_realloc_simple()` and follows `mrb_full_gc()` to avoid an infinite loop where `mrb_env_unshare()` is called again.
This does not occur at this time, but may occur in subsequent patches.
`mrb_vm_run()` is,
- It does not change the fiber context.
- When control is returned, only one ci prepared by the caller is popped.
If the ci equals cibase when called, the ci position does not change.
related commits:
- commit 4e84bdb507
- commit 34dd258c63
- commit ebd6636a1e
- commit c6736357a7
- commit 23a4e7149d
- commit 31a961acf1
Previously `\x80` was incorrectly mapped to `0`.
```ruby
"\x80\x80\x80\x80".unpack("m*")
# before => "\x00\x00\x00"
# after => ""
```
The reason is that the C string terminator is placed in `base64_dec_tab[128]` and the array length is obtained by `sizeof`.
Therefore, the length of `base64_dec_tab[]` is strictly specified and replaced with element-by-element initialization.
Also, similar changes are made to `base64chars[]`.
- Don't create multiple envs on one ci.
- Don't share a env to different ci.
- Don't attach a closed env to any ci.
Changes in `envadjust()` can be simplified with those guarantees.
the worst case for `Array#reject!` (i.e. a proc always returning `true`)
is at least 5x worse than the worst case for `Array#select!` (proc
always returning `false`)
this commit unifies these implementations and inlines the (effective)
call of `#select!` in `#keep_if` and `#reject!` in `#delete_if`
Immediately frees the call stack and data stack at the end of a non-root fiber.
If the env object needs to be detached, the data stack is reused through `mrb_realloc()`.
Previously, it was not necessary to take into account that `c->cibase` could be `NULL`.
Note that this is no longer the case due to this patch.
In fact, changes to "mruby-fiber" are now required.
The state of a fiber switched due to an exception occurrence was incorrectly set to "Suspended".
```ruby
Fiber.new {
begin
Fiber.new { 0 / 0 }.resume
rescue
p Fiber.current
# before => #<Fiber:0x159f61e43ce0 fiber.rb:1 (suspended by resuming)>
# after => #<Fiber:0x159f61e43ce0 fiber.rb:1 (resumed)>
end
}.resume
```
This reverts commit 26e436e247.
After investigation, it is possible to revert by commit e89cc9b9fa.
The build configuration file used in the investigation is shown below.
```ruby
MRuby::Build.new do |conf|
toolchain :clang
enable_debug
enable_bintest
enable_test
cc.command = "clang18"
linker.command = "clang18"
[cc, cxx].each { |c| c.defines << "MRB_GC_STRESS" }
[cc, cxx, linker].each { |cmd| cmd.flags << %w(-fsanitize=address) }
gem github: "iij/mruby-dir" # rev: "89dceefa1250fb1ae868d4cb52498e9e24293cd1"
gem github: "iij/mruby-env" # rev: "056ae324451ef16a50c7887e117f0ea30921b71b"
gem github: "iij/mruby-errno" # rev: "b4415207ff6ea62360619c89a1cff83259dc4db0"
gem github: "iij/mruby-require" # rev: "f0634d785e5cbb73cd7d118ee36deff499e4181e"
gem github: "iij/mruby-tempfile" # rev: "9b883438547020dae328e34c8a2fe736171cd0ab"
end
```
Since the `rake` command needs to be from the past, we used the Ruby 2.6 version.
Currently `e->cxt` is used exclusively to check for `break` / `return` availability.
In other words, there is no need to maintain a reference to a fiber that has reached its end.
Highlights are:
- `Integrate the blocks `if (!ci->proc || MRB_PROC_CFUNC_P(ci->proc))` and `if (loc.irep == NULL)`.
- Folding some other conditionals.
- Assertions ensure that procs are not aliases.
The purpose is to remove the `mid` field from the `mrb_cache_entry` structure.
The resulting RAM requirement for the method cache is reduced from 5 words per entry to 4 words per entry for 32-bit CPUs.
The relevant changes are as follows:
- Removed `MRB_USE_METHOD_T_STRUCT`.
The `mrb_method_t` type is now always defined as a structure.
- Include method IDs in `mrb_method_t`
Change the `flags` member to `uint32_t`.
The bitstring structure should be the same as the keys of the `mt` table in `class.c`.
I believe the impact on API compatibility with previous versions is minimal.
This will be a partial merge of #5317 with the following changes.
- Remove `iclass->iv_c` since `iclass->iv_c` is equivalent to `iclass->c`.
- `class_iv_ptr()` returns a single pointer instead of a double pointer.
It used to check ci to be non NULL in line 37, but we silently assumed
ci was not NULL in the `else` clause too. So instead of checking NULL,
we add assertion. This incomplete check was found by clang-tidy.
The old code assumes unary minus (`-@`) does not cause any side effect
(including errors). Considering the code like `-nil; nil`, the
assumption was too aggressive.
mruby used to use float numbers for overflown integers before we
implemented big integers. Now we don't need bit operations for float
numbers anymore. Also removed tests for shift operations for float
numbers.
Assertions were failing on exit if started with `mrb_fiber_resume()`.
The bug that caused it was introduced by commit 42308c42b5 (#6106).
The bug was moved by commit 990e18ad59.
Previously, `mrb->exc` would remain replaced by a `break` object if a rewind operation was performed during the processing of a `OP_STOP` instruction.
This problem has existed since #5060, when it was introduced in mruby-3.0.
However, as of mruby-3.0, a manual or third-party generator is required to cause the `OP_STOP` instruction to be issued.
Therefore, it is believed that this has not had an impact until now.
Previously, the correct directory could not be obtained in some environments.
Therefore, changed the method to leave an element indicating the parent directory.
fix#6156
When irep->refcnt reaches UINT16_MAX, mrb_irep_incref() raises
exception but the function pack_backtrace_i() is called from
mrb_exc_raise() thus causes the infinite loop problem. So this is a
hack-ish workaround by making irep reference to NULL if refcnt reaches
the maximum count. Probably we will address this issue again to make it
better.
For reference, the flexible array was introduced by commit 3ab2f9371e (#2997).
Subsequently changed for compatibility with C++ by commit 24939723d7 (#5596).
The `MRB_TT_BACKTRACE` object has been added for the purpose.
Previously, "use-after-free" could occur because the reference count in `backtrace_location::irep` was not incremented.
fixed#6160
If you define `SIMPLE_SEARCH_NONASCII`, you can use old, naive
implementation of search_nonascii(). You may want to use the old one for
code size constraint for example.
Use stack variable addresses as identifiers instead of global variable values.
Since the stack variable address is uniquely determined within the call, there is no need to maintain a global variable.
This flag means all the characters in the string can be represented by a
single byte, i.e., the string does not contain any multi-byte character.
Those characters are likely ASCII characters, but may be a part of broken
UTF-8 sequence, so the term 'ASCII' is not sufficient.
Instead of its own version of quick search, now we use str_index_str()
and adjust character position. This change makes searching 4 times
faster in some cases; ref #6143
Current code scan the string twice (once from RSTRING_CHAR_LEN, and once
from chars2bytes), but those scans are not necessary. Just point the end
of the string.
When `mrb->c->prev` is non `NULL` and `mrb->c->noexec` is false, switching source fiber should suspend with `Fiber#resume`.
In this case, the condition `mrb->c->ci == mrb->c->cibase` is not satisfied.
## Implementation Summary
* Only keys and only values of hash table are contiguous to eliminate
structure padding.
* Change upper limit of `iv_tbl` size to `UINT16_MAX` (it seems to be
acceptable in mruby because the total number of classes/modules
immediately after starting Redmine is 20,000 or less).
* `iv_tbl*` point hash buckets directly.
## Benchmark Summary
Only the results of typical situations on 64-bit Word-boxing are present
here. For more detailed information, including consideration, see below
report (although most of the body is written in Japanese).
* https://shuujii.github.io/mruby-iv-benchmark
### Memory Usage
Lower value is better.
| iv_tbl Size | Baseline | New | Factor |
|------------:|---------------:|---------------:|-----------:|
| 4 | 88B | 52B | 0.59091x |
| 30 | 536B | 388B | 0.72388x |
| 100 | 2072B | 1540B | 0.74324x |
| 200 | 4120B | 3076B | 0.74660x |
Although not mentioned in the above report, the memory usage of `mrbtest`
(full-core gembox) is as follows in the result by Valgrind.
* Baseline: 108,086 allocs, 16,313,122 bytes allocated
* New: 94,273 allocs, 15,875,214 bytes allocated
### Performance
Higher value is better.
#### `mrb_obj_iv_set`
| iv_tbl Size | Baseline | New | Factor |
|------------:|---------------:|---------------:|-----------:|
| 4 | 88.63003M i/s | 92.60611M i/s | 1.04486x |
| 30 | 32.97066M i/s | 25.25095M i/s | 0.76586x |
| 100 | 16.33224M i/s | 22.74998M i/s | 1.39295x |
| 200 | 5.64484M i/s | 6.79949M i/s | 1.20455x |
#### `mrb_obj_iv_get`
| iv_tbl Size | Baseline | New | Factor |
|------------:|---------------:|---------------:|-----------:|
| 4 | 217.58391M i/s | 237.59912M i/s | 1.09199x |
| 30 | 139.56195M i/s | 160.49470M i/s | 1.14999x |
| 100 | 143.09716M i/s | 190.95047M i/s | 1.33441x |
| 200 | 89.75291M i/s | 134.78717M i/s | 1.50176x |
### Binary Size
Lower value is better.
| File | Baseline | New | Factor |
|:------------|---------------:|---------------:|-----------:|
| mruby | 697,520B | 697,520B | 1.00000x |
| libmruby.a | 1,046,570B | 1,046,682B | 0.99989x |
## Note
The address in `struct RObject::iv` may change after initialization because
`iv_tbl*` points directly to hash buckets. Therefore, the address cannot be
copied and shared when include/prepend. So, when sharing `iv_tbl`, refer to
it via the sharing source class. As a result, the following bug have also
been fixed.
* [An `iv_tbl` is not shared when a class includes or prepends an empty module](https://gist.github.com/shuujii/0ac23fa24b0c55b2c602b534d81e4a95)
2021-02-03 23:16:09 +09:00
420 changed files with 39538 additions and 13205 deletions
# User visible changes in `mruby3.4` from `mruby3.3`
"**_NOTE_**:" are changes to be aware of.
# The language
- Now `a::B = c` should evaluate `a` then `c`.
- Anonymous arguments `*`, `**`, `&` can be passed for forwarding.
- Multi-precision integer is available now via `mruby-bigint` gem.
- mruby now supports `private` and `protected` visibility ([b0db0bd](https://github.com/mruby/mruby/commit/b0db0bd))
- Maximum length of inlined symbols reduced from 5 to 4 characters to provide space for visibility flags ([6442a01](https://github.com/mruby/mruby/commit/6442a01))
- Many methods are made private according to CRuby visibility ([4a0e806](https://github.com/mruby/mruby/commit/4a0e806))
- Generate OP_SSEND for `self.method` type calls ([111fe4b](https://github.com/mruby/mruby/commit/111fe4b))
- `initialize` method will be always private ([eb8b412](https://github.com/mruby/mruby/commit/eb8b412))
- Add new hooks `method_removed`, `method_undefined` ([9c74f6e](https://github.com/mruby/mruby/commit/9c74f6e))
- Add new hooks `singleton_method_removed`, `singleton_method_undefined` ([0863c08](https://github.com/mruby/mruby/commit/0863c08))
- Updated `OP_DEF` output from codedump ([3a3e877](https://github.com/mruby/mruby/commit/3a3e877))
- Better handling of binary strings, e.g. String#b ([b0127f0](https://github.com/mruby/mruby/commit/b0127f0))
- Hash `to_s` format has changed ([baeeb5e](https://github.com/mruby/mruby/commit/baeeb5e))
- Some encoding related method such as `#force_encoding` ([e47b4ca](https://github.com/mruby/mruby/commit/e47b4ca)), `#b` ([b0127f0](https://github.com/mruby/mruby/commit/b0127f0))
- Constant folding for `String#+` ([6687bdd](https://github.com/mruby/mruby/commit/6687bdd))
- use SWAR technique for strlen performance ([cbb31e6](https://github.com/mruby/mruby/commit/cbb31e6))
- use merge sort for `Array#sort` ([5bd63d6](https://github.com/mruby/mruby/commit/5bd63d6))
# mruby VM and bytecode
# Changes in C API
- `OP_ARYDUP` was renamed to `OP_ARYSPLAT`. The instruction name
was changed but instruction number and basic behavior have not
changed (except that `ARYDUP nil` makes `[]`).
- pool.c renamed to mempool.c (and mrb_pool to mrb_mempool) ([49525fa](https://github.com/mruby/mruby/commit/49525fa))
- mrb_pool_value renamed to mrb_irep_pool to reduce confusion ([62ef5db](https://github.com/mruby/mruby/commit/62ef5db))
- rename BOXNIX_SET_VALUE to BOXNO_SET_VALUE ([#6397](https://github.com/mruby/mruby/pull/6397))
- `MRB_FROZEN_P()` is replaced by `mrb_frozen_p()` ([c11d18e](https://github.com/mruby/mruby/commit/c11d18e))
- rename `color` to `gc_color` ([0e79f6b](https://github.com/mruby/mruby/commit/0e79f6b), [1e36d76](https://github.com/mruby/mruby/commit/1e36d76))
- add `obj->frozen` instead of flags `MRB_SET_FROZEN_FLAG`/`MRB_UNSET_FROZEN_FLAG` ([8276143](https://github.com/mruby/mruby/commit/8276143))
# Tools
# Build & Configuration
## `mruby`
- New Build Target: `test:run:serial`, `test:run:serial:lib`, `test:run:serial:bin` ([#6423](https://github.com/mruby/mruby/pull/6423))
- New Platform: PlayStation Portable ([#6022](https://github.com/mruby/mruby/pull/6465))
- New Platform: emscripten ([#6487](https://github.com/mruby/mruby/pull/6487))
- New Config: no-float (with MRB_NO_FLOAT) ([32200f1](https://github.com/mruby/mruby/commit/32200f1))
- `-b` only specifies the script is the binary. The files loaded by `-r` are not affected by the option.
- `mruby` now loads complied binary if the suffix is `.mrb`.
# Changes in mrbgems
## `mrbc`
- **mruby-print**: removed; if you do not use `mruby-io`, mruby use `#print` etc. in the core ([8c8bbd9](https://github.com/mruby/mruby/commit/8c8bbd9))
- **mruby-enum-lazy**: Add Enumerable::Lazy#grep_v to mruby-enum-lazy ([#6171](https://github.com/mruby/mruby/pull/6171))
- **mruby-io**: Add `level` argument to `File.dirname` ([#6463](https://github.com/mruby/mruby/pull/6463))
- [#6183](https://github.com/mruby/mruby/issues/6183) ".e".to_f returns NAN
- [#6182](https://github.com/mruby/mruby/issues/6182) mrb_read_float() converts "0.3" with a small error compared to strtod()
- [#6210](https://github.com/mruby/mruby/issues/6210) Unary minus seems broken
- [#6255](https://github.com/mruby/mruby/issues/6255) Wrong number of characters in broken UTF-8 string
- [#4038](https://github.com/mruby/mruby/issues/4038) Heap buffer overflow in OP_ENTER
- [#6262](https://github.com/mruby/mruby/issues/6262) Unable to define == for objects when using Array#delete
- [#6267](https://github.com/mruby/mruby/issues/6267) When MRB_UTF8_STRING is enabled, giving byte characters for String#index and String#split gives wrong results
- [#6277](https://github.com/mruby/mruby/issues/6277) MSVC: can't use malloc() in string.c with WIN32_LEAN_AND_MEAN
- [#6240](https://github.com/mruby/mruby/issues/6240) Differentiate between lib and lib64 in the build settings.
- [#6304](https://github.com/mruby/mruby/issues/6304) Calling method_missing with only Kwargs passes arguments incorrectly
- [#6317](https://github.com/mruby/mruby/issues/6317) mrb_gc_register() may cause GC and collect the object being protected
- [#6307](https://github.com/mruby/mruby/issues/6307) Planned change patch for mrb_vm_exec()
- [#6298](https://github.com/mruby/mruby/issues/6298) foo :bar {} is legal in mruby?
- [#6326](https://github.com/mruby/mruby/issues/6326) Detect “Use-after-free” with address sanitizer
- [#5358](https://github.com/mruby/mruby/issues/5358) static warning from getpwnam
- [#6339](https://github.com/mruby/mruby/issues/6339) mrb_ary_delete() may refer to an invalid address (use-after-free)
- [#6346](https://github.com/mruby/mruby/issues/6346) Block kwargs are passed as last positional arg when using yield
- [#6365](https://github.com/mruby/mruby/issues/6365) powl() not available when compiling for Dreamcast
- [#6369](https://github.com/mruby/mruby/issues/6369) 100x Performance Regression from 3.1
- [#6270](https://github.com/mruby/mruby/issues/6270) NODE_ZSUPER from deeply nested blocks will result in a truncated digits for block index in OP_ARGARY
- [#6297](https://github.com/mruby/mruby/issues/6297) Assigning to a block variable changes the actual block (thus affecting block_given? and yield)
- [#6389](https://github.com/mruby/mruby/issues/6389) instance_exec named block args don't work properly
- [#6388](https://github.com/mruby/mruby/issues/6388) Recent commit broke my windows build
- [#6411](https://github.com/mruby/mruby/issues/6411) Wrong function unwinding when using return in a block
- [#6439](https://github.com/mruby/mruby/issues/6439) OP_JMPUW does not call the ensure block when it jumps to the beginning of the begin block
- [#6441](https://github.com/mruby/mruby/issues/6441) break inside while loop will execute the ensure block outside of the while loop
- [#6453](https://github.com/mruby/mruby/issues/6453) Bigint: incorrect behavior of ^ operator
- [#6452](https://github.com/mruby/mruby/issues/6452) Bigint: weird mod behavior
@@ -30,12 +30,10 @@ of the Ministry of Economy, Trade and Industry of Japan.
## How to get mruby
To get mruby, you can download the stable version 3.2.0 from the official mruby
To get mruby, you can download the stable version 3.4.0 from the official mruby
GitHub repository or clone the trunk of the mruby source tree with the "git
clone" command. You can also install and compile mruby using [ruby-install](https://github.com/postmodern/ruby-install), [ruby-build](https://github.com/rbenv/ruby-build) or [rvm](https://github.com/rvm/rvm).
The release candidate version 3.3.0 of mruby can be downloaded via the following URL: [https://github.com/mruby/mruby/archive/3.3.0-rc.zip](https://github.com/mruby/mruby/archive/3.3.0-rc.zip)
The latest development version of mruby can be downloaded via the following URL: [https://github.com/mruby/mruby/zipball/master](https://github.com/mruby/mruby/zipball/master)
The trunk of the mruby source tree can be checked out with the
@@ -87,6 +85,30 @@ extensions in C and/or Ruby. For a guide on how to use mrbgems, consult the
[mrbgems.md](doc/guides/mrbgems.md) file, and for example code, refer to the
[examples/mrbgems/](examples/mrbgems) folder.
## Index of Document
<!--
This section is generated by `rake doc:update-index`.
All manual changes will get lost.
-->
<!-- BEGIN OF MRUBY DOCUMENT INDEX -->
- [About the Limitations of mruby](doc/limitations.md)
- [About the Compile](doc/guides/compile.md)
- [About the Debugger with the `mrdb` Command](doc/guides/debugger.md)
- [About GC Arena](doc/guides/gc-arena-howto.md)
- [About the mruby directory structure](doc/guides/hier.md)
- [About Linking with `libmruby`](doc/guides/link.md)
<!-- summary: About Memory Allocator Customization -->
# Memory Allocation
There are three methods to customize memory allocation in mruby.
In mruby, you can customize how memory is allocated in two ways:
1. Provide your own `realloc()`/`free()`
2.Redefine `mrb_default_allocf()`
3. Specify a function with `mrb_open_allocf()`
1.**Provide your own `malloc()`/`realloc()`/`free()`**
2.**Override `mrb_basic_alloc_func()`**
## Provide your own `realloc()`/`free()`
---
On some platforms, especially on microcontrollers, the standard library may not provide `malloc()`, `realloc()`, and `free()`. In such cases, it may be necessary to define memory allocation functions for the specific platform. mruby uses `realloc()` and `free()` from the standard C library for memory management. By defining these two functions of your own, you can make mruby work. However, note the following two points:
## 1. Provide your own `malloc()`/`realloc()`/`free()`
First, `realloc(NULL, size)` behaves the same as malloc(size). Second,`free(NULL)` exits without doing anything.
On platforms without a full C standard library —such as many microcontrollers— you may need to supply your own implementations of `malloc()`, `realloc()`, and `free()`. mruby’s allocator calls directly into these functions, so replacing them lets you control **every** allocation and deallocation performed by your entire program, including anythird‑party libraries you link against.
## Redefine `mrb_default_allocf()`
Keep in mind:
The only function in mruby that uses the standard C library's memory allocation functions is `mrb_default_allocf()`, defined in `alloc.c`. By defining this function within your application, you can customize the memory management of your application.
- Calling `realloc(NULL, size)` must behave like `malloc(size)`.
- Calling `free(NULL)` must be a no‑op.
## Specify a function with `mrb_open_allocf()`
Simply define these three functions in your code (or link against a library that provides them), and mruby — along with all other code in your process — will use your versions automatically.
If you want to perform different memory management for each `mrb_state` within your application, you can use the `mrb_open_allocf()` function to create the `mrb_state` structure. This allows you to specify a memory allocation function (which is compatible with `mrb_default_allocf`) for each `mrb_state`. Although this scheme is not recommended. It may become obsolete in the future, since I have never seen per mrb_state memory management use-case.
## 2. Override `mrb_basic_alloc_func()`
Inside mruby, all of its own memory allocations go through a single function called mrb_basic_alloc_func() (formerly mrb_default_allocf()). By defining this function in your application before linking, you can intercept and handle **only** the memory operations initiated by mruby itself without affecting other libraries or parts of your program.
@@ -29,12 +31,12 @@ If you need full precision for floating-point numbers, define `MRB_WORDBOX_NO_FL
## NaN Boxing
NaN boxing packs the Ruby data in a floating-point numbers, which represent NaN (Not a Number) values. Under IEEE753 definitions every value that exponent is all set are considered as NaN. That means NaN can represent `2^51` values. NaN boxing is a teaching to pack the values in those NaN representation. In theory, 64bit pointers are too big to fit in NaN, but practically most OS use only 48 bits at most for pointers (except for some OS e.g. Solaris).
NaN boxing packs the Ruby data in a floating-point numbers, which represent NaN (Not a Number) values. Under IEEE753 definitions every value that exponent is all set are considered as NaN. That means NaN can represent `2^51` values. NaN boxing is a teaching to pack the values in those NaN representation. In theory, 64-bit pointers are too big to fit in NaN, but practically most OS use only 48 bits at most for pointers (except for some OS e.g. Solaris).
The NaN boxing packing bit patterns are like following:
@@ -62,7 +62,7 @@ Some configuration macro names are changed for consistency (use `MRB_USE_XXX`
-`MRB_USE_FLOAT32` is changed from `MRB_USE_FLOAT` to make sure `float` here
means using single-precision float, and not the opposite of `MRB_NO_FLOAT`.
-`MRB_USE_METHOD_T_STRUCT` uses `struct` version of `mrb_method_t`. More
portable but consumes more memory. Turned on by default on 32bit platforms.
portable but consumes more memory. Turned on by default on 32-bit platforms.
-`MRB_` prefix is added to those without.
## `MRB_NO_BOXING`
@@ -149,7 +149,8 @@ No more operand extension
## Changed Instructions
Jump addresses used to be specified by absolute offset from the start of `iseq`. Now they are relative offset from the address of the next instruction.
Jump addresses used to be specified by absolute offset from the start of `iseq`. Now they are relative offset from the
**_NOTE_**: Due to this impact, executable files in the `mruby/bin/` directory by default are now symbolic links (batch files on Windows).
If previously relied on those executables, should be replaced with direct references to the entity created under the build directory (e.g. `mruby/build/host/bin/`).
- Encode and decode escape characters for presym ([#6011](https://github.com/mruby/mruby/pull/6011))
# User visible changes in `mruby3.4` from `mruby3.3`
"**_NOTE_**:" are changes to be aware of.
# The language
- mruby now supports `private` and `protected` visibility ([b0db0bd](https://github.com/mruby/mruby/commit/b0db0bd))
- Maximum length of inlined symbols reduced from 5 to 4 characters to provide space for visibility flags ([6442a01](https://github.com/mruby/mruby/commit/6442a01))
- Many methods are made private according to CRuby visibility ([4a0e806](https://github.com/mruby/mruby/commit/4a0e806))
- Generate OP_SSEND for `self.method` type calls ([111fe4b](https://github.com/mruby/mruby/commit/111fe4b))
-`initialize` method will be always private ([eb8b412](https://github.com/mruby/mruby/commit/eb8b412))
- Add new hooks `method_removed`, `method_undefined` ([9c74f6e](https://github.com/mruby/mruby/commit/9c74f6e))
- Add new hooks `singleton_method_removed`, `singleton_method_undefined` ([0863c08](https://github.com/mruby/mruby/commit/0863c08))
- Updated `OP_DEF` output from codedump ([3a3e877](https://github.com/mruby/mruby/commit/3a3e877))
- Better handling of binary strings, e.g. String#b ([b0127f0](https://github.com/mruby/mruby/commit/b0127f0))
- Hash `to_s` format has changed ([baeeb5e](https://github.com/mruby/mruby/commit/baeeb5e))
- Some encoding related method such as `#force_encoding` ([e47b4ca](https://github.com/mruby/mruby/commit/e47b4ca)), `#b` ([b0127f0](https://github.com/mruby/mruby/commit/b0127f0))
- Constant folding for `String#+` ([6687bdd](https://github.com/mruby/mruby/commit/6687bdd))
- use SWAR technique for strlen performance ([cbb31e6](https://github.com/mruby/mruby/commit/cbb31e6))
- use merge sort for `Array#sort` ([5bd63d6](https://github.com/mruby/mruby/commit/5bd63d6))
# Changes in C API
- pool.c renamed to mempool.c (and mrb_pool to mrb_mempool) ([49525fa](https://github.com/mruby/mruby/commit/49525fa))
- mrb_pool_value renamed to mrb_irep_pool to reduce confusion ([62ef5db](https://github.com/mruby/mruby/commit/62ef5db))
- rename BOXNIX_SET_VALUE to BOXNO_SET_VALUE ([#6397](https://github.com/mruby/mruby/pull/6397))
-`MRB_FROZEN_P()` is replaced by `mrb_frozen_p()` ([c11d18e](https://github.com/mruby/mruby/commit/c11d18e))
- rename `color` to `gc_color` ([0e79f6b](https://github.com/mruby/mruby/commit/0e79f6b), [1e36d76](https://github.com/mruby/mruby/commit/1e36d76))
- add `obj->frozen` instead of flags `MRB_SET_FROZEN_FLAG`/`MRB_UNSET_FROZEN_FLAG` ([8276143](https://github.com/mruby/mruby/commit/8276143))
# Build & Configuration
- New Build Target: `test:run:serial`, `test:run:serial:lib`, `test:run:serial:bin` ([#6423](https://github.com/mruby/mruby/pull/6423))
- New Platform: PlayStation Portable ([#6022](https://github.com/mruby/mruby/pull/6465))
- New Platform: emscripten ([#6487](https://github.com/mruby/mruby/pull/6487))
- New Config: no-float (with MRB_NO_FLOAT) ([32200f1](https://github.com/mruby/mruby/commit/32200f1))
# Changes in mrbgems
- **mruby-print**: removed; if you do not use `mruby-io`, mruby use `#print` etc. in the core ([8c8bbd9](https://github.com/mruby/mruby/commit/8c8bbd9))
- **mruby-enum-lazy**: Add Enumerable::Lazy#grep_v to mruby-enum-lazy ([#6171](https://github.com/mruby/mruby/pull/6171))
- **mruby-io**: Add `level` argument to `File.dirname` ([#6463](https://github.com/mruby/mruby/pull/6463))
- [#6183](https://github.com/mruby/mruby/issues/6183) ".e".to_f returns NAN
- [#6182](https://github.com/mruby/mruby/issues/6182) mrb_read_float() converts "0.3" with a small error compared to strtod()
- [#6210](https://github.com/mruby/mruby/issues/6210) Unary minus seems broken
- [#6255](https://github.com/mruby/mruby/issues/6255) Wrong number of characters in broken UTF-8 string
- [#4038](https://github.com/mruby/mruby/issues/4038) Heap buffer overflow in OP_ENTER
- [#6262](https://github.com/mruby/mruby/issues/6262) Unable to define == for objects when using Array#delete
- [#6267](https://github.com/mruby/mruby/issues/6267) When MRB_UTF8_STRING is enabled, giving byte characters for String#index and String#split gives wrong results
- [#6277](https://github.com/mruby/mruby/issues/6277) MSVC: can't use malloc() in string.c with WIN32_LEAN_AND_MEAN
- [#6240](https://github.com/mruby/mruby/issues/6240) Differentiate between lib and lib64 in the build settings.
- [#6304](https://github.com/mruby/mruby/issues/6304) Calling method_missing with only Kwargs passes arguments incorrectly
- [#6317](https://github.com/mruby/mruby/issues/6317) mrb_gc_register() may cause GC and collect the object being protected
- [#6307](https://github.com/mruby/mruby/issues/6307) Planned change patch for mrb_vm_exec()
- [#6298](https://github.com/mruby/mruby/issues/6298) foo :bar {} is legal in mruby?
- [#6326](https://github.com/mruby/mruby/issues/6326) Detect “Use-after-free” with address sanitizer
- [#5358](https://github.com/mruby/mruby/issues/5358) static warning from getpwnam
- [#6339](https://github.com/mruby/mruby/issues/6339) mrb_ary_delete() may refer to an invalid address (use-after-free)
- [#6346](https://github.com/mruby/mruby/issues/6346) Block kwargs are passed as last positional arg when using yield
- [#6365](https://github.com/mruby/mruby/issues/6365) powl() not available when compiling for Dreamcast
- [#6369](https://github.com/mruby/mruby/issues/6369) 100x Performance Regression from 3.1
- [#6270](https://github.com/mruby/mruby/issues/6270) NODE_ZSUPER from deeply nested blocks will result in a truncated digits for block index in OP_ARGARY
- [#6297](https://github.com/mruby/mruby/issues/6297) Assigning to a block variable changes the actual block (thus affecting block_given? and yield)
- [#6389](https://github.com/mruby/mruby/issues/6389) instance_exec named block args don't work properly
- [#6388](https://github.com/mruby/mruby/issues/6388) Recent commit broke my windows build
- [#6411](https://github.com/mruby/mruby/issues/6411) Wrong function unwinding when using return in a block
- [#6439](https://github.com/mruby/mruby/issues/6439) OP_JMPUW does not call the ensure block when it jumps to the beginning of the begin block
- [#6441](https://github.com/mruby/mruby/issues/6441) break inside while loop will execute the ensure block outside of the while loop
- [#6453](https://github.com/mruby/mruby/issues/6453) Bigint: incorrect behavior of ^ operator
- [#6452](https://github.com/mruby/mruby/issues/6452) Bigint: weird mod behavior
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
google-labs-jules[bot]
