It seems to be preferable to be able to handle pointers of type `char` as well.
For this purpose, `mrb_nanbox_tt_inline` has been reorganized.
- `MRB_NANBOX_TT_POINTER` has been split into `MRB_NANBOX_TT_OBJECT` and `MRB_NANBOX_TT_CPTR`
- `MRB_NANBOX_TT_SYMBOL` has been merged into `MRB_NANBOX_TT_MISC`
The main purpose is to increase the chances of finding presym and to prevent errors due to C++11 lambda expressions.
- The argument to receive the class may be written, for example, `mrb_class_get()`.
- The argument that receives the implementation function of the method may be a C++ lambda expression.
In this case, if multiple variable declarations are separated by colons, the preprocessor will recognize them as argument delimiters and report an error.
This patch prevents it from happening.
```c++
// When preprocessing...
func([] { int x, y, z; })
// ^^^^^^^^^^ 1st argument?
// ^ 2nd argument?
// ^^^^ 3rd argument?
```
ref. commit 7f40b645d2
Currently, the build configurations `MRB_USE_COMPLEX` and `MRB_USE_RATIONAL` are not listed in the documentation.
In other words, they are hidden settings.
They are defined in `mrbgems/mruby-{complex,rational}/mrbgem.rake`.
So this patch assumes that it is safe to refer to these functions in core-gems directly from core functions.
However, applications that link with `libmruby_core.a` will have compatibility issues.
In fact, `mrbgems/mruby-bin-mrbc` links with `libmruby_core.a`, so I had to prepare a dummy function.
The main reason for failure is to exceed the time limit, and even when it succeeds, there is less than a minute left.
The 10-minute time limit seems to be too short.
ref. #5613.
I mentioned in #5540 that there was no reentrant to the virtual machine, but in fact it was still a possibility at that point.
Also, the variable `ci` needs to be recalculated at the same time.
ref. #5613
I checked with Valgrind, and the methods that can cause use-after-free are `Array#rotate`, `Array#rotate!`, and `String#byteslice`.
Since `String#rindex` uses `RSTRING_LEN()` indirectly inside the function, no reference to the out-of-bounds range is generated.
Since `mrb_to_integer` and `mrb_to_float` does not convert the object
but checks types, they are named so by historical reason. We introduced
properly named functions.
This commit obsoletes the following functions:
* mrb_to_integer()
* mrb_to_int()
* mrb_to_float()
Use `mrb_ensure_int_type()` instead for the first 2 functions. Use
`mrb_ensure_float_type()` for the last.
Allow void expression on some places e.g. right hand of `rescue`
modifier. In addition, checks added on some places, e.g. left hand of
logical operators.
Previously, it always pointed to the highest scope as the location of the error.
- example code `code.rb`
```ruby
huge_num = "1" + "0" * 300; eval <<CODE, nil, "test.rb", 1
class Object
module A
#{huge_num}
end
end
CODE
```
- Before this patch
```console
% bin/mruby code.rb
test.rb:1: integer too big
trace (most recent call last):
[1] code.rb:1
code.rb:1:in eval: codegen error (ScriptError)
```
- After this patch
```console
% bin/mruby code.rb
test.rb:3: integer too big
trace (most recent call last):
[1] code.rb:1
code.rb:1:in eval: codegen error (ScriptError)
```
Now `iv_get()` returns `pos+1` if it finds the entry, so you don't need
to call `iv_put()`. You can replace the entry value by assigning to
`t->ptr[pos-1]`.
This is a fundamentally simplified reimplementation of #5317
by @shuujii
Instead of having array of `struct iv_elem`, we have sequences of keys
and values packed in single chunk of malloc'ed memory. We don't have to
worry about gaps from alignment, especially on 64 bit architecture,
where `sizeof(struct iv_elem)` probably consumes 16 bytes, but
`sizeof(mrb_sym)+sizeof(mrb_value)` is 12 bytes.
In addition, this change could improve memory access locality.
close#5317
Make "N for M" into the form "given N, expected M".
As I worked, I noticed that the `argnum_error()` function had a part to include the method name in the message.
I think this part is no longer needed by https://github.com/mruby/mruby/pull/5394.
- Before this patch
```console
% bin/mruby -e '[1, 2, 3].each 0'
trace (most recent call last):
[1] -e:1
-e:1:in each: 'each': wrong number of arguments (1 for 0) (ArgumentError)
```
- After this patch
```console
% bin/mruby -e '[1, 2, 3].each 0'
trace (most recent call last):
[1] -e:1
-e:1:in each: wrong number of arguments (given 1, expected 0) (ArgumentError)
```
If the preprocessor check part is only `__clang__`, CI's such as `Ubuntu-2004-clang` will fail to compile.
This is why we limited the addition to FreeBSD and OpenBSD, which have `clang++` in their base systems.
DragonFly BSD and NetBSD have GCC built into their base systems, so nothing is changed.
The `__id__` method implemented in the C function has `MRB_ARGS_NONE()` specified, but it is also effective in the following cases.
```ruby
p nil.__id__ opts: 1 rescue p :a
p nil.method(:__id__).call 1 rescue p :b
p nil.method(:__id__).call opts: 1 rescue p :c
p nil.method(:__id__).to_proc.call 1 rescue p :d
p nil.method(:__id__).to_proc.call opts: 1 rescue p :e
p nil.method(:__id__).unbind.bind_call nil, 1 rescue p :f
p nil.method(:__id__).unbind.bind_call nil, opts: 1 rescue p :g
p nil.__send__ :__id__, 1 rescue p :h
p nil.__send__ :__id__, opts: 1 rescue p :i
```
After applying this patch, all items will output symbols in the same way as CRuby.
For this purpose, add `MRB_PROC_NOARG` to `struct RProc::flags`.
Calling the `Method#{parameters,source_location}` method on a static `Proc` object resulted in `SIGSEGV`.
The trigger is https://github.com/mruby/mruby/pull/5402.
The original implementation of the `Method#{parameters,source_location}` method was to temporarily rewrite the object and then call the method of the same name in `Proc`.
Rewriting of objects placed in the ROM section by #5402 above is prohibited by hardware such as the CPU.
This caused a `SIGSEGV`.
If a splat argument was passed, it could write out of range on the VM stack.
```console
% bin/mruby -e 'def m(*args, **opts, &blk) p [args, opts, blk] end; m(*%w(X Y Z), r: 1, g: 2, b: 3) {}'
[["X", "Y", "Z"], {}, #<Proc:0x80077d7d0>]
```
```console
% c++ -xc++ -std=c++03 -S -Iinclude -DMRB_NAN_BOXING -DMRB_NO_PRESYM -o- src/array.c > /dev/null
In file included from src/array.c:7:
In file included from include/mruby.h:115:
In file included from include/mruby/value.h:201:
include/mruby/boxing_nan.h:95:12: error: cannot initialize return object of type 'enum mrb_vtype' with an rvalue of type 'int'
return (enum mrb_vtype)(o.u >> 8) & 0x1f;
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
```
ref: #5564
```console
% cc -pedantic -S -Iinclude -DMRB_NO_PRESYM -o- src/array.c > /dev/null
In file included from src/array.c:7:
In file included from include/mruby.h:115:
In file included from include/mruby/value.h:204:
include/mruby/boxing_word.h:133:1: warning: must specify at least one argument for '...' parameter of variadic macro [-Wgnu-zero-variadic-macro-arguments]
mrb_static_assert(sizeof(mrb_value) == sizeof(union mrb_value_));
^
include/mruby.h:109:108: note: expanded from macro 'mrb_static_assert'
mrb_static_assert_expand(mrb_static_assert_selector(__VA_ARGS__, mrb_static_assert2, mrb_static_assert1)(__VA_ARGS__))
^
include/mruby.h:100:10: note: macro 'mrb_static_assert_selector' defined here
# define mrb_static_assert_selector(a, b, name, ...) name
^
1 warning generated.
```
Adding `MRB_WORDBOX_NO_FLOAT_TRUNCATE` to the build configuration in 32-bit CPU mode had a double definition.
```console
% cat myconf.rb
MRuby::Build.new do
toolchain "clang"
defines << "MRB_WORDBOX_NO_FLOAT_TRUNCATE"
cc.flags << "-m32"
linker.flags << "-m32"
enable_debug
end
% rake CONFIG=myconf.rb
CPP src/array.c -> build/host/src/array.pi
In file included from /var/tmp/mruby/src/array.c:7:
In file included from /var/tmp/mruby/include/mruby.h:115:
In file included from /var/tmp/mruby/include/mruby/value.h:203:
/var/tmp/mruby/include/mruby/boxing_word.h:11:10: warning:
'MRB_WORDBOX_NO_FLOAT_TRUNCATE' macro redefined [-Wmacro-redefined]
# define MRB_WORDBOX_NO_FLOAT_TRUNCATE
^
<command line>:3:9: note: previous definition is here
#define MRB_WORDBOX_NO_FLOAT_TRUNCATE 1
^
1 warning generated.
...SNIP...
```
The number of registers used is reduced.
Also, previously `R6` and` R7` were used, which exceeded the limit of `new_irep.nregs = 6`.
This could cause the VM stack to overrun.
This adds a build_config that will cross-build a Windows executable
using the MinGW cross-compiler and will also run the unit (i.e.
'rake test') using Wine.
For this to work, I made some modifications to the underlying test
scripts as well as some minor changes to a couple of the tests
themselves.
By uncommenting the line changed by this commit, `ruby -c build_config/default.rb` complains of a syntax error due to the illegally nested double quotes
The Difference
Since Ruby1.9, the keyword arguments were emulated by Ruby using the hash
object at the bottom of the arguments. But we have gradually moved toward
keyword arguments separated from normal (positinal) arguments.
At the same time, we value compatibility, so that Ruby3.0 keyword
arguments are somewhat compromise. Basically, keyword arguments are
separated from positional arguments, except when the method does not
take any formal keyword arguments, given keyword arguments (packed
in the hash object) are considered as the last argument.
And we also allow non symbol keys in the keyword arguments. In that
case, those keys are just passed in the `**` hash (or raise
`ArgumentError` for unknown keys).
The Instruction Changes
We have changed `OP_SEND` instruction. `OP_SEND` instruction used to
take 3 operands, the register, the symbol, the number of (positional)
arguments. The meaning of the third operand has been changed. It is now
considered as `n|(nk<<4)`, where `n` is the number of positional
arguments, and `nk` is the number of keyword arguments, both occupies
4 bits in the operand.
The number `15` in both `n` and `nk` means variable sized arguments are
packed in the object. Positional arguments will be packed in the array,
and keyword arguments will be packed in the hash object. That means
arguments more than 14 values are always packed in the object.
Arguments information for other instructions (`OP_SENDB` and `OP_SUPER`)
are also changed. It works as the third operand of `OP_SEND`. the
difference between `OP_SEND` and `OP_SENDB` is just trivial. It assigns
`nil` to the block hidden arguments (right after arguments).
The instruction `OP_SENDV` and `OP_SENDVB` are removed. Those
instructions are replaced by `OP_SEND` and `OP_SENDB` respectively with
the `15` (variable sized) argument information.
Calling Convention
When calling a method, the stack elements shall be in the order of the
receiver of the method, positional arguments, keyword arguments and the
block argument. If the number of positional or keyword arugument (`n` or
`nk`) is zero, corresponding arguments will be empty. So when `n=0` and
`nk=0` the stack layout (from bottom to top) will be:
+-----------------------+
| recv | block (or nil) |
+-----------------------+
The last elements `block` should be explicitly filled before `OP_SEND`
or assigned to `nil` by `OP_SENDB` internally. In other words, the
following have exactly same behavior:
OP_SENDB clears `block` implicitly:
```
OP_SENDB reg sym 0
```
OP_SEND clears `block` implicitly:
```
OP_LOADNIL R2
OP_SEND R2 sym 0
```
When calling a method with only positional arguments (n=0..14) without
keyword arguments, the stack layout will be like following:
+--------------------------------------------+
| recv | arg1 | ... | arg_n | block (or nil) |
+--------------------------------------------+
When calling a method with arguments packed in the array (n=15) which
means argument splat (*) is used in the actual arguments, or more than
14 arguments are passed the stack layout will be like following:
+-------------------------------+
| recv | array | block (or nil) |
+-------------------------------+
The number of the actual arguments is determined by the length of the
argument array.
When keyword arguments are given (nk>0), keyword arguments are passed
between positional arguments and the block argument. For example, when
we pass one positional argument `1` and one keyword argument `a: 2`,
the stack layout will be like:
+------------------------------------+
| recv | 1 | :a | 2 | block (or nil) |
+------------------------------------+
Note that keyword arguments consume `2*nk` elements in the stack when
`nk=0..14` (unpacked).
When calling a method with keyword arguments packed in the hash object
(nk=15) which means keyword argument splat (**) is used or more than
14 keyword arguments in the actual arguments, the stack layout will
be like:
+------------------------------+
| recv | hash | block (or nil) |
+------------------------------+
Note for mruby/c
When mruby/c authors try to support new keyword arguments, they need
to handle the new meaning of the argument information operand. If they
choose not to support keyword arguments in mruby/c, it just raise
error when `nk` (taken by `(c>>4)&0xf`) is not zero. And combine
`OP_SENDV` behavior with `OP_SEND` when `n` is `15`.
If they want to support keyword arguments seriously, contact me at
<matz@ruby.or.jp> or `@yukihiro_matz`. I can help you.
Existing call stack depth checks are unified into this check in
`cipush()`. The maximum depth is now specified by `MRB_CALL_LEVEL_MAX`
(the default is 512). The older `MRB_FUNCALL_DEPTH_MAX` is no longer
used.
Which represent `obj[int]` and `obj[int]=val` respectively where `obj`
is either `string`, `array` or `hash`, so that index access could be
faster. When `obj` is not assumed type or `R(a+1)` is not integer, the
instructions fallback to method calls.
It used to be compiled to the static string in the compiler. But the
encoding status actually depends on the runtime configuration. A new
method `Kernel#__ENCODING__` is introduced to implement the feature.
The invocation of `initialize` hook can cause infinite recursion too
easily. We stop invoking the method for safety, at the cost of less
flexibility.
The `initialize` methods (e.g. ones defined in `mrblib/10error.rb`) are
called only from `NoMethodError.new(args..)` forms.
Add n elements at once. Reduces instructions for huge array
initialization. In addition, `gen_value` function in `codegen.c` was
refactored and clarified.
`acc` was used as an index of the receiver (if positive), or a flag for
methods implemented in C. We replace `regs[ci->acc]` by `ci[1].stack[0]`.
And renamed `acc` (originally meant accumulator position) to `cci`
(means callinfo for C implemented method).
Otherwise `target_class` can be lost when it differs from `proc`'s
`target_class`, e.g. when called from `instance_eval`.
Also we should not pass `target_class` to `MRB_OBJ_ALLOC` since it
checks instance type from the class, and `target_class` may not have
proper information. ref #5272
They return the checking argument without modification, so the values
are already there. Maybe we should change the return type to `void` but
keep them unchanged for compatibility.
- Removed the `ARGV` macro.
The current path doesn't go into the mruby VM and there's also no need to separate variables.
- Use common functions to check object types.
- Use `mrb_ensure_string_type()` to check the string instead of `mrb_to_str()`.
This is for consistency with array and hash.
- Use `mrb_ensure_array_type()` to check the array instead of `to_ary()`.
- Use `mrb_ensure_hash_type()` to check the hash instead of `to_hash()`.
- Add and use `ensure_class_type()` to check class and module.
- Changed the argument index type from `mrb_int` to `int`.
Even if it is `int16_t`, it is enough.
`mrb_int` is overkill, especially if `MRB_32BIT` and `MRB_INT64` are defined.
* use predefined `mrb_ro_data_p()` for user-mode Linux and macOS
* define `MRB_LINK_TIME_RO_DATA_P` if predefined one is used
* configure macro `MRB_USE_LINK_TIME_RO_DATA_P` is no longer used
* contributions for new platforms are welcome
This now works with the `+` modifier that can be added after each specifier.
- `nil` is bypassed.
- The `s` and `z` specifiers are received in C as a `const char *`, so adding a `+` modifier will raise an exception.
- The `a` specifier is received in C as `const mrb_value *`, so adding a `+` modifier will raise an exception.
- The `|`, `*`, `&`, `?` and `:` specifiers with `+` modifier raises an exception.
If `!`/`+` exceeds one for each specifier, an exception will occur in the subsequent processing.
This is the same behavior as before.
The previously used `given` variable will be merged into the `pickarg` pointer variable, which points to the argument currently being processed for each loop.
- `#include <math.h>` is done in `mruby.h`.
Eliminate the need to worry about the `MRB_NO_FLOAT` macro.
- Include mruby header files before standard header files.
If the standard header file is already placed before `mruby.h`, the standard header file added in the future tends to be placed before `mruby.h`.
This change should some reduce the chances of macros that must be defined becoming undefined in C++ or including problematic header files in a particular mruby build configuration.
Embedding reduce memory consumption, sacrificing precision. It clips least
significant 2 bits from `mrb_float`, so if you need to keep float precision,
define `MRB_USE_FLOAT_FULL_PRECISION`.
`MRB_WORD_BOXING` and `MRB_INT64`:
`mrb_float` (`double`) is embedded in `mrb_value` clipped last 2 bits.
`MRB_WORD_BOXING` and `MRB_INT64` and `MRB_USE_FLOAT_FULL_PRECISION`:
`mrb_float` is allocated in the heaps wrapped by `struct RFloat`.
`MRB_WORD_BOXING` and `MRB_INT32` and `MRB_USE_FLOAT32`:
`mrb_float` (`float`) is embedded in `mrb_value` clipped last 2 bits.
In addition, to reserve bit space in the `mrb_value`, maximum inline
symbol length become 4 (instead of 5) in the configuration.
`MRB_WORD_BOXING` and `MRB_INT32`:
Assume `MRB_USE_FLOAT_FULL_PRECISION` and allocate Float values in heap.
Previously, the `I` specifier only checked if the object was `MRB_TT_ISTRUCT`.
So it was at risk of getting pointers to different C structs if multiple classes were to use the `MRB_TT_ISTRUCT` instance.
Change this behavior and change the C argument corresponding to the `I` specifier to `(void *, struct RClass)`.
This change is not compatible with the previous mruby.
Please note that if the user uses the previous specifications, `SIGSEGV` may occur or the machine stack may be destroyed.
resolve#5527
After building mruby, if the user compiles and links to `libmruby.a`, expose the included directory of the captured gems.
This is a change to the `<build-dir> /lib/libmruby.flags.mak` file and will result in being added to `bin/mruby-config --cflags`.
This eliminates the need for the user to look up the path and add the compiler flag if the user wants to take advantage of her gems publishing features.
In the main build with `rake CONFIG=...`, there is no problem because it can only be seen from the gems that depends directly and indirectly as before.
However, when compiling independently by the user using `bin/mruby-config`, a header file name collision may occur if a unique header directory is added.
`mrb_as_int` implicitly converts the value into the integer, but those
methods are defined for Integer class so that the value should always be
integers.
- `pc0`: `next` destination
- `pc1`: `redo` destination (renamed from `pc2`)
- `pc3`: `break` destination (renamed from `pc3`)
old `pc1` was unused so removed.
When doing `conf.enable_cxx_abi` and compiling with FreeBSD + clang or MinGW, such as `INTPTR_MAX` constant macro is not defined if `#include <stdint.h>` precedes `#include <mruby.h>`.
Currently I get a warning when I use an undefined macro, but if I don't notice it I get confused in a link error.
It can be expected that the problem will be easier to understand by making a clear error.
Adding `-Werror=undef` as a compiler flag can also result in an error, but this can be a problem if the system header file itself uses undefined macros, for example.
This patch does minimal confirmation only, but has no side effects.
When `OP_GETUPVAR` is generated right after `OP_SETUPVAR`, there is no
need to read the upvar back to the register, e.g.
3 008 OP_ADDI R2 1
3 011 OP_SETUPVAR R2 1 0
4 015 OP_GETUPVAR R2 1 0
4 019 OP_LOADI_2 R3
`OP_GETUPVAR` at the address `015` is useless. We can skip it like:
3 008 OP_ADDI R2 1
3 011 OP_SETUPVAR R2 1 0
4 015 OP_LOADI_2 R3
When `disable_cdump` is declared on a Mrbgem spec, the procedure for
loading MRuby code from the gem's `mrblib` directory is slightly
different; instead of loading the mrblib code as a Proc built from the
compiled irep, the compiled irep is loaded directly. The header files
`mruby.h` and `mruby/proc.h` are needed only when the irep is loaded
directly. They are currently included only when presym is disabled, they
should be included whenever either presym is disabled *or* when
`disable_cdump` is called. The `cdump?` predicate method happens to
return true in either case (presym disabled *or* cdump disabled), so
this change should be safe.
It uses BER number compression of delta of instruction positions and line
numbers. BER compression is a variable length number representation.
* `mrb_debug_line_ary`: array of line numbers represented in `uint16_t`.
`[lineno, lineno, ...]`
* `mrb_debug_line_flat_map`: array of `mrb_irep_debug_info_line`, which
is `struct {uint32_t pos; uint16_t lineno}`, for each line.
* `mrb_debug_line_packed_map` [new]: sequence of BER compressed 2
numbers, `pos_delta, lineno_delta`. Deltas are differences from
previous values (starting `0`). `line_entry_counts` represents total
length of a packed map string for this type.
This reverts commit fd10c72319.
I thought it was OK to restrict index value within 1 byte, but in some
cases index value could be 16 bits (2 bytes). I had several ideas to
address the issue, but reverting `fd10c72` is the easiest way. The
biggest reason is `mruby/c` still supports `OP_EXT[123]`, so that they
don't need any additional work.
```console
% for rb in `git ls-files '*/mrblib/*.rb' 'mrblib'`; do ruby30 -cw $rb > /dev/null; done
mrbgems/mruby-array-ext/mrblib/array.rb:389: warning: assigned but unused variable - ary
mrbgems/mruby-array-ext/mrblib/array.rb:663: warning: assigned but unused variable - len
mrbgems/mruby-hash-ext/mrblib/hash.rb:119: warning: possibly useless use of a variable in void context
mrbgems/mruby-hash-ext/mrblib/hash.rb:259: warning: assigned but unused variable - keys
mrbgems/mruby-io/mrblib/io.rb:229: warning: literal in condition
mrbgems/mruby-io/mrblib/io.rb:280: warning: literal in condition
mrbgems/mruby-string-ext/mrblib/string.rb:347: warning: assigned but unused variable - len
mrbgems/mruby-toplevel-ext/mrblib/toplevel.rb:2: warning: parentheses after method name is interpreted as an argument list, not a decomposed argument
```
It does not need to hold an anonymous proc for constant search.
Also, this change can be expected to cause an anonymous proc to be GC'd.
This is useful for metaprogramming that makes heavy use of the `class`/`module`/`def` syntax in the `class_eval`/`eval` method.
Example:
- code
```ruby
p ObjectSpace.count_objects
String.class_eval do
def a
end
end
p ObjectSpace.count_objects
String.class_eval do
eval <<~CODE
def b
end
CODE
end
p ObjectSpace.count_objects
```
- result of building mruby-head (d63c0df6b) with `build_config/default.rb`
```
{:TOTAL=>1024, :FREE=>262, :T_PROC=>495, :T_ENV=>61, ...}
{:TOTAL=>1024, :FREE=>259, :T_PROC=>497, :T_ENV=>62, ...}
{:TOTAL=>1024, :FREE=>255, :T_PROC=>500, :T_ENV=>63, ...}
```
- result of building mruby with this patch and `build_config/default.rb`
```
{:TOTAL=>1024, :FREE=>264, :T_PROC=>494, :T_ENV=>60, ...}
{:TOTAL=>1024, :FREE=>262, :T_PROC=>495, :T_ENV=>61, ...}
{:TOTAL=>1024, :FREE=>261, :T_PROC=>496, :T_ENV=>61, ...}
```
Previously the following code did not produce the expected results:
```ruby
bx = binding
block = bx.eval("a = 1; proc { a }")
bx.eval("a = 2")
p block.call # Expect 2 but return 1 due to a bug
```
The previous implementation of `Binding#eval` evaluated the code and then merged the top layer variables.
This patch will parse and expand the variable space before making a call to `eval`.
This means that the call to `Binding#eval` will do the parsing twice.
In addition, the following changes will be made:
- Make `mrb_parser_foreach_top_variable()`, `mrb_binding_extract_proc()` and `mrb_binding_extract_env()` functions private global functions.
- Remove the `posthook` argument from `mrb_exec_irep()`.
The `posthook` argument was introduced to implement the `binding` method.
This patch is unnecessary because it uses a different implementation method.
ref #5362fixed#5491
If no new variable was defined in the `eval` method, the variable was hidden from the nested `eval` method.
```ruby
a = 1
p eval %(b = 2; eval %(a)) # => 1 (good)
p eval %(eval %(a)) # => undefined method 'a' (NoMethodError)
```
This issue has occurred since mruby 3.0.0.
- ` mrb_block_given_p()` -- The name comes from CRuby's `rb_block_given_p ()`
At the same time, it applies to `f_instance_eval()` and `f_class_eval()` of `mruby-eval`.
This reverts commit ee3017496b.
I misunderstood something and the new behavior was different from CRuby.
The issue was reported by @dearblue, regarding #5478
Running pre-commit with GitHub Actions now gives us more tests and coverage
Remove duplicate GitHub Actions for merge conflicts and trailing whitespace
Remove duplicate checks for markdownlint and yamllint from the GitHub Super-Linter
Add new custom pre-commit hook running with a shell script to sort alphabetically and uniquify codespell.txt
Add new pre-commit hook to check spelling with codespell
https://github.com/codespell-project/codespell
Fix spelling
`mrb_float_to_str()` used to take `fmt` argument. We thought no one used
the function, and OK to remove the argument. But at least `mruby-redis`
gem used the function.
* renamed from redundant `readint_mrb_int()`
* supports only base upto 16
* no base validation (already done in parser)
* no negative read (negate after read)
* overflow detection using `mrb_int_{mul,add}_overflow()`
Instead of using copyrighted `strtod`, now use the public domain
implementation of `strtod` by Yasuhiro Matsumoto (@mattn). The function
has been renamed to `mrb_float_read()`; ref #5460 as well.
The Ruby version of `Array#rotate!` generated a rotated array and
replaced the receiver, but the C version rotates the receiver array
in-place. So the performance is improved a lot both in speed and memory
consumption. Look for the comments in `array.c` for the in-place rotating
algorithm, if you are interested.
- move `TRUE/FALSE` definition from `mrbconf.h` (not configurable)
- use C/C++ definition of boolean type for `mrb_bool`
The fix is originally written by @take-cheeze.
This work was done as follows:
- check: `git grep $'\011' -- :^oss-fuzz`
- convert: `ruby -pi -e 'nil while $_.sub!(/^(.*?)\t/) { $1 + " " * (8 - $1.size % 8) }'`
The `doc/guide/{compile,mrbgems}.md` file adds and removes whitespace to make the directory tree look the same.
In `mrbgems/mruby-socket/src/socket.c`, there is a part where the indent is changed from 4 to 2 at the same time as the tab is changed.
In the ancient Ruby, symbols are represented by integers. In that era,
to get string representation from integers, we used `Integer#id2sym`
method. Later, `Symbol` was introduced, and `id2sym` was used for
compatibility. Today, no one uses `id2sym` any longer. It is described
in ISO 30170:2012 standard but I consider it as a mistake.
Previously, the task was executed as many times as there were targets, including `mruby-compiler`.
```console
% rm mrbgems/mruby-compiler/core/y.tab.c
% rake `pwd`/mrbgems/mruby-compiler/core/y.tab.c
YACC mrbgems/mruby-compiler/core/parse.y -> mrbgems/mruby-compiler/core/y.tab.c
YACC mrbgems/mruby-compiler/core/parse.y -> mrbgems/mruby-compiler/core/y.tab.c
```
The purpose is two-fold:
1. to be able to specify a pointer directly when user data is used
When using `mrb_protect()`, it is necessary to allocate objects by `mrb_obj_cptr()` function when using user data.
Adding `mrb_protect_raw()` will make it simpler to reimplement `mrbgems/mruby-error`.
2. to correctly unwind callinfo when an exception is raised from a C function defined as a method (the main topic)
If a method call is made directly under `mrb_protect()` and a C function is called, control is returned from `mrb_protect()` if an exception occurs there.
In this case, callinfo is not restored, so it is out of sync.
Moreover, returning to mruby VM (`mrb_vm_exec()` function) in this state will indicate `ci->pc` of C function which is equal to `NULL`, and subsequent `JUMP` will cause `SIGSEGV`.
Following is an example that actually causes `SIGSEGV`:
- `crash.c`
```c
#include <mruby.h>
#include <mruby/compile.h>
#include <mruby/error.h>
static mrb_value
level1_body(mrb_state *mrb, mrb_value self)
{
return mrb_funcall(mrb, self, "level2", 0);
}
static mrb_value
level1(mrb_state *mrb, mrb_value self)
{
return mrb_protect(mrb, level1_body, self, NULL);
}
static mrb_value
level2(mrb_state *mrb, mrb_value self)
{
mrb_raise(mrb, E_RUNTIME_ERROR, "error!");
return mrb_nil_value();
}
int
main(int argc, char *argv[])
{
mrb_state *mrb = mrb_open();
mrb_define_method(mrb, mrb->object_class, "level1", level1, MRB_ARGS_NONE());
mrb_define_method(mrb, mrb->object_class, "level2", level2, MRB_ARGS_NONE());
mrb_p(mrb, mrb_load_string(mrb, "p level1"));
mrb_close(mrb);
return 0;
}
```
- compile & run
```console
% `bin/mruby-config --cc --cflags --ldflags` crash.c `bin/mruby-config --libs`
% ./a.out
zsh: segmentation fault (core dumped) ./a.out
```
After applying this patch, it will print exception object and exit normally.
The `mrb_protect()`, `mrb_ensure()` and `mrb_rescue_exceptions()` in `mrbgems/mruby-error` have been rewritten using `mrb_protect_raw()`.
The GitHub Super Linter is a more robust and better supported
tool than the current GitHub Actions we are using.
Running these checks:
ERROR_ON_MISSING_EXEC_BIT: true
VALIDATE_BASH: true
VALIDATE_BASH_EXEC: true
VALIDATE_EDITORCONFIG: true
VALIDATE_MARKDOWN: true
VALIDATE_SHELL_SHFMT: true
VALIDATE_YAML: true
https://github.com/marketplace/actions/super-linterhttps://github.com/github/super-linter
Added the GitHub Super Linter badge to the README.
Also updated the pre-commit framework and added
more documentation on pre-commit.
Added one more pre-commit check: check-executables-have-shebangs
Added one extra check for merge conflicts to our
GitHub Actions.
EditorConfig and Markdown linting.
Minor grammar and spelling fixes.
Update linter.yml
`state.c` makes a prototype declaration for the private
`mrb_protect_atexit` which is defined in `error.c`. `error.c` defines
this function with a void return type, but `state.c` defines the
prototype with an `int` return type.
This mismatch prevents mruby from compiling on stricter compilers like
emscripten.
The following methods will be made static.
- `Class#new`
- `Proc#call`
- `Kernel#catch`
Previously, static const RProc could not be registered as a method, but this has been changed to allow it.
Use `mrb_exec_irep()`. If possible, re-entry into the VM will be suppressed.
Note that due to the effect of being a tail-call, the backtrace of `Method#call` will be lost, and it will look as if the target method was called directly.
This change fixes the problem of infinite loops when redefining methods that make block calls using `mruby-method`.
```console
% bin/mruby -e 'mm = method(:proc); define_method(:proc, ->(*a, &b) { mm.call(*a, &b) }); p proc { 1 }'
trace (most recent call last):
[257] -e:1
[256] -e:1:in proc
[255] -e:1:in proc
...SNIP...
[1] -e:1:in proc
-e:1:in proc: stack level too deep (SystemStackError)
```
Change the old `mrb_exec_irep()` as-is to static `mrb_exec_irep_vm()`.
Extract the VM entry part from the old `exec_irep()` in `mruby-eval/src/eval.c` and make it the core of the new `mrb_exec_irep()`.
When argument information is not available. So it should not happen for
`yield` (error). In contrast, the error from `super` should be handled
in run time (ignored).
Adds debug source information (line/file) when mrbc uses -g.
This commit results in usable backtraces for all gems when build_config
is setup with enable_debug.
- `mrb_num_div_int(mrb,x,y)` -> `mrb_div_int(mrb,x,y)`
- `mrb_num_div_flo(mrb,x,y)` -> `mrb_div_flo(x,y)`
They are internal function not supposed to be used outside of the core.
Previously, the following code would cause a `SIGSEGV`.
```ruby
mm = method(:throw)
define_method(:throw, ->(*args) { mm.call(*args) })
catch { |tag| throw tag }
```
I think the reason is in the `mrb_yield_with_class()` function:
- Even if a C function is called, `CI_ACC_SKIP` is used
- `cipop()` is not done if globally jumping from a C function
Tests for (`Float` or `Integer`) `op` `Complex`. Also added test
dependency to `mruby-rational` since `int_div` definition relies on
`Rational` when `MRB_USE_RATIONAL` is defined.
It removes non-static function, so that strictly saying, it's
an incompatible change. But the function was added recently and I am
sure no one uses it yet.
- define `MRB_TT_COMPLEX`
- change object structure (`struct RComplex`)
- add memory management for `MRB_TT_COMPLEX`
- avoid operator overloading as much as possible
- as a result, performance improved a log
- should work with and without `Rational` defined
- define `MRB_TT_RATIONAL`
- change object structure (`struct RRational`)
- add memory management for `MRB_TT_RATIONAL`
- avoid operator overloading as much as possible
- implement division overloading in C
- as a result, performance improved a lot
Since `mruby` does not have `Bignum`, `Float#divmod` could overflow, so
it will return `Float` values when the divided value does not fit in
`mrb_int`. This behavior will be changed when `Bignum` is introduced to
`mruby` in the future.
#### Before this patch:
```console
$ bin/mruby -e 'p(Float::NAN/0)'
Infinity
```
#### After this patch (same as Ruby):
```console
$ bin/mruby -e 'p(Float::NAN/0)'
NaN
```
If I break out of a block given to `MRuby::Build.new` with `break` or `throw`, I will get a seemingly inexplicable error because the `presym`-related initialization is not done.
```console
% cat build_config1.rb
MRuby::Build.new do
toolchain
break
end
% rake CONFIG=build_config1.rb
rake aborted!
external mrbc or mruby-bin-mrbc gem in current('host') or 'host' build is required
/var/tmp/mruby/lib/mruby/build.rb:332:in `mrbcfile'
/var/tmp/mruby/tasks/mrblib.rake:9:in `block in <top (required)>'
/var/tmp/mruby/lib/mruby/build.rb:18:in `instance_eval'
/var/tmp/mruby/lib/mruby/build.rb:18:in `block in each_target'
/var/tmp/mruby/lib/mruby/build.rb:17:in `each'
/var/tmp/mruby/lib/mruby/build.rb:17:in `each_target'
/var/tmp/mruby/tasks/mrblib.rake:1:in `<top (required)>'
/var/tmp/mruby/Rakefile:27:in `load'
/var/tmp/mruby/Rakefile:27:in `<top (required)>'
(See full trace by running task with --trace)
```
If a non-exceptional global jump occurs, it can be initialized by `ensure` to solve this problem.
It used to be return the default value if available, but it should
ignore the default value for behavior consistency. CRuby will adopt
this behavior too in the future. [ruby-bugs:16908]
Add new pool value type `IREP_TT_BIGINT` and generate integer overflow
error in the VM. In the future, `mruby` will support `Bignum` for
integers bigger than `mrb_int` (probably using `mpz`).
- Added to `mruby-binding-core`
- `Binding#local_variable_defined?`
- `Binding#local_variable_get`
- `Binding#local_variable_set`
- `Binding#local_variables`
- `Binding#receiver`
- `Binding#source_location`
- `Binding#inspect`
- Added to `mruby-proc-binding`
- `Proc#binding`
The reason for separating `Proc#binding` is that core-mrbgems has a method that returns a closure object to minimize possible problems with being able to manipulate internal variables.
By separating it as different mrbgem, each user can judge this problem and incorporate it arbitrarily.
I get an error because the current mruby does not have a `Kernel#warn` method.
But the warning itself is useful and I'll just comment it out in case it's implemented in the future.
Normally a single spell checker can't find all the mistakes or check all types of code.
These mistakes were found by another spell checker inside my editor with a more manual sift / find.
Official -> "The individual jobs in a workflow can interact with (and compromise) other jobs. For example, a job querying the environment variables used by a later job, writing files to a shared directory that a later job processes, or even more directly by interacting with the Docker socket and inspecting other running containers and executing commands in them.
This means that a compromise of a single action within a workflow can be very significant, as that compromised action would have access to all secrets configured on your repository, and can use the GITHUB_TOKEN to write to the repository. Consequently, there is significant risk in sourcing actions from third-party repositories on GitHub. "
https://docs.github.com/en/actions/learn-github-actions/security-hardening-for-github-actions#using-third-party-actions
In `h_check_modified()`, in the case of `MRB_NO_BOXING`, `ht_ea()` or
`ht_ea_capa()` for AR may read uninitialized area. Therefore, do not use
those macros for AR in `MRB_NO_BOXING` (but in the case of `MRB_64BIT`,
`ht_ea_capa()` is the same as `ar_ea_capa()`, so use it).
fix#5332
### Example
##### example.rb
```ruby
h = {}
(1..17).each{h[_1] = _1}
(1..16).each{h.delete(_1)}
h.rehash
```
##### ASAN report
```console
$ bin/mruby example.rb
==52587==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x602000006998 at pc 0x55a29cddf96b bp 0x7fff7b1b1720 sp 0x7fff7b1b1710
READ of size 4 at 0x602000006998 thread T0
#0 0x55a29cddf96a in ib_it_next /mruby/src/hash.c:639
#1 0x55a29cde2ca2 in ht_rehash /mruby/src/hash.c:900
#2 0x55a29cde379f in h_rehash /mruby/src/hash.c:996
#3 0x55a29cde7f3d in mrb_hash_rehash /mruby/src/hash.c:1735
#4 0x55a29ce77b62 in mrb_vm_exec /mruby/src/vm.c:1451
#5 0x55a29ce5fa88 in mrb_vm_run /mruby/src/vm.c:981
#6 0x55a29ceb87e1 in mrb_top_run /mruby/src/vm.c:2874
#7 0x55a29cf36bdf in mrb_load_exec mrbgems/mruby-compiler/core/parse.y:6805
#8 0x55a29cf36f25 in mrb_load_detect_file_cxt mrbgems/mruby-compiler/core/parse.y:6848
#9 0x55a29cdba0a2 in main /mruby/mrbgems/mruby-bin-mruby/tools/mruby/mruby.c:347
#10 0x7f24ef43b0b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)
#11 0x55a29cdb4a6d in _start (/mruby/bin/mruby+0x2a3a6d)
0x602000006998 is located 0 bytes to the right of 8-byte region [0x602000006990,0x602000006998)
allocated by thread T0 here:
#0 0x7f24f01cfffe in __interceptor_realloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10dffe)
#1 0x55a29ceb9440 in mrb_default_allocf /mruby/src/state.c:68
#2 0x55a29cdba747 in mrb_realloc_simple /mruby/src/gc.c:228
#3 0x55a29cdba928 in mrb_realloc /mruby/src/gc.c:242
#4 0x55a29cde12e5 in ht_init /mruby/src/hash.c:749
#5 0x55a29cde2b8e in ht_rehash /mruby/src/hash.c:897
#6 0x55a29cde379f in h_rehash /mruby/src/hash.c:996
#7 0x55a29cde7f3d in mrb_hash_rehash /mruby/src/hash.c:1735
#8 0x55a29ce77b62 in mrb_vm_exec /mruby/src/vm.c:1451
#9 0x55a29ce5fa88 in mrb_vm_run /mruby/src/vm.c:981
#10 0x55a29ceb87e1 in mrb_top_run /mruby/src/vm.c:2874
#11 0x55a29cf36bdf in mrb_load_exec mrbgems/mruby-compiler/core/parse.y:6805
#12 0x55a29cf36f25 in mrb_load_detect_file_cxt mrbgems/mruby-compiler/core/parse.y:6848
#13 0x55a29cdba0a2 in main /mruby/mrbgems/mruby-bin-mruby/tools/mruby/mruby.c:347
#14 0x7f24ef43b0b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)
```
When there is a corresponding tag, the `RBreak` object is used to make a global jump.
Like CRuby, it can't be caught by `rescue`.
It is also the same as CRuby that it can be canceled in the middle by `ensure`.
### How to find the corresponding tag with `throw`
The called `catch` method remains in the call stack, and the tag also remains in the stack at that time.
So it is possible to find the called location by searching the two.
Note that no method can be given to the `proc` object specified in `RBreak`.
Therefore, inside the `catch` method, the argument block is called in a seemingly meaningless closure.
Also, as a countermeasure against `alias` etc., the `proc` object, which is the body of the `catch` method, is saved when mrbgem is initialized.
Because if the configuration file didn't contain any `conf.enable_test`, `rake test` would report an exception and exit.
```console
% cat my_config.rb
MRuby::Build.new { toolchain }
% rake MRUBY_CONFIG=my_config.rb test
...SNIP...
rake aborted!
NoMethodError: undefined method `invoke' for nil:NilClass
/var/tmp/mruby/tasks/test.rake:24:in `block (3 levels) in <top (required)>'
Tasks: TOP => test => test:build => test:build:lib
(See full trace by running task with --trace)
```
If the current directory is different from `MRUBY_ROOT` and it has` conf.build_dir` and `conf.enable_cxx_exception` set, it was generating a pathname outside of` build_dir`.
As a result, in some cases files unrelated to mruby could be linked.
```console
% pwd
/tmp/mruby/1/2/3/4/5/6
% mruby_dir=/tmp/mruby/a/b/c/d/mruby
% cat my_config.rb
MRuby::Build.new("host", "build/to/custom/directory") do
toolchain
enable_cxx_exception
end
% rake MRUBY_CONFIG=my_config.rb -f $mruby_dir/Rakefile > logs
% grep CXX logs
CXX a/b/c/d/mruby/src/error-cxx.cxx -> a/b/c/d/mruby/src/error-cxx.o
CXX a/b/c/d/mruby/src/gc-cxx.cxx -> a/b/c/d/mruby/src/gc-cxx.o
CXX a/b/c/d/mruby/src/vm-cxx.cxx -> a/b/c/d/mruby/src/vm-cxx.o
CXX a/b/c/d/mruby/mrbgems/mruby-compiler/core/codegen-cxx.cxx -> a/b/c/d/mruby/mrbgems/mruby-compiler/core/codegen-cxx.o
CXX a/b/c/d/mruby/mrbgems/mruby-compiler/core/y.tab-cxx.cxx -> a/b/c/d/mruby/mrbgems/mruby-compiler/core/y.tab-cxx.o
CXX ../a/b/c/d/mruby/src/error-cxx.cxx -> ../a/b/c/d/mruby/src/error-cxx.o
CXX ../a/b/c/d/mruby/src/gc-cxx.cxx -> ../a/b/c/d/mruby/src/gc-cxx.o
CXX ../a/b/c/d/mruby/src/vm-cxx.cxx -> ../a/b/c/d/mruby/src/vm-cxx.o
```
Prevents the auto-generated mrbc target source code from being compiled under host conditions.
This is because a build error occurred when `conf.enable_cxx_exception` was set.
* Use `_Complex` instead of `complex` (MSYS2 do not support `complex`)
* Use `_Dcomplex` instead of `_Complex` on MSCV
* Avoid operator division and multiplication of complex
### Example
```ruby
begin
throw 1
rescue Exception => e
puts e.message
end
```
#### Before this patch:
```console
$ bin/mruby example.rb
uncaught throw :1
```
#### After this patch (same as Ruby):
```console
$ bin/mruby example.rb
uncaught throw 1
```
This gem uses C99 `_Complex` features. You need a C compiler that
supports `_Complex` to enable this gem. All `gcc`, `clang`, `VC` support
`_Complex` so there should not be a big problem.
`.pi` files are created for `.o` files that `build.products` depends on, but
an error will occur if the build rule is unknown, so add a check.
I don't think this situation would normally arise. However, in
`mattn/mruby-onig-regexp`, when using bundled onigmo, onigmo's `.o` files
are added to dependency of `libmruby.a` in the second and subsequent builds,
and mruby does not know the build rule, so the following error had occured.
```console
rake aborted!
Don't know how to build task '/mruby/build/host/mrbgems/mruby-onig-regexp/onigmo-6.2.0/libonig_objs/ascii.pi' (See the list of available tasks with `rake --tasks`)
```
We have introduced following new instructions.
* `OP_LAMBDA16`
* `OP_BLOCK16`
* `OP_METHOD16`
* `OP_EXEC16`
Each instruction uses 16 bits operand for `reps` index. Since new
instructions are added, `mruby/c` VM should be updated.
Due to new instructions, dump format compatibility is lost, we have
increased `RITE_BINARY_MAJOR_VER`.
In addition, we have decreased the size of `refcnt` in `mrb_irep` from
`uint32_t` to `uint16_t`, which is reasonably big enough.
The `init_SYMBOLS()` function implicitly defined in `MRB_PRESYM_DEFINE_VAR_AND_INITER()` requires some familiarity when trying to find it from the caller.
By introducing `MRB_PRESYM_INIT_SYMBOLS()`, it is possible to find directly from the identifier.
If we modify an option that may have been specified by users, we may
make unintended changes, so it is better not to modify it as much as
possible, IMO.
Because a structure that is an element of `presym_table` has padding, split
it into individual arrays for name and length.
#### Result (64-bit CPU with full-core gembox)
| | mruby | libmruby.a |
|--------|------------|------------|
| Before | 1,087,444B | 1,476,872B |
| After | 1,079,340B | 1,469,784B |
Instead of including `mruby/presym.h` everywhere, we provided the
fallback `mruby/presym.inc` under `include/mruby` directory, and specify
`-I<build-dir>/include` before `-I<top-dir>/include` in `presym.rake`.
So even when someone drops `-I<build-dir>/include` in compiler options,
it just compiles without failure.
`calc_crc_16_ccitt()` consumes a lot of clock cycles in programs like
`mrbtest` which loads a lot of dumped binary. Error detection for flaky
channels should be done in the higher level.
Note: `mruby/c` should be updated to support this change.
The Hash implementation assumed that there were always empty buckets, but
sometimes there were only active or deleted buckets (no empty buckets).
Therefore, fix it so that this situation does not occur.
### Example
```ruby
# example.rb
class A
attr_reader :v
def initialize(v) @v = v end
def ==(o) @v == o.v end
def hash; @v end
def to_s; "#{self.class}[#{@v}]" end
alias eql? ==
alias inspect to_s
end
keys = (0..31).map{A.new(_1)}
h = {}
(0..16).each{h[keys[_1]] = _1}
(17..31).each do
k = keys[_1]
h[k] = _1
h.delete(k)
end
p h.keys
```
#### Before this patch:
```console
$ bin/mruby example.rb
[A[0], A[1], A[2], A[3], A[4], A[5], A[6], A[7], A[8], A[9], A[10], A[11], A[12], A[13], A[14], A[15], A[16], A[30], A[31]]
```
#### After this patch:
```console
$ bin/mruby example.rb
[A[0], A[1], A[2], A[3], A[4], A[5], A[6], A[7], A[8], A[9], A[10], A[11], A[12], A[13], A[14], A[15], A[16]]
```
When I `#call` the "proc" object created by the `mrb_proc_new_cfunc()` function from Ruby space, the return value did not go into the correct stack position.
This can destroy the calling variable.
This issue is now caused by #5272. sorry.
The cause is that `mrbgem.rake` of` mruby-test` gem is loaded when test
code is requested to be built, but when `mrbgem.rake` is loaded,
`MRuby::Gem.current` is updated, which is not thread safe.
Address this by not loading `mrbgem.rake` in parallel.
Previously I used the `RData` object to avoid a memory leak in `mrb_irep` if `src/load.c` failed.
ref: https://github.com/mruby/mruby/pull/4250
commit: f1523d2404
Considering that the `RProc` object will be created in the subsequent process, it is preferable to create the `RProc` object from the beginning.
Along with this, the inside of `read_irep()` is replaced with the processing centered on the `RProc` object.
The global function that returns the `mrb_irep` pointer is still provided for compatibility.
This enhances self-containment.
- Changed the `mrb_callinfo::pc` field to point to itself.
Previously it indicated the return destination of the previous call level.
`mrb_callinfo::pc` will now hold the address to its own `proc->body.irep->iseq`.
- Removed `mrb_callinfo::err` field.
This is because `mrb_callinfo::pc - 1` is semantically the same as the previous `err`.
- The `pc0` and `pc_save` variables in `mrb_vm_exec()` are no longer needed and have been deleted.
- It removes the argument because `cipush()` doesn't need to save the previous `pc`.
This enhances self-containment.
Previously `mrb_context::stack` had the current call level stack, but now it owns it.
The `mrb_context::stack` field, which is no longer needed, will be removed.
With this change, the test code will not be built unless `rake test` is
run, so there will be almost no side effects even if `enable_test` is
always set (but, gems specified by `add_test_dependency` are included
in `libmruby.a`).
Also added are `test: build` task, which only builds the test code
(including the main code), and `test: run` task, which only runs tests
independent of build. Therefore, the idiom for building in parallel and
not running tests in parallel is `rake -m test:build && rake test:run`.
#### Before this patch:
```ruby
__send__ #=> wrong number of arguments
{}.default(1,2) #=> wrong number of arguments
```
#### After this patch:
```ruby
__send__ #=> wrong number of arguments (given 0, expected 1+)
{}.default(1,2) #=> wrong number of arguments (given 2, expected 0..1)
```
The comment says that if `mrb_parse_nstring()` returns `NULL`, it is only out of memory.
I'm worried about compatibility if I set the exception class to `NoMemoryError`, so it's still `RuntimeError`.
- You can use a CODEOWNERS file to define individuals or teams that are responsible for code in a repository.
- Code owners are automatically requested for review when someone opens a pull request that modifies code that they own. Code owners are not automatically requested to review draft pull requests.
- https://docs.github.com/en/free-pro-team@latest/github/creating-cloning-and-archiving-repositories/about-code-owners
- This creates a bit more automation so that after you merge this PR, next time someone submits a PR @matz will be automatically assigned as the PR reviewer as CODEOWNER
- CODEOWNERS can even be assigned directories or file types
I think that it is easy to use if it is divided according to the type of library and the general purpose.
It is a subdivision from the previous `default.gembox`.
By type gembox:
- `stdlib`: Add some standard Ruby methods not provided by core.
- `stdlib-ext`: Add some standard Ruby methods that are not provided by `stdlib`.
- `stdlib-io`: Provides `IO`, `File`, `Socket`, `print`, etc. Conflict with `MRB_NO_STDIO`
- `math`: Add a class related to math. Conflict with `MRB_NO_FLOAT`
- `metaprog`: Adds features related to metaprogramming.
Purpose gembox:
- `default.gembox`: Import the same gems as before
- `default-no-stdio.gembox`: Import the` stdlib` and `math`
- `default-no-fpu.gembox`: Import the` stdlib` only
* Skip if commit message contains `ci skip`, `skip ci`, or `skip gha`
ENCLOSED WITH BRACKETS (excluding lint job).
* Separate build and test. This is because builds can be run in parallel,
but running tests in parallel can cause logs to get mixed up or not
finished.
* Don't use Chocolatey because it seems to take 1-2 minutes to start up.
* Use the cache better.
* Use `actions/checkout@v2` instead of `actions/checkout@v1`.
* Remove unnecessary package installation.
* Remove unnecessary flag settings.
* Remove `-j` for rake because it doesn't seem to have any effect.
* Rename `main.yml` to `oss-fuzz.yml` to clarify.
Run on pull request only
Using https://www.npmjs.com/package/markdownlint-cli
Lint Markdown for rules:
- MD009/no-trailing-spaces
- MD012/no-multiple-blanks
- MD022/blanks-around-headings
- MD031/blanks-around-fences
- MD032/blanks-around-lists
Currently, there are build configuration files for CI, `travis.rb` and
`appveyor.rb`, but they are used for GCC/Clang and MSVC, not for Travis CI
and Appveyor, respectively.
Therefore, rename them to `gcc-clang.rb` and `msvc.rb`, respectively, and
move them under `build_config/ci/` to clarify that they are for CI.
Union initialization initializes the first member. The first member of
`RVALUE` is `struct free_obj`, but because it is only 4-words, it seems that
initialization after the 5th word is not ensured.
Therefore, I created 6-words `struct RVALUE_initializer` for initialization
and made it the first member.
The accuracy is greatly improved by using the C preprocessor to scan C
sources for presym. C preprocessor can perfectly interpret all comments and
preprocessor directives, so it can detect all symbols defined, for example
`mrbgems/mruby-socket/src/const.cstub`.
Also, as described later, this change will greatly improve the accuracy of
presym detection from Ruby sources.
## Result
The number of lines in the `presym` file for all gems is as follows:
```console
Previous: 999 (false positive = 89, undetected = 297)
New: 1207
```
## Build process
The new build process (with presym) is as follows:
1. Build `mrbc` without presym (more on building without presym later).
2. Compile Ruby sources to C struct format with the `mrbc` created in
step 1, and create` mrblib.c` and `gem_init.c`. Note that the symbols
in the created files are output as `MRB_SYM` family macros or
`mrb_intern_lit` instead of IDs (details will be described later).
3. C preprocessor processes C sources including the created files of
step 2 and outputs them as `.i` files. In these files, for example,
`MRB_IVSYM(foo)` is converted to `<@! "@" "foo" !@>` and
`mrb_define_module(mrb, "Foo")` is converted to `<@! "Foo" !@>`.
4. Scan the files created in step 3 and create `presym` and` presym.inc`
files.
The files created in step 2 should output all static symbols defined in Ruby
sources, including local variables, so we can detect all presyms by just
scanning C sources without scanning Ruby sources directly.
Further, by this process, the files to be scanned becomes the same as the
files to be compiled, so that there is no excess or deficiency.
## Related changes
The following changes have been made in relation to realizing this feature.
### Allow build without presym
It enables build without presym to achieve the "Build process: 1". This
incorporates #5202, see its issue for details.
Note that when presym is enabled, even adding a local variable to a Ruby
source may change contents of presym and require recompilation of almost
all C sources. This is inconvenient, especially during trial and error in
development, but this feature is also useful because it does not cause
this problem if presym is disabled.
### Automatically create build target for `mrbc` without presym
The `mrbc` used in the "Build process: 1" will be built by automatically
creating a build target for it. The build name is `SOURCE_BUILD_NAME/mrbc`.
### Constantize output of C struct format by `mrbc`
To realizing the "Build process: 2", as mentioned above, symbol IDs are not
output directly in C struct format output by `mrbc`. As a result, the output
becomes constant regardless of the state of presym at the time of `mrbc`
build, and it is possible to detect symbols of Ruby sources in the same way
as other C sources.
Note that `mrb_intern_lit` is used for symbols that do not become presym,
but in this state, the corresponding element in the symbol array cannot be
statically initialized, so it is initialized at run time (therefore, in this
case, the `const` qualifier is not added to the symbol array).
### Specify arbitrary `mrbc` file
To realizing the "Build process: 2", enabled to specify `mrbc` created by
another build target or pre-built` mrbc`. Use `MRuby::Build#mrbcfile =` to
specify it explicitly. You can omit the "Build process: 1" by specifying
pre-built `mrbc`, and you can always use an optimized build to compile Ruby
sources faster. I think changes that affect the output of `mrbc` are rare,
so in many cases it helps to improve efficiency.
With presym, the build will be a little slower due to more build steps, but
this feature will improve it a bit.
### Create presym files for each build target
This feature was proposed at #5194 and merged once, but was reverted in
5c205e6e due to problems especially with cross-compilation. It has been
introduced again because this change solves the problem.
The presym files will be created below.
* `build/NAME/presym`
* `build/NAME/include/mruby/presym.inc`
### Other changes
* Because presym detection accuracy is greatly improved as mentioned above,
`MRuby::Gem::Specification#cdump?` is set to true by default, and
`disable_cdump` is added instead of `enable_cdump`. Also, support for gem
specific presym files has been discontinued (https://github.com/mruby/mruby/issues/5151#issuecomment-730967232).
* Previously, `mrbc` was automatically created for the `host` build, but it
will not be created if the build target for `mrbc` mentioned above is
automatically created. At this time, `mrbc` file of the `mrbc` build is
copied to` bin/`.
* Two types of `.d` files will be created, `.o.d` and `.i.d`. oThis is
because if `.i` depends on `presym.inc`, the dependency will circulate, so
the `.d` file cannot be shared.
* Changed file created with `enable_cxx_exception` to `X-cxx.cxx` from
`X.cxx` to use the mruby standard Rake rule.
### Note
Almost all C sources will need to be recompiled if there are any changes to
`persym.inc` (if not recompiled properly, it will often result in run-time
error). If `gcc` toolchain is used, dependencies are resolved by the `.d`
file, so it become automatically recompile target, but if not (e.g. MSVC),
it is necessary to manually make it recompile target.
Also, even if `gcc` toolchain is used, it may not become recompile target if
external gems does not use the mruby standard Rake rule. In particular, if
the standard rule is overwritten, such as
https://github.com/mruby/mruby/pull/5112/files, `.d` file will not be read,
so be careful.
Previously, `libmruby.a` was created even if only `mruby-bin-mrbc` or`
mruby-compiler` was specified for gem, but by specifying `disable_libmruby`,
the creation of `libmruby.a` can be suppressed.
### Note
The https://github.com/mruby/mruby/pull/5084#issuecomment-723521971
incompatibility seems to be difficult for users to avoid, so the original
behavior has been restored. Therefore, if we need `mrbc` other than the
`host` build, we need to explicitly specify `mruby-bin-mrbc` gem.
Due to the above changes, `build_config/boxing.rb` etc. will not work, but I
have added` mruby-bin-mrbc` to `default.gembox` to fix it. I don't think
this change is a big deal because originally `mruby-compiler` was included.
Module#include and Module#prepend now affect classes and modules
that have already included or prepended the receiver, mirroring the
behavior if the arguments were included in the receiver before
the other modules and classes included or prepended the receiver.
```ruby
class C; end
module M1; end
module M2; end
C.include M1
M1.include M2
p C.ancestors #=> [C, M1, M2, Object, Kernel, BasicObject]
```
Currently, adding gems to build configuration and rebuilding does not
regenerate `presym`, which can lead to build errors.
Therefore in this case we need to remove the `presym` file and then rebuild
mruby, but when the` presym` file is regenerated we will need to recompile
most of the files, so it seems useful to have the `presym` file deleted by
`rake clean`.
Due to the above changes, it may not work with the existing build
configurations in cross-compilation (even if we can build without presym),
therefore revert it once (ref https://github.com/mruby/mruby/pull/5202#issuecomment-735412643). Sorry for the lack of consideration.
Currently, the size of writing in heap by write_irep_record() is
bigger than The size that is calculated by get_irep_record_size.
Therefore, irep is dumped over the size of allocating memory when we
execute dump_irep().
Previously, presym files were always created in `build/{presym,presym.inc}`.
However, this constraint is inconvenient because it is common to use
multiple build configurations and build targets in a single mruby tree.
Therefore, change to create presym file for each build target.
Jump target address is `operand (16bit)` + `address of next instruction`.
In addition, `ilen` was made `uint32_t` so that `iseq` length limitation
of 65536 is removed. Only jump target address should be within signed
16bit (-32768 .. 32767).
For example, in the case of the C++ compiler, it is output as
`CXX build/host/src/gc.cxx -> build/host/src/gc.cxx.o` (FYI, in the case
of `enable_cxx_abi`, it outputs `CC ...` because the option to compile as
C++ is added to C compiler).
Chang to compare string length first.
### Benchmark
#### Code
* https://github.com/shuujii/mruby-presym_find-benchmark
#### Result
```console
Previous: 10.240772M i/s (25M times in 2.441222s)
New: 16.412985M i/s (25M times in 1.523184s)
```
If it gives clang-11.0 `-Wembedded-directive`, a warning will be reported in `include/mruby/boxing_nan.h`.
```
include/mruby/boxing_nan.h:48:2: warning: embedding a directive within macro arguments has undefined behavior [-Wembedded-directive]
include/mruby/boxing_nan.h:52:2: warning: embedding a directive within macro arguments has undefined behavior [-Wembedded-directive]
```
The cause of this is #5117.
ref. e993b83c50
* Output `GEN` log for generated files
* `MRBC` log is outputted one for each `mrbc` execution
#### Before this patch:
```console
CC src/array.c -> build/host/src/array.o
(snip)
GEN mrblib/*.rb -> build/host/mrblib/mrblib.c
MRBC mrblib/00class.rb
MRBC mrblib/10error.rb
(snip)
CC mrbgems/mruby-time/src/time.c -> build/host/mrbgems/mruby-time/src/time.o
MRBC mrbgems/mruby-time/mrblib/time.rb
(snip)
CC mrbgems/mruby-socket/test/sockettest.c -> build/host/mrbgems/mruby-socket/test/sockettest.o
MRBC mrbgems/mruby-socket/test/addrinfo.rb
MRBC mrbgems/mruby-socket/test/basicsocket.rb
(snip)
```
#### After this patch:
```console
GEN build/presym
GEN build/presym.inc
CC src/array.c -> build/host/src/array.o
(snip)
GEN mrblib/*.rb -> build/host/mrblib/mrblib.c
MRBC mrblib/00class.rb
mrblib/10error.rb
(snip)
CC mrbgems/mruby-time/src/time.c -> build/host/mrbgems/mruby-time/src/time.o
GEN build/host/mrbgems/mruby-time/gem_init.c
MRBC mrbgems/mruby-time/mrblib/time.rb
(snip)
CC mrbgems/mruby-socket/test/sockettest.c -> build/host/mrbgems/mruby-socket/test/sockettest.o
GEN build/host/mrbgems/mruby-socket/gem_test.c
MRBC mrbgems/mruby-socket/test/addrinfo.rb
MRBC mrbgems/mruby-socket/test/basicsocket.rb
(snip)
```
Integrate the implementation with `Module#define_method`.
- Introduce the internal function `mrb_mod_define_method_m()` (no static)
- The `Object#define_singleton_method` method can now accept a second argument
It is not decides by the extension.
In order to be recognized as a `.mrb` file, the following three points must be satisfied:
- File starts with "RITE"
- At least `sizeof(struct rite_binary_header)` bytes can be read
- `NUL` is included in the first 64 bytes of the file
If these are not met, it is judged as a text file and it is processed as a Ruby script.
The `bin/mruby -b` switch is still available which treats the given file as a `.mrb` file.
New `MRB_API` function:
- `include/mruby/compile.h` and `mrbgems/mruby-compiler/core/parse.y`
- `mrb_load_detect_file_cxt()` (remove with `MRB_DISABLE_STDIO`)
NOTE:
- Even script files now always open in binary mode for `bin/mruby`.
The `\r\n` is handled by the `nextc()` function already, so there is no problem even on Windows.
- The `nextc0()` function in `mrbgems/mruby-compiler/core/parse.y` can now specify a string buffer and a file pointer at the same time.
In this case, get it from the string buffer first.
This patch includes modifies by comment of https://github.com/mruby/mruby/pull/5157.
I tried `$GITHUB_PATH` and `$GITHUB_ENV` instead of `set-env`, but
for some reason path was not recognized, so I changed to using
`ACTIONS_ALLOW_UNSECURE_COMMANDS`.
ref: https://github.com/mruby/mruby/pull/4959#discussion_r402086196
Compiles twice because it falls back to `build.filename(command)` when
`command` fails. This process was added at 9968af4 to support `ccache gcc`
etc. At that time, it seems that it was necessary because
`build.filename(command)` quoted the whole `command`, but now it does not
quote, so we can just run `build.filename(command)`.
### Example
```console
$ echo 1 > src/a.c
$ rake -v
```
#### Before this patch:
```console
(snip)
gcc -std=gnu99 -g -O3 -Wall -Wundef -Wdeclaration-after-statement -Werror-implicit-function-declaration -Wwrite-strings -I"/mruby/mruby/include" -MMD -o "/mruby/mruby/build/host/src/a.o" -c "/mruby/mruby/src/a.c"
/mruby/mruby/src/a.c:1:1: error: expected identifier or '('
1
^
1 error generated.
gcc -std=gnu99 -g -O3 -Wall -Wundef -Wdeclaration-after-statement -Werror-implicit-function-declaration -Wwrite-strings -I"/mruby/mruby/include" -MMD -o "/mruby/mruby/build/host/src/a.o" -c "/mruby/mruby/src/a.c"
/mruby/mruby/src/a.c:1:1: error: expected identifier or '('
1
^
1 error generated.
rake aborted!
(snip)
```
#### After this patch:
```console
(snip)
gcc -std=gnu99 -g -O3 -Wall -Wundef -Wdeclaration-after-statement -Werror-implicit-function-declaration -Wwrite-strings -I"/mruby/mruby/include" -MMD -o "/mruby/mruby/build/host/src/a.o" -c "/mruby/mruby/src/a.c"
/mruby/mruby/src/a.c:1:1: error: expected identifier or '('
1
^
1 error generated.
rake aborted!
(snip)
```
* The term `hash_table` can be misleading because the return value of this
function includes memory usage of entire `Hash` object, including not only
hash table part but also entry list part, etc.
* This function takes a `Hash` object as a receiver and is defined in
`src/hash.c`, so it is natural to have a `mrb_hash_` prefix.
To be also able to build mruby without presym in the future. However,
`MRB_QSYM` has been removed and changed as follows:
### Example
| Type | Symbol | Previous Style | New Style |
|---------------------------|--------|------------------|----------------|
| Operator | & | MRB_QSYM(and) | MRB_OPSYM(and) |
| Class Variable | @@foo | MRB_QSYM(00_foo) | MRB_CVSYM(foo) |
| Instance Variable | @foo | MRB_QSYM(0_foo) | MRB_IVSYM(foo) |
| Method with Bang | foo! | MRB_QSYM(foo_b) | MRB_SYM_B(foo) |
| Method with Question mark | foo? | MRB_QSYM(foo_p) | MRB_SYM_Q(foo) |
| Mmethod with Equal | foo= | MRB_QSYM(foo_e) | MRB_SYM_E(foo) |
This change makes it possible to define, for example, `MRB_IVSYM(foo)` as
`mrb_intern_lit(mrb, "@" "foo")`, which is useful if we support building
without presym in the future.
We don't need to require valid STDIN/STDOUT/STDERR. If we require it,
we can't use mruby on an environment that doesn't have valid
STDIN/STDOUT/STDERR such as Windows service process. Windows service
process doesn't have valid STDIN.
In the case of `y.tab.o`, `y.tab.d` was not read, so `y.tab.c` was not
recompiled by updating header files.
It seemed that the cause was overwriting the rake rule, so I deleted
unnecessary settings.
New instructions:
* OP_LOADL16
* OP_LOADSYM16
* OP_STRING16
Size of pools, symbols are `int16_t` but offset representation in the
bytecode was 8 bits. Size of child `irep` array is `int16_t`, too but
this change does not address it.
Since `R-assignment` in CRuby is abandoned. Single-line pattern matching
in `mruby` only matches single local variable at the moment. Currently
it works as a right assignment to a local variable. It will be enhanced
in the future.
When `rake -m` and so on are used to build in parallel, building may be
started before presym files are generated. Then, for example, the following
error occurs and this issue is fixed.
```console
In file included from /Users/shuujii/mruby/mruby/include/mruby.h:92:
/mruby/mruby/include/mruby/presym.h:16:10: fatal error: '../build/presym.inc' file not found
#include <../build/presym.inc>
^~~~~~~~~~~~~~~~~~~~~
```
If you confirm the gem is cdump safe, add `spec.enable_cdump` in
`mrbgem.rake` file. Some external gems e.g. `mruby-mgem-template` do
not work with cdump.
It was relative from the directory of the build configuration file, but
the document says it's relative from `MRUBY_ROOT`. When the default
configuration file was `MRUBY_ROOT/build_config.rb`, it was OK for most
of the cases.
In addition, update the documents referring `build_config.rb` which is
no longer used. The new `build_config.rb` describes the new configuration
structure in the comment.
The `mrb_get_argv()` function will now return `const mrb_value *`.
This is because it is difficult for the caller to check if it is a splat argument (array object) and to write-barrier if necessary.
The "a"/"*" specifier of the `mrb_get_args()` function will now return `const mrb_value *`.
This is because it is difficult for the caller to check if it is an array object and write-barrier if necessary.
And it requires calling `mrb_ary_modify()` on the unmodified array object, which is also difficult (this is similar to #5087).
* In explanation of mruby, the expression `build_config.rb` is frequently
used including official documents, so I think that it will not make sense
if the file is no longer used.
* The `MRUBY_TARGET` mechanism seems to have little improvement, so I don't
think it should be changed to avoid unnecessary confusion.
* `MRUBY_TARGET` and `MRuby.targets` represent somewhat different things,
so using the same term "target" is a bit confusing.
The mechanism that can be written short when using a file under
`build_config` (renamed from `target`) directory remains
(`build_config/${MRUBY_CONFIG}.rb` is used if the path specified
in `MRUBY_CONFIG` doesn't exist).
Files under `test/t` and `mrbgem/*/test` are for tests, not for actual
execution. So symbols in those files need not to be pre-allocated. This
change slightly reduce the memory consumption.
This reverts commit 8746a6fe4e7bda8a0fbc0eaece9314ec51a0c255.
We already have `mrb_protect()`, `mrb_ensure()` and `mrb_rescue()`
functions. If you need to handle exceptions from C functions, use those
functions above.
`mrb_exc_protect()` takes two C functions, `body` to be executed first,
and `resc` to be executed when an error happens during `body` execution.
Since `mrb_exc_protect()` should be compiled with the proper compiler,
we will not see the problem like #5088 that was caused by `setjmp()` and
`throw` mixture.
`MRB_TRY()` does not work when compiled by C compiler with `cxx_exception`,
due to the mixture of `setjmp()` used by `mirb.c` and `throw` used by
the core. The original intension of e2e6554b56 is to protect code from
signal interruption, but the signal interruption is not well-defined in
mruby anyway.
- `mrb_check_intern()` to return `mrb_value`
- `mrb_intern_check()` to return `mrb_sym` [NEW]
Other new functions:
- `mrb_intern_check_cstr()`
- `mrb_intern_check_str()`
Rename new functions:
- `mrb_convert_type(mrb,val,type,tname,method)`
=> `mrb_type_convert(mrb,val,type,tname,method)`
- `mrb_check_convert_type(mrb,val,type,tname,method)`
=> `mrb_type_convert_check(mrb,val,type,tname,method)`
Old names are defined by macros (support `tname` drop and
`char*` => `mrb_sym` conversion).
Linux `atan2(3)` man page says:
```
If y is positive infinity (negative infinity) and x is positive infinity,
+pi/4 (-pi/4) is re‐ turned.
```
But on Microsoft VC/MinGW, `atan2()` returns `NaN` if either of
arguments is infinite. So we skip those tests on the platforms.
In the past code, the current `callinfo (ci)` was modified, thus it was
possible to pop `ci` beyond the `cibase`, that could cause out of memory
bound access for the code like the following:
```ruby
def m2
lambda {
Proc.new {
return :return # return from the method
}
}.call.call
:never_reached
end
p m2
```
By definition `mrb_assert()` called only when `MRB_DEBUG` is defined too.
But make I wanted to make clear that the local variable `current_checkpoint_tag`
is only accessed when `MRB_DEBUG` is set by wrapping with `DEBUG_ONLY_EXPR()`.
To make packed inline symbols within 31 bits, because the new method
hash tables allows only 31 bits of symbols. They use top 1 bit to maek
unused slots.
They are basically the copy of instance variable tables. On my Linux
box, memory consumption of `mrbtest` measured by `valgrind` is:
- old: 17,683,830 bytes
- new: 14,283,749 bytes
- Integrate `Fixnum` and `Integer`
- Remove `Integral`
- `int / int -> int`
- Replace `mrb_fixnum()` to `mrb_int()`
- Replace `mrb_fixnum_value()` to `mrb_int_value()`.
- Use `mrb_integer_p()` instead of `mrb_fixnum_p()`
They used to return `mrb_value` but now return `mrb_sym` for consistency
with other `intern` functions. If symbols are not defined, `check`
functions return `0`, instead of `nil` in the past.
It causes API incompatibility but I believe few people use those
functions out of the core, and those changes are very easy to handle,
hopefully.
Trailing `.0` is removed from `Float#to_s` and `Float#inspect` at
9d08025b. However, I think the more human-readable format is better
for `Float#inspect`.
For example, in the `Float#to_s` format, the failure message is not
well understood when testing values including types by `eql?` (e.g.
`Numeric#step` test).
```ruby
assert "example" do
exp = 1.0
act = 1
assert_operator(exp, :eql?, act) #=> Expected 1 to be eql? 1.
end
```
* The `Fixnum` constant is now an alias for the `Integer` class.
* Remove `struct mrb_state::fixnum_class` member.
If necessary, use `struct mrb_state::integer_class` instead.
- `MRB_WITHOUT_FLOAT` => `MRB_NO_FLOAT`
- `MRB_USE_FLOAT` => `MRB_USE_FLOAT32`
The former is to use `USE_XXX` naming convention. The latter is to make
sure `float` is 32bit float and not floating point number in general.
On platforms where `time_t` is unsigned, negative time can be a result
of integer casting. Out-of-range error is too strict for those cases.
This fix does not address wrong time value in `MRB_WORD_BOXING`. It will
be addressed later (by introducing "big" integers).
Changes made after `setjmp()` are destroyed and need reassignment.
This problem is now caused by the addition of the `OP_JUW` instruction.
When actually building on FreeBSD 12.1 with `clang10 -fsanitize=address`, mrbtest "NameError#name [15.2.31.2.1]" is failed.
However, qualifying `pc` with `volatile` slows down significantly and increases the object code.
Suppress them by qualifying only the variables that restore `pc`.
When a global jump occurs, look at the catch handler table to determine where to jump.
In that case, `pc` already shows the following instruction, but since the table shows `begin_offset ... end_offset`, the comparison is done with `begin_offset < pc && pc <= end_offset`.
If there is a corresponding handler, move `pc` to `handler.target_offset` and continue running the VM.
When a global jump across `ensure` is made by `return`, `break`, `next`, `redo` and `retry`, the extended `RBreak` object saves and restores the C-level execution position.
This extended `RBreak` can have tag information, which makes it a pseudo coroutine (the "tag" mimics CRuby).
The implementation of pseudo coroutines by `RBreak` is summarized by `CHECKPOINT_RESTORE ... CHECKPOINT_MAIN ... CHECKPOINT_END` and `throw_tagged_break` / `unwind_ensure` macros.
The restart of processing is branched by `RBREAK_TAG_FOREACH(DISPATCH_CHECKPOINTS)`.
- Not only `rescue` blocks but also `ensure` blocks are now sandwiched between `OP_EXCEPT` and `OP_RAISEIF`.
- Remove the function `ecall()`.
It is no longer necessary to re-enter the VM to perform an "ensure block".
This will resolves#1888.
- Added instruction `OP_JUW` (Jump while UnWind).
It jumps unconditionally like `OP_JMP`, but searches the catch handler table and executes the ensure block.
Since it searches the catch handler table, it is much heavier than `OP_JMP`.
- `OP_EXCEPT` checks if `mrb->exc` is `NULL`, `MRB_TT_EXCEPTION` or
`MRB_TT_BREAK`.
If `mrb->exc` is `NULL`, it will be replaced with `nil`.
- If `OP_RAISE` is `nil`, it does nothing and the immediately
following instruction is executed (like `OP_NOP`).
Also, in case of `RBreak` object, it moves to the processing for
`break`.
With this change, the instruction name is changed from
`OP_RAISE` to `OP_RAISEIF`.
The catch handler table is combined with iseq block.
This is to prevent the structure from growing by adding a field for the
catch handler table to the `mrb_irep` structure.
"iseq block" and "catch handler table":
[number of catch handler table (2 bytes)]
[number of byte code (4 bytes)]
[iseq (any bytes)]
[catch handlers (multiple of 7 bytes)]
catch handler:
[catch type (1 byte)]
[begin offset (2 bytes)]
[end offset (2 bytes)]
[target offset (2 bytes)]
catch type: enum mrb_catch_type (0 = rescue, 1 = ensure)
begin offset: Includes the specified instruction address
end offset: Does not include the specified instruction address
target offset: replaces pc with the specified instruction address
This table is not expanded by `read_irep_record_1()`.
The necessary elements are expanded one by one when used.
When the array is very big, the simpler `mrb_write_barrier` causes
calling `gc_mark_children` for big arrays repeatedly. That would hinder
performance very badly.
Introduced `MRB_NO_METHOD_CACHE` which is inverse of `MRB_METHOD_CACHE`
that should be enabled intestinally. In addition, the default cache is
made bigger (128 -> 256).
Type `mrbc -S -B<init> -o<outfile> <rbfiles...>` to generate the C
source code that holds compiled `mrb_irep`.
Appending the following code to the bottom of the generated code,
`mruby` executes the compiled code:
```C
int
main()
{
mrb_state *mrb = mrb_open();
struct RProc *p = mrb_proc_new(mrb, &init_irep);
mrb_vm_run(mrb, p, mrb_top_self(mrb), 0);
mrb_close(mrb);
return 0;
}
```
Eventually static compile should use this representation, instead
of `uint8_t` array that holds `mrb` data, so that we can skip
interpreting `mrb` data.
Changes:
- `pool format is completely replaced
- supported types: `STR`, `INT32`, `INT64`, `FLOAT`
- `FLOAT` may be replaced by binary representation in the future
- insert `NUL` after string literals in `mrb` files
- `irep->pool` no longer store values in `mrb_value`
- instead it stores in `mrb_pool_value`
- less allocation
- `mrb_irep` can be stored in ROM
You have to specify `TARGET` to specify a configuration, e.g.
```
rake TARGET=host-debug all test
```
When you port `mruby` to a new configuration:
1. copy an existing configuration under `target` directory
2. modify the new configuration file
3. build using the new configuration
4. send PR if you please
I sometimes see Bison related problems in setting up build environments.
Therefore to remove Bison from build time dependencies, add `y.tab.c`
generated by Bison to the repository.
The reduction of dependency at build time also reduces the labor and time
for setup and installation in CI.
In addition, a path in `#line` directive is converted to a relative path so
that its path is constant regardless of development environments.
- `mrb_convert_type`
- `mrb_check_convert_type`
Those function no longer take `tname` string representation of desired
type, and take method symbols instead of `const char*` names. This is
incompatible change. I hope no third-party gems use those functions.
Where `QSYM` means quoted symbols, which cannot be represented C
symbols, so specify aliases instead.
- operators: name of the operation, e.g. add for `+`
- predicates: add `_p` suffix instead of `?`
- bang methods: add `_b` suffix instead of `!`
- instance variables: add `a_` prefix instead of `@`
- global variables: add `d_` prefix instead of `@`
- class variables: unsupported; don't use them
You have to specify `TARGET` to specify a configuration, e.g.
```
rake TARGET=host-debug all test
```
When you port `mruby` to a new configuration:
1. copy an existing configuration under `target` directory
2. modify the new configuration file
3. build using the new configuration
4. send PR if you please
Fixed inproper argument of mrb_fixnum_value() called in integral_div(),
when built with MRB_WITHOUT_FLOAT flag.
Co-authored-by: taiyoslime <t@iyosli.me>
Co-authored-by: n4o847 <22975590+n4o847@users.noreply.github.com>
- The `s` specifier is a string pointer obtained without performing `mrb_str_modify()`, so it cannot be changed.
- The `z` specifier cannot be changed because it is a string pointer obtained by `RSTRING_CSTR()` which returns `const char *`.
Before this commit:
mruby -e 'CRubyExtension.ruby_method' # => A Ruby Extension
mruby -e 'CRubyExtension.c_method' # => A C Extension
mruby -e 'CExtension.c_method' # => A C Extension
mruby -e 'RubyExtension.ruby_method' # => A Ruby Extension
After this commit:
mruby -e 'CRubyExtension.ruby_method' # => CRubyExtension: A Ruby Extension
mruby -e 'CRubyExtension.c_method' # => CRubyExtension: A C Extension
mruby -e 'CExtension.c_method' # => CExtension: A C Extension
mruby -e 'RubyExtension.ruby_method' # => RubyExtension: A Ruby Extension
Before this commit:
$ rake
...
$ rm build/host/src/gc.o
$ rake
rake aborted!
Don't know how to build task '/home/foo/mruby/build/host/src/gc.o' (See the list of available tasks with `rake --tasks`)
...
After this commit:
$ rake
...
$ rm build/host/src/gc.o
$ rake
CC src/gc.c -> build/host/src/gc.o
...
With gcc 9.3.0 on Ubuntu 20.04, build/host/src/array.d looks like:
/build/host/src/array.o: /src/array.c \
/include/mruby.h /include/mrbconf.h \
/include/mruby/common.h \
...
/include/mruby/range.h \
/src/value_array.h
and it has been parsed to:
[
# /src/array.c missing
"/include/mruby.h /include/mrbconf.h", # incorrectly parsed
"/include/mruby/common.h",
...
"/src/value_array.h",
]
After this change, *.d will be parsed correctly.
[
"/src/array.c",
"/include/mruby.h",
"/include/mrbconf.h",
"/include/mruby/common.h",
...
"/src/value_array.h",
]
Adding MRB_MINGW32_LEGACY in common.h in order to identify the legacy MinGW environment (i.e. NOT to be confused with MinGW-w64).
For more info about MinGW defined macros, see: https://sourceforge.net/p/predef/wiki/Compilers/
* `_Static_assert` can also be used with `-std=gnu99` on GCC >= 4.6.
* `static_assert` can be used on MSVC.
* `static_assert` can be used even on old G++/Clang++ if
`__GXX_EXPERIMENTAL_CXX0X__` is defined.
- Use POSIX format instead of GNU extension for `time` command.
For example FreeBSD's `time(1)` does not have GNU extensions available.
- Sort `benchmark/bm_*.rb`.
This is because the order of the bar graph cannot be uniquely
determined depending on the result of `Dir.glob`.
The function corresponding to `ht_hash_func()` was as follows in the days of
khash implementation (before d78acc7a).
```c
mrb_hash_ht_hash_func(mrb_state *mrb, mrb_value key)
{
enum mrb_vtype t = mrb_type(key);
...
switch (t) {
...
default:
hv = mrb_funcall(mrb, key, "hash", 0);
h = (khint_t)t ^ (khint_t)mrb_fixnum(hv);
break;
}
...
}
```
When switched to the segmented list implementation (d78acc7a), this function
was changed as follows.
```c
sg_hash_func(mrb_state *mrb, seglist *t, mrb_value key)
{
enum mrb_vtype tt = mrb_type(key);
...
switch (tt) {
...
default:
hv = mrb_funcall(mrb, key, "hash", 0);
h = (size_t)t ^ (size_t)mrb_fixnum(hv);
break;
}
...
}
```
Since the argument `t` was added, the variable for type tag was changed from
`t` to `tt`, but the variable used in the expression of `h` remained `t`.
Probably this is an omission of change, so fixed it.
The function corresponding to `ht_hash_func()` was as follows in the days of
khash implementation (before d78acc7a).
```c
mrb_hash_ht_hash_func(mrb_state *mrb, mrb_value key)
{
enum mrb_vtype t = mrb_type(key);
...
switch (t) {
...
default:
hv = mrb_funcall(mrb, key, "hash", 0);
h = (khint_t)t ^ (khint_t)mrb_fixnum(hv);
break;
}
...
}
```
When switched to the segmented list implementation (d78acc7a), this function
was changed as follows.
```c
sg_hash_func(mrb_state *mrb, seglist *t, mrb_value key)
{
enum mrb_vtype tt = mrb_type(key);
...
switch (tt) {
...
default:
hv = mrb_funcall(mrb, key, "hash", 0);
h = (size_t)t ^ (size_t)mrb_fixnum(hv);
break;
}
...
}
```
Since the argument `t` was added, the variable for type tag was changed from
`t` to `tt`, but the variable used in the expression of `h` remained `t`.
Probably this is an omission of change, so fixed it.
And, in the calculation of the instance variable size, the fraction was
always rounded down because of division of integers, so fix it.
At the same time, test items that are no longer passed due to this
change are deleted.
The responsibility moved to caller to avoid confusion. Currently the
function is called from only 2 places, so it is relatively easy to
ensure not to update `gray_list` in the caller. But the assumption
may change in the future.
When the array is very big, the simpler `mrb_write_barrier` causes
calling `gc_mark_children` for big arrays repeatedly. That would hinder
performance very badly.
The `${build_dir}/mrbgems/gem_init.c` generation condition has been changed
at #5010, but it is somewhat insufficient, for example, there was a problem
with `rake && rake test`, which was also regenerated in `rake test`.
The POSIX `realloc` keep the original pointer untouched, so it can
easily leads to memory leakage. `mrb_realloc()` should handle those
bookkeeping, while `mrb_realloc_simple()` keeps the original `realloc`
behavior.
`mrb_get_arg1()` raises `ArgumentError` if the method does not receive one
argument.
And replaces all `mrb_get_args(mrb, "o", &arg)` by the new function.
Check if the constant GEMNAME is defined before use in `assert.rb`.
This is added to prevent an undefined constant error when using
`assert.rb` in other environments - for example, testing CRuby.
commit 2d7d545c4c4bfce7fdcbcbe9baaeb437915742f0
Merge: 625a1249 b178914b
Author: Yukihiro "Matz" Matsumoto <matz@ruby.or.jp>
Date: Fri Jun 5 14:35:13 2020 +0900
Merge branch 'fix-mrb_open-with-nomem' of https://github.com/dearblue/mruby into dearblue-fix-mrb_open-with-nomem
commit b178914b11
Author: dearblue <dearblue@users.noreply.github.com>
Date: Sat Jan 19 22:22:44 2019 +0900
Fix invalid pointer free inside other heap's block
1. `e = mrb_obj_alloc(...)`
2. `e->stack = mrb->c->stack` (`mrb->c->stack` is anywhere in the range `stbase...stend`)
3. And raised exception by `mrb_malloc()`!
4. `mrb_free(e->stack)` by GC part (wrong free)
commit 52e3d5d858
Author: dearblue <dearblue@users.noreply.github.com>
Date: Sat Jan 19 21:55:36 2019 +0900
Fix memory leak for temporary symbols when out of memory
commit 4c5499b88e
Author: dearblue <dearblue@users.noreply.github.com>
Date: Sun Jan 20 11:42:07 2019 +0900
Fix uninitialized pointer dereference for debug section
commit 8e993167de
Author: dearblue <dearblue@users.noreply.github.com>
Date: Sun Jan 20 11:41:09 2019 +0900
Fix memory leak for temporary filenames when out of memory
commit 8b422577e6
Author: dearblue <dearblue@users.noreply.github.com>
Date: Sun Jan 20 10:57:51 2019 +0900
Fix memory leak for irep when out of memory
commit 6b35ebf49a
Author: dearblue <dearblue@users.noreply.github.com>
Date: Sun Jan 20 10:55:50 2019 +0900
Fix uninitialized pointer dereference when do not finished initializing irep
commit 2531f2631e
Author: dearblue <dearblue@users.noreply.github.com>
Date: Sun Jan 20 10:48:15 2019 +0900
Fix NULL pointer dereference when do not finished initializing irep
commit e2d6896eba
Author: dearblue <dearblue@users.noreply.github.com>
Date: Sat Jan 19 12:54:19 2019 +0900
Fix memory leak for irep when out of memory by `mrb_proc_new()`
commit b6214ff8a0
Author: dearblue <dearblue@users.noreply.github.com>
Date: Sat Jan 19 12:53:07 2019 +0900
Fix memory leak for `khash_t` in `kh_init_size()` when out of memory by `kh_alloc()`
commit 19162dd6c1
Author: dearblue <dearblue@users.noreply.github.com>
Date: Sun Jan 20 02:15:07 2019 +0900
Fix memory leak for symbol string when out of memory in `kh_put()`
commit 15e67297ff
Author: dearblue <dearblue@users.noreply.github.com>
Date: Sun Jan 20 02:12:24 2019 +0900
Fix keep wrong symbol index when out of memory
commit 3f8e2b3752
Author: dearblue <dearblue@users.noreply.github.com>
Date: Sun Jan 20 02:08:13 2019 +0900
Fix keep wrong symbol capacity when out of memory
commit a3cfe755ab
Author: dearblue <dearblue@users.noreply.github.com>
Date: Sat Jan 19 10:11:37 2019 +0900
Fix NULL pointer dereference `mrb->c` by `mark_context()`
commit d9c7b6be6e
Author: dearblue <dearblue@users.noreply.github.com>
Date: Sun Jan 20 15:25:09 2019 +0900
Fix protect exception for print error message
commit 100642750e
Author: dearblue <dearblue@users.noreply.github.com>
Date: Sun Jan 20 11:59:02 2019 +0900
Protect exception for mruby core initialization
commit 7a0418304e
Author: dearblue <dearblue@users.noreply.github.com>
Date: Fri Jan 18 20:38:27 2019 +0900
Fix memory leak for string object when out of memory
The `mrb_str_pool()` function has a path to call `malloc()` twice.
If occurs `NoMemoryError` exception in second `malloc()`,
first `malloc()` pointer is not freed.
commit fef1c152ce
Author: dearblue <dearblue@users.noreply.github.com>
Date: Sat Jan 19 13:05:09 2019 +0900
Fix stack overflow when out of memory
As a result of this change, no backtrace information is set
for NoMemoryError (`mrb->nomem_err`).
Detailes:
When generating a backtrace, called `mrb_intern_lit()`,
`mrb_str_new_cstr()` and `mrb_obj_iv_set()` function with
`exc_debug_info()` function in `src/error.c`.
If a `NoMemoryError` exception occurs at this time,
the `exc_debug_info()` function will be called again,
and in the same way `NoMemoryError` exception raised will result
in an infinite loop to occurs stack overflow (and SIGSEGV).
commit da7d7f881b
Author: dearblue <dearblue@users.noreply.github.com>
Date: Sun Jan 20 12:00:38 2019 +0900
Fix NULL pointer dereference `mrb->nomem_err` when not initialized
Add internal functions (not `static`):
* `mrb_raise_nomemory()`
* `mrb_core_init_abort()`
- It can now deal with operands in the range of `OP_EXT*`.
- It can now call the same method as the variable name without arguments.
```ruby
def a
"Safe!"
end
a = "Auto!"
eval "a()" # call method `a`
```
We made it `64` which should be big enough:
- Format modifier: 4 characters max
- Maximum width: 19 digits max
- Period between width and precision: 1 character
- Maximum precision: 19 digits max
- Format specifier: 1 character
- NUL terminator: 1 byte
- Total: 45 < 64
I sometimes see Bison related problems in setting up build environments.
Therefore to remove Bison from build time dependencies, add `y.tab.c`
generated by Bison to the repository.
The reduction of dependency at build time also reduces the labor and time
for setup and installation in CI.
In addition, a path in `#line` directive is converted to a relative path so
that its path is constant regardless of development environments.
Formerly, `__send__(*args)` modified `args` with `Array#shift`.
This bug affects optcarrot.
This changeset avoids the array destruction by using
`args = args[1, len-1]`.
The following error occurs when using `rake -m` on AppVeyor:
```
fatal error C1041: cannot open program database 'C:\projects\mruby\vc140.pdb'; if multiple CL.EXE write to the same .PDB file, please use /FS
```
Therefore, the issue is solved by not creating the PDB file. It is expected
to be somewhat faster by not generating debugging information (I don't think
debugging information is necessary for normal CI).
### Example
```ruby
# example.rb
class A
def eql?(o)
p self.class
super
end
def hash
1
end
end
class B < A; end
h = {A.new => 1}
h[B.new]
```
#### Before this patch:
```console
$ bin/mruby example.rb
A
```
#### After this patch (same as Ruby)
```console
$ bin/mruby example.rb
B
```
It was making a negative integer if the highest-order bit of a 16-bit
integer was 1.
no patched:
```ruby
p 0x7fff # => 32767
p 0x8000 # => -32768
p 0xffff # => -1
p 0x10000 # => 65536
```
The behavior is different from CRuby, but we believe this is a right
behavior for mruby, which only supports either ASCII or UTF-8
exclusively; fix#4983, ref #4982
```
$ printf '\xe3\x81' | ruby -e 'p STDIN.readchar'
"\xE3\x81"
```
```
$ printf '\xe3\x81' | mruby -e 'p STDIN.readchar'
"\xE3"
```
Which loads 16bit integer to the register. The instruction number should
be reorder on massive instruction refactoring. The instruction is added
for `mruby/c` which had performance issue with `OP_EXT`. With this
instruction, `mruby/c` VM can just raise errors on `OP_EXT` extension
instructions.
`mrb_run` requires to push callinfo stack before calling, which is very
hard from outside of `vm.c`. So there should be virtually no correct
usage of the function, hence the cause of #4488. We removed it.
You can use `mrb_top_run(mrb, proc, self, 0)` instead of
`mrb_run(mrb, proc self)`.
When using `mrb_any_to_s()` for debugging purposes, giving an object
whose class is `NULL` no longer causes a SIGSEGV and no crash.
This is achieved by making `mrb_class_name()` and `mrb_str_cat_cstr()`
null safe.
The behavior is different from CRuby, but we believe this is a right
behavior for mruby, which only supports either ASCII or UTF-8
exclusively; fix#4983, ref #4982
```
$ printf '\xe3\x81' | ruby -e 'p STDIN.readchar'
"\xE3\x81"
```
```
$ printf '\xe3\x81' | mruby -e 'p STDIN.readchar'
"\xE3"
```
Recent `bison` warns for `%pure-parser`. We kept it since MacOS only
provide ancient `bison`, but the warning is noisy and there's no hope
that Apple will upgrade `bison`. MacOS users must install the newer
version of `bison`, by typing `brew install bison` for example.
Note that `brew` does not overwrite the `bison` execution path
automatically, so you need to update your `.bash_profile` as instructed
by `brew`.
The comment is for messages when `MRB_INT16` and `MRB_WITHOUT_FLOAT` are
specified at the same time.
The comment itself is no longer needed now that `MRB_INT16` is gone.
When removing elements from an array, it is possible to avoid creating
an empty array.
Before this patch:
```c
mrb_ary_splice(mrb, ary, head, len, mrb_ary_new(mrb));
```
After this patch:
```c
mrb_ary_splice(mrb, ary, head, len, mrb_undef_value());
```
The difference between `mrb_singleton_class` and `mrb_singleton_class_ptr`:
- `mrb_singleton_class_ptr` returns `struct RClass*`.
- `mrb_singleton_class_ptr` returns `NULL` on immediate values where
`mrb_singleton_class` raises exceptions.
Previously, if `MRB_NAN_BOXING` is defined, for example,
`mrb_cptr_value()` could not keep an odd address.
If it is `MRB_32BIT`, it can be embedded in `NaN` as it is.
If it is `MRB_64BIT`, some operations are shared with `MRB_WORD_BOXING`.
In this case, the MRB_API function `mrb_nan_boxing_cptr_value()` is
defined.
They behave similar to their POSIX equivalents, except
mrb_malloc_simple() is used for memory allocation and
errno might not be set since ISO C99 doesn't have ENOMEM.
mruby is an open-source project which is looking forward to each contribution.
Contributors agree to license their contribution(s) under MIT license.
## Your Pull Request
@@ -9,7 +10,7 @@ things in mind before submitting your pull request:
* Work on the latest possible state of **mruby/master**
* Create a branch which is dedicated to your change
* Test your changes before creating a pull request (```rake test```)
* Test your changes before creating a pull request (`rake test`)
* If possible write a test case which confirms your change
* Don't mix several features or bug-fixes in one pull request
* Create a meaningful commit message
@@ -17,6 +18,41 @@ things in mind before submitting your pull request:
* Use mrbgem to provide non ISO features (classes, modules and methods) unless
you have a special reason to implement them in the core
## pre-commit
A framework for managing and maintaining multi-language `pre-commit` hooks.
`pre-commit` can be [installed](https://pre-commit.com/#installation) with `pip`, `curl`, `brew` or `conda`.
You need to first install `pre-commit` and then install the `pre-commit` hooks with `pre-commit install`.
Now `pre-commit` will run automatically on git commit!
It's usually a good idea to run the hooks against all the files when adding new hooks (usually `pre-commit` will only run on the changed files during git hooks).
Use `pre-commit run --all-files` to check all files.
To run a single hook use `pre-commit run --all-files <hook_id>`
@@ -6,7 +9,7 @@ mruby is the lightweight implementation of the Ruby language complying to (part
of) the [ISO standard][ISO-standard]. Its syntax is Ruby 2.x compatible.
mruby can be linked and embedded within your application. We provide the
interpreter program "mruby" and the interactive mruby shell "mirb" as examples.
interpreter program "mruby", and the interactive mruby shell "mirb" as examples.
You can also compile Ruby programs into compiled byte code using the mruby
compiler "mrbc". All those tools reside in the "bin" directory. "mrbc" is
also able to generate compiled byte code in a C source file, see the "mrbtest"
@@ -17,34 +20,30 @@ of the Ministry of Economy, Trade and Industry of Japan.
## How to get mruby
The stable version 2.1.1 of mruby can be downloaded via the following URL: [https://github.com/mruby/mruby/archive/2.1.1.zip](https://github.com/mruby/mruby/archive/2.1.1.zip)
The stable version 3.1.0 of mruby can be downloaded via the following URL: [https://github.com/mruby/mruby/archive/3.1.0.zip](https://github.com/mruby/mruby/archive/3.1.0.zip)
The latest development version of mruby can be downloaded via the following URL: [https://github.com/mruby/mruby/zipball/master](https://github.com/mruby/mruby/zipball/master)
The trunk of the mruby source tree can be checked out with the
following command:
$ git clone https://github.com/mruby/mruby.git
```
$ git clone https://github.com/mruby/mruby.git
```
You can also install and compile mruby using [ruby-install](https://github.com/postmodern/ruby-install), [ruby-build](https://github.com/rbenv/ruby-build) or [rvm](https://github.com/rvm/rvm).
## mruby home-page
The URL of the mruby home-page is: https://mruby.org.
The URL of the mruby home-page is: <https://mruby.org>.
## Mailing list
We don't have a mailing list, but you can use [GitHub issues](https://github.com/mruby/mruby/issues).
## How to compile and install (mruby and gems)
## How to compile, test, and install (mruby and gems)
See the [compile.md](https://github.com/mruby/mruby/blob/master/doc/guides/compile.md) file.
## Running Tests
To run the tests, execute the following from the project's root directory.
$ rake test
See the [compile.md](doc/guides/compile.md) file.
## Building documentation
@@ -52,23 +51,27 @@ There are two sets of documentation in mruby: the mruby API (generated by yard)
To build both of them, simply go
rake doc
```
rake doc
```
You can also view them in your browser
rake view_api
rake view_capi
```
rake view_api
rake view_capi
```
## How to customize mruby (mrbgems)
mruby contains a package manager called *mrbgems*. To create extensions
in C and/or Ruby you should create a *GEM*. For a documentation of how to
use mrbgems consult the file [mrbgems.md](https://github.com/mruby/mruby/blob/master/doc/guides/mrbgems.md).
use mrbgems consult the file [mrbgems.md](doc/guides/mrbgems.md).
For example code of how to use mrbgems look into the folder *examples/mrbgems/*.
## License
mruby is released under the [MIT License](https://github.com/mruby/mruby/blob/master/LICENSE).
mruby is released under the [MIT License](LICENSE).
## Note for License
@@ -82,7 +85,7 @@ In the future, mruby might ask you to distribute your new code
(that you will commit,) under the MIT License as a member of
"mruby developers" but contributors will keep their copyright.
(We did not intend for contributors to transfer or waive their copyrights,
Actual copyright holder name (contributors) will be listed in the AUTHORS
actual copyright holder name (contributors) will be listed in the [AUTHORS](AUTHORS)
file.)
Please ask us if you want to distribute your code under another license.
@@ -90,12 +93,10 @@ Please ask us if you want to distribute your code under another license.
## How to Contribute
See the [contribution guidelines][contribution-guidelines], and then send a pull
request to <http://github.com/mruby/mruby>. We consider you have granted
non-exclusive right to your contributed code under MIT license. If you want to
be named as one of mruby developers, please include an update to the AUTHORS
file in your pull request.
request to <https://github.com/mruby/mruby>. We consider you have granted
non-exclusive right to your contributed code under MIT license.
The mruby objects and data are represented by C data type `mrb_value`. There are three options how to pack the data values in the `mrb_value`.
* Word Boxing
* NaN Boxing
* No Boxing
## Word Boxing
Word boxing packs the Ruby data in a word, which is a natural integer size that equals to the size of pointers (`intptr_t`). Word boxing can be specified by `MRB_WORD_BOXING`, and it's default configuration for most platforms.
Some values (called immediate values, e.g. integers, booleans, symbols, etc.) are directly packed in the word. The other data types are represented by pointers to the heap allocated structures.
The Word boxing packing bit patterns are like following:
| Types | Bit Pattern |
|--------|-------------------------------------|
| object | xxxxxxxx xxxxxxxx xxxxxxxx xxxxx000 |
| fixnum | xxxxxxxx xxxxxxxx xxxxxxxx xxxxxxx1 |
| nil | 00000000 00000000 00000000 00000000 |
| true | 00000000 00000000 00000000 00001100 |
| false | 00000000 00000000 00000000 00000100 |
| undef | 00000000 00000000 00000000 00010100 |
| symbol | xxxxxxxx xxxxxxxx xxxxxxxx xxxxxx10 |
On 64 bit platform (unless `MRB_WORDBOX_NO_FLOAT_TRUNCATE`), float values are also packed in the `mrb_value`. In that case, we drop least significant 2 bits from mantissa.
If you need full precision for floating point numbers, define `MRB_WORDBOX_NO_FLOAT_TRUNCATE`.
## NaN Boxing
NaN boxing packs the Ruby data in a floating point numbers, which represent NaN (Not a Number) values. Under IEEE753 definitions every value that exponent is all set are considered as NaN. That means NaN can represent `2^51` values. NaN boxing is a teaching to pack the values in those NaN representation. In theory, 64 bits pointers are too big to fit in NaN, but practically most OS uses only 48 bits at most for pointers (except for some OS e.g. Solaris).
The NaN boxing packing bit patterns are like following:
The object values appear far more frequently than floating point numbers, so we offset the value so that object pointers are unchanged. This technique is called "favor pointer"".
## No Boxing
No boxing represents `mrb_value` by the C struct with `type` and the value union. This is the most portable (but inefficient) representation. No boxing can be specified by `MRB_NO_BOXING`, and it's default for debugging configuration (e.g. `host-debug`).
* `MRB_USE_FLOAT32` is changed from `MRB_USE_FLOAT` to make sure `float` here
means using single precision float, and not the opposite of `MRB_NO_FLOAT`.
* `MRB_USE_METHOD_T_STRUCT` uses `struct` version of `mrb_method_t`. More
portable but consumes more memory. Turned on by default on 32bit platforms.
* `MRB_` prefix is added to those without.
## `MRB_NO_BOXING`
Uses `struct` to represent `mrb_value`. Consumes more memory
but easier to investigate the internal and to debug. It used
to be default `mrb_value` representation. Now the default is
`MRB_WORD_BOXING`.
## `MRB_WORD_BOXING`
Pack `mrb_value` in an `intptr_t` integer. Consumes less
memory compared to `MRB_NO_BOXING` especially on 32 bit
platforms. `Fixnum` size is 31 bits so some integer values
does not fit in `Fixnum` integers.
## `MRB_NAN_BOXING`
Pack `mrb_value` in a floating pointer number. Nothing
changed from previous versions.
## `MRB_USE_MALLOC_TRIM`
Call `malloc_trim(0)` from mrb_full_gc() if this macro is defined.
If you are using glibc malloc, this macro could reduce memory consumption.
# Command Line Program
## `bin/mruby` (by mrbgems/mruby-bin-mruby)
The mruby3 now automatically detects `*.mrb` files without the `-b`
switch. Therefore, it can be mixed with the `*.rb` file in combination
with the `-r` switch and specified at the same time.
Here's an example that works fine:
```console
$ bin/mruby app.mrb
$ bin/mruby -r lib1.mrb -r lib2.rb app.rb
$ bin/mruby -r lib1.rb -r lib2.rb < app.mrb
```
# Internal Changes
## New Instructions
`mruby3` introduces a few new instructions.
Instructions that access pool[i]/syms[i] where i>255.
* `OP_LOADL16`
* `OP_STRING16`
* `OP_LOADSYM16`
Instructions that load a 32 bit integer.
* `OP_LOADI32`
Instruction that unwinds jump table for rescue/ensure.
* `OP_JMPUW`
Renamed from `OP_RAISE`
* `OP_RAISEIF`
Instruction that is reserved for the future keyword argument support.
* OP_SENDVK
## Removed Instructions
Instructions for old exception handling
* `OP_ONERR`
* `OP_POPERR`
* `OP_EPUSH`
* `OP_EPOP`
No more operand extension
* `OP_EXT1`
* `OP_EXT2`
* `OP_EXT3`
## Changed Instructions
Jump addresses used to be specified by absolute offset from the start of `iseq`. Now they are relative offset from the address of the next instruction.
## `Random` now use `xoshiro128++`.
For better and faster random number generation.
## Preallocated Symbol
Preallocated symbols are interned at compile-time. They can be accessed via symbols macros (e.g. `MRB_SYM()`).
See [Symbols](https://github.com/mruby/mruby/blob/master/doc/guides/symbol.md).
- Replace `R-assignment` by `single-line pattern matching` [Ruby:Feature#15921](https://bugs.ruby-lang.org/issues/15921)
- Support squiggly heredocs. [#5246](https://github.com/mruby/mruby/pull/5246)
- Hash value omission [Ruby:Feature#14579](https://bugs.ruby-lang.org/issues/14579)
## Configuration Options Changed
Some configuration macros are available:
- `MRB_WORDBOX_NO_FLOAT_TRUNCATE`: by default, float values are packed in the word if possible, but define this macro to allocate float values in the heap.
- `MRB_USE_RO_DATA_P_ETEXT`: define this macro if `_etext` is available on your platform.
- `MRB_NO_DEFAULT_RO_DATA_P`: define this macro to avoid using predefined `mrb_ro_data_p()` function
---
# Updated Features
## New build configurations
We have added several new build configurations in the `build_config` directory.
- `cross-mingw-winetest.rb`
- `cross-mingw.rb`
- `nintendo_switch.rb`
- `serenity.rb`
- `minimal`: minimal configuration
- `host-f32`: compiles with `mrb_float` as 32 bit `float`
- `host-nofloat`: compiles with no float configuration
- `android_arm64_v8a.rb`: renamed from `android_arm64-v8a.rb`
## Core Libraries
### New Methods
- `Array#product`
- `Array#repeated_combination`
- `Array#repeated_permutation`
- `Kernel#__ENCODING__`
- `Random.bytes`
- `Random#bytes`
- `String#center`
### New Gem Enhancement
- `mrbgems/mruby-pack` now supports `M` directive (Q encoding)
- `mrbgems/mruby-pack` now supports `X` directive (back-up by bytes)
- `mrbgems/mruby-pack` now supports `@` directive (absolute position)
- `mrbgems/mruby-pack` now supports `w` directive (BER compression)
## Tools
- `mruby-config` now supports `--cc` and `--ld` options.
- Remove `OP_` prefix from `mruby -v` code dump output.
- Prohibit use of `OP_EXT{1,2,3}` by `mrbc` with `--no-ext-ops` option.
## Features for mruby Developer
- Add new specifier `c` to `mrb_get_args()` for receive Class/Module.
---
# Breaking Changes
## Incompatibly Changed Methods
- `Kernel#printf` (`mruby-sprintf`) Format specifiers `%a` and `%A` are removed.
- `Kernel#puts` (`mruby-print`) Now expand Array arguments.
## mruby VM and bytecode
Due to improvements in the binary format, mruby binaries are no longer backward compatible.
To run the mruby binaries on mruby 3.1, recompile with the mruby 3.1 `mrbc`.
- Upgrade mruby VM version `RITE_VM_VER` to `0300` (means mruby 3.0 or after).
- Upgrade mruby binary version `RITE_BINARY_FORMAT_VER` to `0300`.
## Reintroduced Instructions
`mruby3.0` removed `OP_EXT1`, `OP_EXT2`, `OP_EXT3` for operand extension. But the operand size limitations was too tight for real-world application.
`mruby3.1` reintroduces those extension instructions.
## Removed Instructions
`mruby3.1` removed following instructions.
- `OP_LOADL16`
- `OP_LOADSYM16`
- `OP_STRING16`
- `OP_LAMBDA16`
- `OP_BLOCK16`
- `OP_METHOD16`
- `OP_EXEC16`
Those instructions are no longer needed by reintroduction of extension instructions.
- `OP_SENDV`
- `OP_SENDVB`
Those instructions for method calls with variable number of arguments are no longer needed. They are covered by `OP_SEND` instruction with `n=15`.
Execute `obj[int]` and `obj[int] = value` respectively, where `obj` is `string|array|hash`.
### `OP_SSEND` and `OP_SSENDB`
They are similar to `OP_SEND` and `OP_SENDB` respectively. They initialize the `R[a]` by `self` first so that we can skip one `OP_LOADSELF` instruction for each call.
### `OP_SYMBOL`
Extracts the character string placed in the pool as a symbol.
## Changed Instructions
### `OP_SEND` and `OP_SENDB`
Method calling instructions are unified. Now `OP_SEND` and `OP_SENDB` (method call with a block) can support both splat arguments and keyword arguments as well.
Operand C specifies the number of arguments. Lower 4 bits (`n`) represents the number of ordinal arguments, and higher 4 bits (`nk`) represents the number of keyword arguments.
When `n == 15`, the method takes arguments packed in an array. When `nk == 15`, the method takes keyword arguments are packed in a hash.
### `OP_ARYPUSH`
Now takes 2 operands and pushes multiple entries to an array.
## Boxing Updated
### Word Boxing
`MRB_WORD_BOXING` now packs floating point numbers in the word, if the size of `mrb_float` is equal or smaller than the size of `mrb_int` by default.
If the size of `mrb_float` and `mrb_int` are same, the last 2 bits in the `mrb_float` are trimmed and used as flags. If you need full precision, you need to define `MRB_WORDBOX_NO_FLOAT_TRUNCATE` as described above.
### NaN Boxing
Previous NaN boxing packs values in NaN representation, but pointer retrievals are far more frequent than floating point number references. So we add constant offset to NaN representation to clear higher bits of pointer representation. This representation is called "Favor Pointer" NaN Boxing.
Also, previous NaN boxing limit the size of `mrb_int` to 4 bytes (32 bits) to fit in NaN values. Now we allocate integer values in the heap, if the value does not fit in the 32 bit range, just like we did in Word Boxing.
## Constant Folding
The code generator was updated to reduce the number of instructions, e.g.
```
a = 2 * 5
```
will be interpreted as
```
a = 10
```
In addition, we have improved peephole optimizations, for example:
```
GETIV R4 :@foo
MOVE R1 R4
```
to
```
GETIV R1 :@foo
```
## `String#hash` now use `FNV1a` algorithm
For better and faster hash values.
---
# Major bug fixes
- Fix infinite recursive call bugs in integer division [98799aa6](https://github.com/mruby/mruby/commit/98799aa6)
- Fix to raise TypeError with super inside instance_eval / class_eval [#5476](https://github.com/mruby/mruby/pull/5476)
- Fix to call `method_added` hooks on method definitions; [#2339](https://github.com/mruby/mruby/pull/2339)
- Fix a potential buffer overflow in `time_zonename` [26340a88](https://github.com/mruby/mruby/commit/26340a88)
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
mimaki