Commit Graph

288 Commits

Author SHA1 Message Date
Yukihiro "Matz" Matsumoto 7e7f1b2f1d parse.y: numbered parameters should not appear on toplevel.
Toplevel includes the top of the method/class/module definitions.
2022-01-11 14:52:05 +09:00
Yukihiro "Matz" Matsumoto 3a30cc27da parse.y: move numbered parameter indexing to lexer. 2022-01-11 14:49:41 +09:00
Yukihiro "Matz" Matsumoto 5dfbfb3ecd parse.y: p->nvar (numbered argument index list) may be NULL. 2022-01-10 15:44:15 +09:00
Yukihiro "Matz" Matsumoto 27e57aad0b parse.y: prohibit numbered parameters as arguments; fix #5605 2021-12-27 10:55:17 +09:00
Yukihiro "Matz" Matsumoto 39191cd9fa parse.y: allow arguments start with _ to appear multiple times; fix #5604 2021-12-24 07:41:37 +09:00
Yukihiro "Matz" Matsumoto 42a6872c2b parse.y: adjust void_expr_error(); fix #5606
Allow void expression on some places e.g. right hand of `rescue`
modifier. In addition, checks added on some places, e.g. left hand of
logical operators.
2021-12-22 14:08:04 +09:00
dearblue 6b8582c95d Add bin/mrbc --no-ext-ops switch
Print an error if `OP_EXT[123]` is needed when generating mruby binary.
This may be useful for mruby/c.

Inspired by #5590.
2021-12-17 23:02:04 +09:00
Yukihiro "Matz" Matsumoto 6ac26e4442 parse.y: support anonymous block argument introduced by Ruby3.1. 2021-12-02 18:24:48 +09:00
Yukihiro "Matz" Matsumoto 01cadd3e70 parse.y: check duplicated argument names; fix #5575 2021-11-21 19:05:50 +09:00
Yukihiro "Matz" Matsumoto 2e41b20590 parse.y: unnamed rest argument should be mul (*). 2021-11-20 08:50:52 +09:00
Yukihiro "Matz" Matsumoto 9e064f2353 parse.y: skip unnecessary local_add_f(p, 0). 2021-11-20 08:50:51 +09:00
Yukihiro "Matz" Matsumoto 7c1878669a parse.y: should allow newline after .... 2021-11-12 15:02:04 +09:00
Yukihiro "Matz" Matsumoto 885a87e408 parse.y: allow ... in formal arguments without parentheses. 2021-11-10 22:25:02 +09:00
Yukihiro "Matz" Matsumoto 90484cac63 codegen.c: skip OP_LOADSELF using OP_SSEND. 2021-10-19 12:13:05 +09:00
Yukihiro "Matz" Matsumoto 0dc5fcec7b parse.y: check car part of return node. 2021-10-16 10:33:35 +09:00
Yukihiro "Matz" Matsumoto dccd66f9ef Support Ruby3.0 keyword arguments.
The Difference

Since Ruby1.9, the keyword arguments were emulated by Ruby using the hash
object at the bottom of the arguments. But we have gradually moved toward
keyword arguments separated from normal (positinal) arguments.

At the same time, we value compatibility, so that Ruby3.0 keyword
arguments are somewhat compromise. Basically, keyword arguments are
separated from positional arguments, except when the method does not
take any formal keyword arguments, given keyword arguments (packed
in the hash object) are considered as the last argument.

And we also allow non symbol keys in the keyword arguments. In that
case, those keys are just passed in the `**` hash (or raise
`ArgumentError` for unknown keys).

The Instruction Changes

We have changed `OP_SEND` instruction. `OP_SEND` instruction used to
take 3 operands, the register, the symbol, the number of (positional)
arguments. The meaning of the third operand has been changed. It is now
considered as `n|(nk<<4)`, where `n` is the number of positional
arguments, and `nk` is the number of keyword arguments, both occupies
4 bits in the operand.

The number `15` in both `n` and `nk` means variable sized arguments are
packed in the object. Positional arguments will be packed in the array,
and keyword arguments will be packed in the hash object. That means
arguments more than 14 values are always packed in the object.

Arguments information for other instructions (`OP_SENDB` and `OP_SUPER`)
are also changed. It works as the third operand of `OP_SEND`. the
difference between `OP_SEND` and `OP_SENDB` is just trivial. It assigns
`nil` to the block hidden arguments (right after arguments).

The instruction `OP_SENDV` and `OP_SENDVB` are removed. Those
instructions are replaced by `OP_SEND` and `OP_SENDB` respectively with
the `15` (variable sized) argument information.

Calling Convention

When calling a method, the stack elements shall be in the order of the
receiver of the method, positional arguments, keyword arguments and the
block argument. If the number of positional or keyword arugument (`n` or
`nk`) is zero, corresponding arguments will be empty. So when `n=0` and
`nk=0` the stack layout (from bottom to top) will be:

+-----------------------+
| recv | block (or nil) |
+-----------------------+

The last elements `block` should be explicitly filled before `OP_SEND`
or assigned to `nil` by `OP_SENDB` internally. In other words, the
following have exactly same behavior:

OP_SENDB clears `block` implicitly:

```
OP_SENDB reg sym 0
```

OP_SEND clears `block` implicitly:

```
OP_LOADNIL  R2
OP_SEND     R2 sym 0
```

When calling a method with only positional arguments (n=0..14) without
keyword arguments, the stack layout will be like following:

+--------------------------------------------+
| recv | arg1 | ... | arg_n | block (or nil) |
+--------------------------------------------+

When calling a method with arguments packed in the array (n=15) which
means argument splat (*) is used in the actual arguments, or more than
14 arguments are passed the stack layout will be like following:

+-------------------------------+
| recv | array | block (or nil) |
+-------------------------------+

The number of the actual arguments is determined by the length of the
argument array.

When keyword arguments are given (nk>0), keyword arguments are passed
between positional arguments and the block argument. For example, when
we pass one positional argument `1` and one keyword argument `a: 2`,
the stack layout will be like:

+------------------------------------+
| recv | 1 | :a | 2 | block (or nil) |
+------------------------------------+

Note that keyword arguments consume `2*nk` elements in the stack when
`nk=0..14` (unpacked).

When calling a method with keyword arguments packed in the hash object
(nk=15) which means keyword argument splat (**) is used or more than
14 keyword arguments in the actual arguments, the stack layout will
be like:

+------------------------------+
| recv | hash | block (or nil) |
+------------------------------+

Note for mruby/c

When mruby/c authors try to support new keyword arguments, they need
to handle the new meaning of the argument information operand. If they
choose not to support keyword arguments in mruby/c, it just raise
error when `nk` (taken by `(c>>4)&0xf`) is not zero. And combine
`OP_SENDV` behavior with `OP_SEND` when `n` is `15`.

If they want to support keyword arguments seriously, contact me at
<matz@ruby.or.jp> or `@yukihiro_matz`. I can help you.
2021-10-12 20:16:36 +09:00
Yukihiro "Matz" Matsumoto 906f9f2ba7 parse.y: __ENCODING__ should be determined in run-time; fix #5552
It used to be compiled to the static string in the compiler. But the
encoding status actually depends on the runtime configuration. A new
method `Kernel#__ENCODING__` is introduced to implement the feature.
2021-09-27 10:10:35 +09:00
Yukihiro "Matz" Matsumoto 6dbf49fdb4 parse.y: cons_free unused node (empty string node). 2021-09-13 12:20:11 +09:00
Yukihiro "Matz" Matsumoto 100eac3fec parse.y: allow non-local variable access from hash value omission.
For example, `{p:}` (means `{p:p}`) and `{String:}` (`{String:String}`)
should be allowed like CRuby.
2021-09-13 12:15:10 +09:00
Yukihiro "Matz" Matsumoto 6c76926d05 parse.y: allow value omission in Hash literals introduced in Ruby3.1.
`{x:, y:}` now is a syntax sugar of `{x: x, y: y}`.

This fix also includes the update of #4815 fix.
2021-09-13 11:04:27 +09:00
Yukihiro "Matz" Matsumoto 31fc74f5a8 parse.y: fix nint (int to node) and intn (node to int). 2021-09-12 23:14:43 +09:00
Yukihiro "Matz" Matsumoto 9e86d204e5 parse.y: avoid adding zero length strings. 2021-09-12 07:58:57 +09:00
Yukihiro "Matz" Matsumoto 2fb06aabe6 parse.y: refactor mrb_parser_parse().
- remove `mrb_jmpbuf` from `truct mrb_parser_state`
- unify exception handling of `mrb_state` and `mrb_parser_state`.
2021-09-07 07:31:21 +09:00
Yukihiro "Matz" Matsumoto ad3404229f parse.y: refactoring mrb_parser_parser(). 2021-09-06 13:58:01 +09:00
Yukihiro "Matz" Matsumoto 1f1dca1746 parse.y: fix an integer cast warning. 2021-07-26 11:46:22 +09:00
Yukihiro "Matz" Matsumoto 984161e6bb parse.y: mrb_int_read() returns mrb_int not unsigned long.
Also, now too big capture group index just gives `nil`, not error.
2021-07-26 11:20:59 +09:00
Yukihiro "Matz" Matsumoto 4803718b14 parse.y: unify redundant functions yywarn() and yywarning(). 2021-07-26 11:12:10 +09:00
Yukihiro "Matz" Matsumoto 8de8521cb6 parse.y: replace strtoul() by mrb_int_read(). 2021-07-25 13:07:10 +09:00
Yukihiro "Matz" Matsumoto e42f192827 codegen.c: skip -@ call if the argument is a literal integer. 2021-07-21 13:20:34 +09:00
Yukihiro "Matz" Matsumoto d63c0df6bc Merge pull request #5493 from dearblue/binding.2
Fixed finding variables from `proc` in `binding.eval` failed
2021-06-28 09:13:54 +09:00
dearblue c182903ea0 Fixed finding variables from proc in binding.eval failed
Previously the following code did not produce the expected results:
```ruby
bx = binding
block = bx.eval("a = 1; proc { a }")
bx.eval("a = 2")
p block.call  # Expect 2 but return 1 due to a bug
```

The previous implementation of `Binding#eval` evaluated the code and then merged the top layer variables.
This patch will parse and expand the variable space before making a call to `eval`.
This means that the call to `Binding#eval` will do the parsing twice.

In addition, the following changes will be made:
- Make `mrb_parser_foreach_top_variable()`, `mrb_binding_extract_proc()` and `mrb_binding_extract_env()` functions private global functions.
- Remove the `posthook` argument from `mrb_exec_irep()`.
  The `posthook` argument was introduced to implement the `binding` method.
  This patch is unnecessary because it uses a different implementation method.

ref #5362
fixed #5491
2021-06-26 13:43:36 +09:00
dearblue 52b29f41a3 Fixed finding variables defined in the upper proc failed
If no new variable was defined in the `eval` method, the variable was hidden from the nested `eval` method.

```ruby
a = 1
p eval %(b = 2; eval %(a)) # => 1 (good)
p eval %(eval %(a)) # => undefined method 'a' (NoMethodError)
```

This issue has occurred since mruby 3.0.0.
2021-06-26 11:00:32 +09:00
Yukihiro "Matz" Matsumoto 07c8470278 parse.y: allow "command" syntax in endless method definition.
This change allows `def hello = puts "Hello"` without parentheses.
This syntax has been introduced since Ruby3.1.
2021-05-18 16:26:36 +09:00
Yukihiro "Matz" Matsumoto d50c83cca7 parse.y: endless singleton method definition can omit parens. 2021-05-18 16:25:53 +09:00
John Bampton 9d32d440eb feat(CI): add the GitHub Super Linter
The GitHub Super Linter is a more robust and better supported
tool than the current GitHub Actions we are using.

Running these checks:

ERROR_ON_MISSING_EXEC_BIT: true
VALIDATE_BASH: true
VALIDATE_BASH_EXEC: true
VALIDATE_EDITORCONFIG: true
VALIDATE_MARKDOWN: true
VALIDATE_SHELL_SHFMT: true
VALIDATE_YAML: true

https://github.com/marketplace/actions/super-linter
https://github.com/github/super-linter

Added the GitHub Super Linter badge to the README.

Also updated the pre-commit framework and added
more documentation on pre-commit.

Added one more pre-commit check: check-executables-have-shebangs

Added one extra check for merge conflicts to our
GitHub Actions.

EditorConfig and Markdown linting.

Minor grammar and spelling fixes.

Update linter.yml
2021-04-16 16:37:52 +09:00
Yukihiro "Matz" Matsumoto f1c2096f8e parse.y: endless_method_name should allow ==.
e.g. `def ==(v) = true`. The issue is reported by @shuujii
2021-02-24 22:20:16 +09:00
Yukihiro "Matz" Matsumoto 386b53e6d9 parse.y: upgrade endless def syntax.
- prohibit defining setter method
- allow endless def without parentheses
2021-02-24 16:56:55 +09:00
John Bampton be40e9c783 chore: fix spelling
Normally a single spell checker can't find all the mistakes or check all types of code.

These mistakes were found by another spell checker inside my editor with a more manual sift / find.
2021-02-13 17:21:17 +10:00
Yukihiro "Matz" Matsumoto 94fb86f89b parse.y: string buffer (tokbuf) can overflow.
Check if `esclen` is smaller than `len` (original string length).
2021-02-13 14:06:37 +09:00
Yukihiro "Matz" Matsumoto c43dd75ea9 Avoid Heap Overflow in heredoc_remove_indent; fix #5316 2021-02-03 13:06:13 +09:00
Yukihiro "Matz" Matsumoto 17ecf14511 Revert "Minimize the changes in #5277"
This reverts commit dc51d89ac2.
2021-01-26 10:57:07 +09:00
Yukihiro "Matz" Matsumoto f20e10686c Prohibit r suffix after scientific notation (e.g. 1e10).
As CRuby does.
2021-01-24 21:43:20 +09:00
Yukihiro "Matz" Matsumoto dc51d89ac2 Minimize the changes in #5277
Instead of including `mruby/presym.h` everywhere, we provided the
fallback `mruby/presym.inc` under `include/mruby` directory, and specify
`-I<build-dir>/include` before `-I<top-dir>/include` in `presym.rake`.
So even when someone drops `-I<build-dir>/include` in compiler options,
it just compiles without failure.
2021-01-22 18:38:53 +09:00
Yukihiro "Matz" Matsumoto 6c5ccd5abe Merge branch 'avoid-including-presym.inc-in-existing-header-files' of https://github.com/shuujii/mruby into shuujii-avoid-including-presym.inc-in-existing-header-files 2021-01-21 14:53:18 +09:00
Yukihiro "Matz" Matsumoto 438d114917 Fix a bug that :_1 to cause syntax error; fix #5295 2021-01-21 14:45:06 +09:00
Yukihiro "Matz" Matsumoto 56fdc449eb Better error messages with yacc tokens. 2021-01-21 14:42:39 +09:00
Yukihiro "Matz" Matsumoto 4bba769b95 Avoid syntax error regarding colons in the expression; fix #5290
Reported and inspired by @hasumikin; based on CRuby's `parse.y`.
2021-01-15 22:49:40 +09:00
KOBAYASHI Shuji 90b53f4c29 Avoid including presym.inc in existing header files
Addressed an issue where existing programs linking `libmruby.a` could only
be built by adding `<build-dir>/include` to compiler's include path.
2021-01-11 09:21:07 +09:00
dearblue 58e9442737 Unified target_class and env of mrb_callinfo
If there is `env`, `env->c` means `target_class`.
2021-01-10 13:23:28 +09:00
Yukihiro "Matz" Matsumoto ec6f46c05d Use nint() for implicit cast; ref #5260 2021-01-07 11:49:50 +09:00