Commit Graph

213 Commits

Author SHA1 Message Date
Yukihiro "Matz" Matsumoto c779413df3 Fix out of bound access in parse.y. 2020-10-15 18:32:27 +09:00
Yukihiro "Matz" Matsumoto d2b548de60 Remove the length of `Float' pool from the binary dump.
Also fixed the size calculation of `irep` dump, that could cause memory
corruption.
2020-10-12 18:20:28 +09:00
Yukihiro "Matz" Matsumoto bff79112a6 Dump/load 16 bits for ilen and slen in irep.
Those types are `uint16_t` in definition. Also we no longer need padding
for `iseq`.
2020-10-12 18:20:26 +09:00
Yukihiro "Matz" Matsumoto b9dfd0438f Don't compare int' with size_t (from sizeof()`). 2020-10-12 18:20:08 +09:00
Yukihiro "Matz" Matsumoto 471479e723 Change float representation in mrb binary files.
From human readable (ASCII) string representation to binary dump of
IEEE754 in little endian.
2020-10-12 18:20:06 +09:00
Yukihiro "Matz" Matsumoto 7e3d22f095 Raname mrb_exc_new_str_lit() to mrb_exc_new_lit().
It uses `mrb_str_new_lit()` internally, but it doesn't need to express
it in the name of the function (macro).
2020-10-12 18:20:06 +09:00
Yukihiro "Matz" Matsumoto 8a87549315 Rename float configuration option names.
- `MRB_WITHOUT_FLOAT` => `MRB_NO_FLOAT`
- `MRB_USE_FLOAT` => `MRB_USE_FLOAT32`

The former is to use `USE_XXX` naming convention. The latter is to make
sure `float` is 32bit float and not floating point number in general.
2020-10-12 16:21:40 +09:00
Yukihiro "Matz" Matsumoto 500f721f70 Fix typo _hander -> _handler. 2020-10-12 16:21:34 +09:00
dearblue a54a3df32c Extended mruby binary format
The catch handler table is combined with iseq block.
This is to prevent the structure from growing by adding a field for the
catch handler table to the `mrb_irep` structure.

"iseq block" and "catch handler table":
  [number of catch handler table (2 bytes)]
  [number of byte code (4 bytes)]
  [iseq (any bytes)]
  [catch handlers (multiple of 7 bytes)]

catch handler:
  [catch type (1 byte)]
  [begin offset (2 bytes)]
  [end offset (2 bytes)]
  [target offset (2 bytes)]

catch type: enum mrb_catch_type (0 = rescue, 1 = ensure)
begin offset: Includes the specified instruction address
end offset: Does not include the specified instruction address
target offset: replaces pc with the specified instruction address

This table is not expanded by `read_irep_record_1()`.
The necessary elements are expanded one by one when used.
2020-10-12 16:21:32 +09:00
Yukihiro "Matz" Matsumoto 3d8a38bea4 You don't need to keep index in local variables info in irep. 2020-10-12 16:21:22 +09:00
Yukihiro "Matz" Matsumoto dec34d6c7b Split MRB_BINARY_FORMAT to major and minor.
The minor versions should be upper compatible. So mere opcode, section
addition can be done without breaking compiled binary.
2020-10-12 16:21:20 +09:00
Yukihiro "Matz" Matsumoto 925cd2ab8f Use int instead of mrb_int for arena index. 2020-10-12 16:21:14 +09:00
Yukihiro "Matz" Matsumoto d2f267a13d Add casts to silence warnings. 2020-10-12 16:21:12 +09:00
Yukihiro "Matz" Matsumoto ee111dd175 Clarify the use of MRB_64BIT and MRB_INT64 in dump.c and load.c.
- `MRB_64BIT`: the size of a pointer is 64 bits
- `MRB_INT64`: the size of `mrb_int` is 64 bits
2020-10-12 16:21:11 +09:00
Yukihiro "Matz" Matsumoto 52507b1083 Generate C struct from irep instead of binary dump. 2020-10-12 16:21:10 +09:00
Yukihiro "Matz" Matsumoto d428fa0c4a Replace entire irep->pool.
Changes:
- `pool format is completely replaced
  - supported types: `STR`, `INT32`, `INT64`, `FLOAT`
  - `FLOAT` may be replaced by binary representation in the future
- insert `NUL` after string literals in `mrb` files
- `irep->pool` no longer store values in `mrb_value`
- instead it stores in `mrb_pool_value`
- less allocation
- `mrb_irep` can be stored in ROM
2020-10-12 16:21:08 +09:00
Yukihiro "Matz" Matsumoto daa37be549 Rename struct mrb_locals to struct mrb_lvinfo.
That stands for "local variable information".
2020-10-12 16:21:08 +09:00
Yukihiro "Matz" Matsumoto 5a3e014e49 Constify irep members.
- `pool`
- `syms`
- `reps`
2020-10-12 16:21:03 +09:00
Yukihiro "Matz" Matsumoto 5e55b61c44 Fix mrb_int and size_t combination warnings. 2020-08-11 15:06:51 +09:00
Yukihiro "Matz" Matsumoto aa30b13b36 Change arena index from mrb_int to int. 2020-06-05 17:26:40 +09:00
Yukihiro "Matz" Matsumoto f1523d2404 Squashed commit of the following:
commit 2d7d545c4c4bfce7fdcbcbe9baaeb437915742f0
Merge: 625a1249 b178914b
Author: Yukihiro "Matz" Matsumoto <matz@ruby.or.jp>
Date:   Fri Jun 5 14:35:13 2020 +0900

    Merge branch 'fix-mrb_open-with-nomem' of https://github.com/dearblue/mruby into dearblue-fix-mrb_open-with-nomem

commit b178914b11
Author: dearblue <dearblue@users.noreply.github.com>
Date:   Sat Jan 19 22:22:44 2019 +0900

    Fix invalid pointer free inside other heap's block

     1. `e = mrb_obj_alloc(...)`
     2. `e->stack = mrb->c->stack` (`mrb->c->stack` is anywhere in the range `stbase...stend`)
     3. And raised exception by `mrb_malloc()`!
     4. `mrb_free(e->stack)` by GC part (wrong free)

commit 52e3d5d858
Author: dearblue <dearblue@users.noreply.github.com>
Date:   Sat Jan 19 21:55:36 2019 +0900

    Fix memory leak for temporary symbols when out of memory

commit 4c5499b88e
Author: dearblue <dearblue@users.noreply.github.com>
Date:   Sun Jan 20 11:42:07 2019 +0900

    Fix uninitialized pointer dereference for debug section

commit 8e993167de
Author: dearblue <dearblue@users.noreply.github.com>
Date:   Sun Jan 20 11:41:09 2019 +0900

    Fix memory leak for temporary filenames when out of memory

commit 8b422577e6
Author: dearblue <dearblue@users.noreply.github.com>
Date:   Sun Jan 20 10:57:51 2019 +0900

    Fix memory leak for irep when out of memory

commit 6b35ebf49a
Author: dearblue <dearblue@users.noreply.github.com>
Date:   Sun Jan 20 10:55:50 2019 +0900

    Fix uninitialized pointer dereference when do not finished initializing irep

commit 2531f2631e
Author: dearblue <dearblue@users.noreply.github.com>
Date:   Sun Jan 20 10:48:15 2019 +0900

    Fix NULL pointer dereference when do not finished initializing irep

commit e2d6896eba
Author: dearblue <dearblue@users.noreply.github.com>
Date:   Sat Jan 19 12:54:19 2019 +0900

    Fix memory leak for irep when out of memory by `mrb_proc_new()`

commit b6214ff8a0
Author: dearblue <dearblue@users.noreply.github.com>
Date:   Sat Jan 19 12:53:07 2019 +0900

    Fix memory leak for `khash_t` in `kh_init_size()` when out of memory by `kh_alloc()`

commit 19162dd6c1
Author: dearblue <dearblue@users.noreply.github.com>
Date:   Sun Jan 20 02:15:07 2019 +0900

    Fix memory leak for symbol string when out of memory in `kh_put()`

commit 15e67297ff
Author: dearblue <dearblue@users.noreply.github.com>
Date:   Sun Jan 20 02:12:24 2019 +0900

    Fix keep wrong symbol index when out of memory

commit 3f8e2b3752
Author: dearblue <dearblue@users.noreply.github.com>
Date:   Sun Jan 20 02:08:13 2019 +0900

    Fix keep wrong symbol capacity when out of memory

commit a3cfe755ab
Author: dearblue <dearblue@users.noreply.github.com>
Date:   Sat Jan 19 10:11:37 2019 +0900

    Fix NULL pointer dereference `mrb->c` by `mark_context()`

commit d9c7b6be6e
Author: dearblue <dearblue@users.noreply.github.com>
Date:   Sun Jan 20 15:25:09 2019 +0900

    Fix protect exception for print error message

commit 100642750e
Author: dearblue <dearblue@users.noreply.github.com>
Date:   Sun Jan 20 11:59:02 2019 +0900

    Protect exception for mruby core initialization

commit 7a0418304e
Author: dearblue <dearblue@users.noreply.github.com>
Date:   Fri Jan 18 20:38:27 2019 +0900

    Fix memory leak for string object when out of memory

    The `mrb_str_pool()` function has a path to call `malloc()` twice.

    If occurs `NoMemoryError` exception in second `malloc()`,
    first `malloc()` pointer is not freed.

commit fef1c152ce
Author: dearblue <dearblue@users.noreply.github.com>
Date:   Sat Jan 19 13:05:09 2019 +0900

    Fix stack overflow when out of memory

    As a result of this change, no backtrace information is set
    for NoMemoryError (`mrb->nomem_err`).

    Detailes:

    When generating a backtrace, called `mrb_intern_lit()`,
    `mrb_str_new_cstr()` and `mrb_obj_iv_set()` function with
    `exc_debug_info()` function in `src/error.c`.

    If a `NoMemoryError` exception occurs at this time,
    the `exc_debug_info()` function will be called again,
    and in the same way `NoMemoryError` exception raised will result
    in an infinite loop to occurs stack overflow (and SIGSEGV).

commit da7d7f881b
Author: dearblue <dearblue@users.noreply.github.com>
Date:   Sun Jan 20 12:00:38 2019 +0900

    Fix NULL pointer dereference `mrb->nomem_err` when not initialized

    Add internal functions (not `static`):

      * `mrb_raise_nomemory()`
      * `mrb_core_init_abort()`
2020-06-05 14:40:07 +09:00
Yukihiro "Matz" Matsumoto 87576b819e Remove endian information/flags from compiled binary format.
Since `mruby 2.0`, compiled bytecode no longer depends on the
endian of the machine.
2020-05-07 08:38:46 +09:00
Yukihiro "Matz" Matsumoto 2532e625ed Refactor mrb_cstr_to_dbl; ref #4920 2020-01-06 15:52:39 +09:00
Yukihiro "Matz" Matsumoto 111045ecad Avoid creating temporary objects in read_irep_record_1; close #4920
The basic idea of this change is from @dearblue.

Note: the arguments of `mrb_str_pool()` have changed, but the function
is provided for internal use (No `MRB_API`). So basically you don't have
to worry about the change.
2020-01-06 15:52:39 +09:00
Fangrui Song 7f044341f9 Rename MRB_USE_ETEXT_EDATA to MRB_USE_LINK_TIME_RO_DATA_P and support lld linked programs
In lld linked programs, .rodata comes before .text, thus mrb_ro_data_p
will return false for strings in .rodata. Change the lower bound from
_etext to __ehdr_start to catch these cases. This works for ld.bfd, gold
and lld, and it does not have false positives even if .init_array does
not exist.

Remove the branch that uses _edata: strings in .data can be modified so
this is semantically incorrect. Delete the __APPLE__ branch (its
manpages say get_etext() and get_edata() are strongly discouraged).
.init_array has been adopted by most ELF platforms to supersede .ctors.

Neither _etext nor _edata is used, so rename MRB_USE_ETEXT_EDATA to
MRB_USE_EHDR_START.
2019-09-21 17:55:14 -07:00
dearblue 279c21b816 Prohibit changes to iseq in principle 2019-08-18 15:00:32 +09:00
Yukihiro "Matz" Matsumoto 85a2dfc841 Merge pull request #4403 from dearblue/read-irep-from-buf
Read irep from buffers
2019-05-22 08:58:50 +09:00
dearblue 67fc3428cb Remove "LINE" section reader
Because it is not currently output by `mrbc`.
2019-05-19 11:09:25 +09:00
dearblue e86e360b36 Fix FLAG_SRC_STATIC always set in mrb_read_irep() with MRB_USE_CUSTOM_RO_DATA_P 2019-05-03 11:42:14 +09:00
dearblue 8f6f36f654 Add mruby binary loader functions from buffer memory
Add new functions (with `MRB_API`):
- `mrb_read_irep_buf()`
- `mrb_load_irep_buf()`
- `mrb_load_irep_buf_cxt()`
2019-04-24 22:56:39 +09:00
dearblue 58d525c9fa Check mruby binary size 2019-04-24 22:56:39 +09:00
dearblue 4d85019e4c Check mruby binary version 2019-04-24 22:56:38 +09:00
Yukihiro "Matz" Matsumoto 2871d0cdc5 Avoid keeping pointers from mrb_sym2name_len(); fix #4342
The addresses for packed inline symbols reference `mrb->symbuf` that
could be overridden by the later call of `mrb_sym2name_len`. Since
file names in call stack information are kept as symbols, keeping the
address in the C structures could cause problems like #4342.

This changes small incompatible changes in function prototypes:
* `mrb_parser_get_filename`: return value changed to `mrb_sym`.
* `mrb_debug_get_filename`: add `mrb_state*` as a first argument.
* `mrb_debug_get_line`: ditto.

I believe above functions are almost internal, and no third-party
mrbgem use them.
2019-04-01 14:13:06 +09:00
KOBAYASHI Shuji 68735d1261 Fix dump/load float leteral evaluate to infinity
Example:

  # example.rb
  p(2e308)
  p(-2e308)

  Good:

    $ bin/mruby example.rb
    inf
    -inf

  Bad:

    $ bin/mrbc example.rb
    $ bin/mruby -b example.mrb
    0
    -0

Cause:

  Float infinity representation is `inf` on dump and it is converted by
  corresponding `String#to_f` on load.

Treatment:

  - Introduce new representations (`i`: +infinity, `I`: -infinity)
  - Allow old representations (`inf`, `-inf`, `infinity`, `-infinity`) too
  - Raise error for unknown representations (use corresponding `Kernel#Float`)
2019-01-08 20:43:23 +09:00
Yukihiro "Matz" Matsumoto 61f49690e4 Remove filename&lines from mrb_irep struct.
This patch slightly reduce memory consumption (2% for my test).
2018-11-15 21:01:56 +09:00
take-cheeze 659c680a5b Fix dump and load with endianess 2018-11-02 15:47:28 +09:00
Yukihiro "Matz" Matsumoto 471288f37d Reduce integer casting warnings. 2018-08-25 16:58:01 +09:00
Yukihiro "Matz" Matsumoto 891839b976 New bytecode implementation of mruby VM. 2018-07-30 22:57:54 +09:00
Yukihiro "Matz" Matsumoto 0fee6e1ee9 Fix broken support of MRB_WITHOUT_FLOAT; fix #4015 2018-04-28 15:15:28 +09:00
Takeshi Watanabe 87ef6aeb02 Fallback to float when caompiled binary with 64bit compiler.
closes #3997.
2018-04-17 22:59:07 +09:00
YAMAMOTO Masaya acdc2d1f24 Add MRB_WITHOUT_FLOAT 2017-10-11 17:58:11 +09:00
Yukihiro "Matz" Matsumoto c082249838 Exclude the top-level closure from each_object; fix #3793 2017-08-29 09:29:14 +09:00
Yukihiro "Matz" Matsumoto 8bf492f127 Reduce integer type mismatch warnings in VC. 2017-08-12 09:35:35 +09:00
Nobuyoshi Nakada 6511bfd79a Removed trailing spaces 2016-09-28 12:29:41 +09:00
Kouhei Sutou 91bd3ebd32 Remove needless MRB_API
ref #3215

If a function (such as mrb_read_irep_file()) is declared without MRB_API
in header file (such as include/mruby/dump.h), implementation of the
function in source file (such as src/load.c) should also defined without
MRB_API.

If MRB_API is mismatch, Visual C++ reports link error with C2375 error
code: https://msdn.microsoft.com/en-us/library/5k6kw95a.aspx
2016-09-25 21:47:49 +09:00
Yukihiro "Matz" Matsumoto 7a382e8bdf Remove unnecessary MRB_API from read_irep related functions; ref #3215 2016-09-24 05:26:51 +09:00
Yukihiro "Matz" Matsumoto f3cce24035 replace mrb_toplevel_run() by mrb_top_run() 2016-01-07 22:48:22 +09:00
Kouhei Sutou a561bdb25f Support backtrace after method calls
GitHub: fix #2902, #2917

The current implementation traverses stack to retrieve backtrace. But
stack will be changed when some operations are occurred. It means that
backtrace may be broken after some operations.

This change (1) saves the minimum information to retrieve backtrace when
exception is raised and (2) restores backtrace from the minimum
information when backtrace is needed. It reduces overhead for creating
backtrace Ruby objects.

The space for the minimum information is reused by multiple
exceptions. So memory allocation isn't occurred for each exception.
2015-12-29 20:36:12 +09:00
Yukihiro "Matz" Matsumoto 5c405dea3d include changed from by quotes ("") to by brackets (<>); close #3032 2015-11-27 17:48:23 +09:00
Yukihiro "Matz" Matsumoto 4440566b95 DISABLE_STDIO/ENABLE_DEBUG macros to rename; close #3014
changes:
 * rename DISABLE_STDIO -> MRB_DISABLE_STDIO
 * rename ENABLE_DEBUG -> MRB_ENABLE_DEBUG_HOOK
 * no more opposite macro definitions (e.g. ENABLE_STDIO, DISABLE_DEBUG).
 * rewrite above macro references throughout the code.
 * update documents
2015-11-17 07:30:34 +09:00