Commit Graph

122 Commits

Author SHA1 Message Date
KOBAYASHI Shuji e1fa01b9c8 Remove build/presym{,.inc} by rake clean [ci skip]
Currently, adding gems to build configuration and rebuilding does not
regenerate `presym`, which can lead to build errors.

Therefore in this case we need to remove the `presym` file and then rebuild
mruby, but when the` presym` file is regenerated we will need to recompile
most of the files, so it seems useful to have the `presym` file deleted by
`rake clean`.
2020-12-02 23:15:29 +09:00
KOBAYASHI Shuji 5c205e6ea1 Revert "Create presym files for each build target" (58ba883e)
Due to the above changes, it may not work with the existing build
configurations in cross-compilation (even if we can build without presym),
therefore revert it once (ref https://github.com/mruby/mruby/pull/5202#issuecomment-735412643). Sorry for the lack of consideration.
2020-12-01 12:16:55 +09:00
KOBAYASHI Shuji fdce0e712b Fix build failure; ref #5194
Fix the following two issues:

1. Compile failure when external gem is specified.
   * https://github.com/mruby/mruby/pull/5194#issuecomment-734303442
2. When there are multiple build targets, presym parsing sometimes fails or
   output result of presym is broken.
   * https://github.com/mruby/mruby/pull/5194#issuecomment-734322142

The root cause of 2 is unknown, but it seems to occur when presym parsing is
performed in parallel, therefore I change them so that they are not run in
parallel.
2020-11-27 07:59:57 +09:00
KOBAYASHI Shuji 58ba883ee9 Create presym files for each build target
Previously, presym files were always created in `build/{presym,presym.inc}`.
However, this constraint is inconvenient because it is common to use
multiple build configurations and build targets in a single mruby tree.
Therefore, change to create presym file for each build target.
2020-11-26 19:05:07 +09:00
KOBAYASHI Shuji db00fb238b Optimize presym_find
Chang to compare string length first.

### Benchmark

#### Code

* https://github.com/shuujii/mruby-presym_find-benchmark

#### Result

```console
Previous: 10.240772M i/s (25M times in 2.441222s)
     New: 16.412985M i/s (25M times in 1.523184s)
```
2020-11-24 16:30:52 +09:00
Abinoam Praxedes Marques Junior 4803c872e2 Fix #5177 - set MRUBY_CONFIG early 2020-11-23 15:18:22 -03:00
KOBAYASHI Shuji 2a97e97db4 Refine build log for generated files
* Output `GEN` log for generated files
* `MRBC` log is outputted one for each `mrbc` execution

#### Before this patch:

```console
CC    src/array.c -> build/host/src/array.o
(snip)
GEN   mrblib/*.rb -> build/host/mrblib/mrblib.c
      MRBC mrblib/00class.rb
      MRBC mrblib/10error.rb
(snip)
CC    mrbgems/mruby-time/src/time.c -> build/host/mrbgems/mruby-time/src/time.o
      MRBC mrbgems/mruby-time/mrblib/time.rb
(snip)
CC    mrbgems/mruby-socket/test/sockettest.c -> build/host/mrbgems/mruby-socket/test/sockettest.o
      MRBC mrbgems/mruby-socket/test/addrinfo.rb
      MRBC mrbgems/mruby-socket/test/basicsocket.rb
(snip)
```

#### After this patch:

```console
GEN   build/presym
GEN   build/presym.inc
CC    src/array.c -> build/host/src/array.o
(snip)
GEN   mrblib/*.rb -> build/host/mrblib/mrblib.c
      MRBC mrblib/00class.rb
           mrblib/10error.rb
(snip)
CC    mrbgems/mruby-time/src/time.c -> build/host/mrbgems/mruby-time/src/time.o
GEN   build/host/mrbgems/mruby-time/gem_init.c
      MRBC mrbgems/mruby-time/mrblib/time.rb
(snip)
CC    mrbgems/mruby-socket/test/sockettest.c -> build/host/mrbgems/mruby-socket/test/sockettest.o
GEN   build/host/mrbgems/mruby-socket/gem_test.c
      MRBC mrbgems/mruby-socket/test/addrinfo.rb
      MRBC mrbgems/mruby-socket/test/basicsocket.rb
(snip)
```
2020-11-22 20:31:01 +09:00
KOBAYASHI Shuji 1c7a1926a3 Move global function for build to lib/mruby/core_ext.rb
Because `_pp` is originally defined in `lib/mruby/core_ext.rb`, other global
functions are moved to the file.
2020-11-22 16:53:13 +09:00
KOBAYASHI Shuji 551bf0c635 Move some .rake files to tasks directory for consistency 2020-11-21 15:36:46 +09:00
KOBAYASHI Shuji 56a60fa7b7 Move lib/mruby-core-ext.rb to lib/mruby/core_ext.rb for consistency 2020-11-20 16:58:08 +09:00
Yukihiro "Matz" Matsumoto aae8237335 Scan all active C source files; ref 600e333
In addition, `include/mruby.h` will be scanned for error class symbols.
2020-11-17 09:00:06 +09:00
KOBAYASHI Shuji e334c02a50 Fix paths of gem files to be scanned for presym
When building from outside `MRUBY_ROOT` (e.g. `rake -f`), gem files were not
scanned.
2020-11-15 20:16:12 +09:00
Yukihiro "Matz" Matsumoto 600e3330b3 Scan source files only from cdump enabled gems. 2020-11-14 18:52:57 +09:00
KOBAYASHI Shuji 89f591485b Change name and usage of presym macros
To be also able to build mruby without presym in the future. However,
`MRB_QSYM` has been removed and changed as follows:

### Example

|       Type                | Symbol |  Previous Style  |   New Style    |
|---------------------------|--------|------------------|----------------|
| Operator                  | &      | MRB_QSYM(and)    | MRB_OPSYM(and) |
| Class Variable            | @@foo  | MRB_QSYM(00_foo) | MRB_CVSYM(foo) |
| Instance Variable         | @foo   | MRB_QSYM(0_foo)  | MRB_IVSYM(foo) |
| Method with Bang          | foo!   | MRB_QSYM(foo_b)  | MRB_SYM_B(foo) |
| Method with Question mark | foo?   | MRB_QSYM(foo_p)  | MRB_SYM_Q(foo) |
| Mmethod with Equal        | foo=   | MRB_QSYM(foo_e)  | MRB_SYM_E(foo) |

This change makes it possible to define, for example, `MRB_IVSYM(foo)` as
`mrb_intern_lit(mrb, "@" "foo")`, which is useful if we support building
without presym in the future.
2020-11-13 13:41:20 +09:00
KOBAYASHI Shuji 82a2a5657c Reduce recompilation when updating presym target files
Fix that updating presym target files would recompile almost all files even
if symbols did not change.
2020-11-03 16:03:46 +09:00
KOBAYASHI Shuji 9a5c78dc26 Fix parallel build
When `rake -m` and so on are used to build in parallel, building may be
started before presym files are generated. Then, for example, the following
error occurs and this issue is fixed.

```console
In file included from /Users/shuujii/mruby/mruby/include/mruby.h:92:
/mruby/mruby/include/mruby/presym.h:16:10: fatal error: '../build/presym.inc' file not found
#include <../build/presym.inc>
         ^~~~~~~~~~~~~~~~~~~~~
```
2020-11-02 14:13:33 +09:00
Yukihiro "Matz" Matsumoto e4d4192c3a Exclude symbols from mruby-test gem only used for tests. 2020-10-29 23:16:54 +09:00
KOBAYASHI Shuji 81c2108819 Revert MRUBY_TARGET mechanism; ref #5096
* In explanation of mruby, the expression `build_config.rb` is frequently
  used including official documents, so I think that it will not make sense
  if the file is no longer used.
* The `MRUBY_TARGET` mechanism seems to have little improvement, so I don't
  think it should be changed to avoid unnecessary confusion.
* `MRUBY_TARGET` and `MRuby.targets` represent somewhat different things,
  so using the same term "target" is a bit confusing.

The mechanism that can be written short when using a file under
`build_config` (renamed from `target`) directory remains
(`build_config/${MRUBY_CONFIG}.rb` is used if the path specified
in `MRUBY_CONFIG` doesn't exist).
2020-10-19 15:03:22 +09:00
Yukihiro "Matz" Matsumoto 5b40bb8d15 Avoid using C struct dump for test Ruby code.
Files under `test/t` and `mrbgem/*/test` are for tests, not for actual
execution. So symbols in those files need not to be pre-allocated. This
change slightly reduce the memory consumption.
2020-10-14 15:52:14 +09:00
Yukihiro "Matz" Matsumoto 7932523052 Remove host-debug internal target.
Target `host-debug` to use `host` internal target with debugging
configuration.
2020-10-12 18:20:23 +09:00
Kondo Uchio ed97c1779e Remove duplicated pattern 2020-10-12 18:20:21 +09:00
Uchio Kondo 0d8793047d Scan ruby files in directories 2020-10-12 18:20:20 +09:00
Kondo Uchio cbb46f6975 Exact match to allowed method/variable names 2020-10-12 18:20:20 +09:00
Uchio Kondo 0c242e278d Ensure exact match for symbols like foo!/foo?/foo=
e.g. symbols like "foo[]=" make invalid C codes
2020-10-12 18:20:20 +09:00
Yukihiro "Matz" Matsumoto 932b93fd12 Skip C comments from scan target; close #5072
The original PR was skipping Ruby comments as well, but caused some
issues in test suites.
2020-10-12 18:20:19 +09:00
Yukihiro "Matz" Matsumoto afcf19b727 Scan source code lines from downloaded mrbgems; fix #5071 2020-10-12 18:20:18 +09:00
Yukihiro "Matz" Matsumoto b7fe929232 Update Rakefile.
So that you can omit `host` target. Now `host-debug` works.
2020-10-12 16:21:51 +09:00
Yukihiro "Matz" Matsumoto ef9df5dc7d Allow MRUBY_CONFIG to specify target file out of source tree. 2020-10-12 16:21:39 +09:00
Yukihiro "Matz" Matsumoto 52507b1083 Generate C struct from irep instead of binary dump. 2020-10-12 16:21:10 +09:00
Yukihiro "Matz" Matsumoto f1c06eef7c Scan more symbols from Ruby files by rake gensym. 2020-10-12 16:21:09 +09:00
Yukihiro "Matz" Matsumoto a68f005fd7 Remove remaining chomp:true option from Rakefile. 2020-10-12 16:21:05 +09:00
Yukihiro "Matz" Matsumoto 26da07db15 Avoid using chomp:true option to File.readlines.
The option is not available in the old version of Ruby.
2020-10-12 16:21:04 +09:00
Yukihiro "Matz" Matsumoto 7ab7cadbbb Stringify non C identifier symbols to stop macro errors by old gcc. 2020-10-12 16:21:04 +09:00
Yukihiro "Matz" Matsumoto 312f54b580 Add Rakefile to build/presym dependency. 2020-10-12 16:20:55 +09:00
Yukihiro "Matz" Matsumoto 0881e34496 Add dependency from all to gensym.
You don't need to invoke `rake gensym` explicitly any longer.
2020-10-12 16:20:54 +09:00
Yukihiro "Matz" Matsumoto e2063fc267 Add operators to presym from Rakefile. 2020-10-12 16:20:53 +09:00
Yukihiro "Matz" Matsumoto 246c76e261 Rename MRB_OPSYM() to MRB_QSYM().
Where `QSYM` means quoted symbols, which cannot be represented C
symbols, so specify aliases instead.

- operators: name of the operation, e.g. add for `+`
- predicates: add `_p` suffix instead of `?`
- bang methods: add `_b` suffix instead of `!`
- instance variables: add `a_` prefix instead of `@`
- global variables: add `d_` prefix instead of `@`
- class variables: unsupported; don't use them
2020-10-12 16:20:53 +09:00
Yukihiro "Matz" Matsumoto 6128ae61a8 Simplified Rakefile. 2020-10-12 16:20:52 +09:00
Yukihiro "Matz" Matsumoto 1bcf6d9f3c Create MRB_OPSYM() macro to refer symbols corresponding operators.
For example, `MRB_OPSYM(add)` refers a symbol for `+`.
2020-10-12 16:20:51 +09:00
Yukihiro "Matz" Matsumoto acd5317fb1 Create build directory first before gensym. 2020-10-12 16:20:50 +09:00
Yukihiro "Matz" Matsumoto 8d90c7c0fa Remove unused variable from Rakefile. 2020-10-12 16:20:50 +09:00
Yukihiro "Matz" Matsumoto 5a88904fe8 Use File.open instead of File.write shortcut. 2020-10-12 16:20:49 +09:00
Yukihiro "Matz" Matsumoto b3db34c2a8 Remove presym files before writing just to make sure. 2020-10-12 16:20:49 +09:00
Yukihiro "Matz" Matsumoto fdbfeaf533 Build process updated:
You have to specify `TARGET` to specify a configuration, e.g.

```
rake TARGET=host-debug all test
```

When you port `mruby` to a new configuration:
1. copy an existing configuration under `target` directory
2. modify the new configuration file
3. build using the new configuration
4. send PR if you please
2020-10-12 16:20:47 +09:00
Yukihiro "Matz" Matsumoto 5738ed0474 Keep build/presym through rake clean. 2020-10-12 16:20:44 +09:00
Yukihiro "Matz" Matsumoto 3174061593 Prepend gensym rule to depfiles. 2020-10-12 16:20:44 +09:00
Yukihiro "Matz" Matsumoto a2fbb98fa6 Remove build/presym and build/presym.inc on rake clean. 2020-10-12 16:20:42 +09:00
Yukihiro "Matz" Matsumoto 721f934b9b Scan .rb files as well to generate presym table. 2020-10-12 16:20:42 +09:00
Yukihiro "Matz" Matsumoto f221f4e0fa Update Rakefile to generate presym. 2020-10-12 16:20:41 +09:00
Yukihiro "Matz" Matsumoto 1e156974d5 Generate a table for preallocated symbols (presym).
`presym` are symbols used in the C source files. `gensym` rake rule
scans the entire C source files and collect symbols referenced from
them.
2020-10-12 16:20:40 +09:00