Commit Graph

121 Commits

Author SHA1 Message Date
KOBAYASHI Shuji 5c205e6ea1 Revert "Create presym files for each build target" (58ba883e)
Due to the above changes, it may not work with the existing build
configurations in cross-compilation (even if we can build without presym),
therefore revert it once (ref https://github.com/mruby/mruby/pull/5202#issuecomment-735412643). Sorry for the lack of consideration.
2020-12-01 12:16:55 +09:00
KOBAYASHI Shuji fdce0e712b Fix build failure; ref #5194
Fix the following two issues:

1. Compile failure when external gem is specified.
   * https://github.com/mruby/mruby/pull/5194#issuecomment-734303442
2. When there are multiple build targets, presym parsing sometimes fails or
   output result of presym is broken.
   * https://github.com/mruby/mruby/pull/5194#issuecomment-734322142

The root cause of 2 is unknown, but it seems to occur when presym parsing is
performed in parallel, therefore I change them so that they are not run in
parallel.
2020-11-27 07:59:57 +09:00
KOBAYASHI Shuji 58ba883ee9 Create presym files for each build target
Previously, presym files were always created in `build/{presym,presym.inc}`.
However, this constraint is inconvenient because it is common to use
multiple build configurations and build targets in a single mruby tree.
Therefore, change to create presym file for each build target.
2020-11-26 19:05:07 +09:00
KOBAYASHI Shuji db00fb238b Optimize presym_find
Chang to compare string length first.

### Benchmark

#### Code

* https://github.com/shuujii/mruby-presym_find-benchmark

#### Result

```console
Previous: 10.240772M i/s (25M times in 2.441222s)
     New: 16.412985M i/s (25M times in 1.523184s)
```
2020-11-24 16:30:52 +09:00
Abinoam Praxedes Marques Junior 4803c872e2 Fix #5177 - set MRUBY_CONFIG early 2020-11-23 15:18:22 -03:00
KOBAYASHI Shuji 2a97e97db4 Refine build log for generated files
* Output `GEN` log for generated files
* `MRBC` log is outputted one for each `mrbc` execution

#### Before this patch:

```console
CC    src/array.c -> build/host/src/array.o
(snip)
GEN   mrblib/*.rb -> build/host/mrblib/mrblib.c
      MRBC mrblib/00class.rb
      MRBC mrblib/10error.rb
(snip)
CC    mrbgems/mruby-time/src/time.c -> build/host/mrbgems/mruby-time/src/time.o
      MRBC mrbgems/mruby-time/mrblib/time.rb
(snip)
CC    mrbgems/mruby-socket/test/sockettest.c -> build/host/mrbgems/mruby-socket/test/sockettest.o
      MRBC mrbgems/mruby-socket/test/addrinfo.rb
      MRBC mrbgems/mruby-socket/test/basicsocket.rb
(snip)
```

#### After this patch:

```console
GEN   build/presym
GEN   build/presym.inc
CC    src/array.c -> build/host/src/array.o
(snip)
GEN   mrblib/*.rb -> build/host/mrblib/mrblib.c
      MRBC mrblib/00class.rb
           mrblib/10error.rb
(snip)
CC    mrbgems/mruby-time/src/time.c -> build/host/mrbgems/mruby-time/src/time.o
GEN   build/host/mrbgems/mruby-time/gem_init.c
      MRBC mrbgems/mruby-time/mrblib/time.rb
(snip)
CC    mrbgems/mruby-socket/test/sockettest.c -> build/host/mrbgems/mruby-socket/test/sockettest.o
GEN   build/host/mrbgems/mruby-socket/gem_test.c
      MRBC mrbgems/mruby-socket/test/addrinfo.rb
      MRBC mrbgems/mruby-socket/test/basicsocket.rb
(snip)
```
2020-11-22 20:31:01 +09:00
KOBAYASHI Shuji 1c7a1926a3 Move global function for build to lib/mruby/core_ext.rb
Because `_pp` is originally defined in `lib/mruby/core_ext.rb`, other global
functions are moved to the file.
2020-11-22 16:53:13 +09:00
KOBAYASHI Shuji 551bf0c635 Move some .rake files to tasks directory for consistency 2020-11-21 15:36:46 +09:00
KOBAYASHI Shuji 56a60fa7b7 Move lib/mruby-core-ext.rb to lib/mruby/core_ext.rb for consistency 2020-11-20 16:58:08 +09:00
Yukihiro "Matz" Matsumoto aae8237335 Scan all active C source files; ref 600e333
In addition, `include/mruby.h` will be scanned for error class symbols.
2020-11-17 09:00:06 +09:00
KOBAYASHI Shuji e334c02a50 Fix paths of gem files to be scanned for presym
When building from outside `MRUBY_ROOT` (e.g. `rake -f`), gem files were not
scanned.
2020-11-15 20:16:12 +09:00
Yukihiro "Matz" Matsumoto 600e3330b3 Scan source files only from cdump enabled gems. 2020-11-14 18:52:57 +09:00
KOBAYASHI Shuji 89f591485b Change name and usage of presym macros
To be also able to build mruby without presym in the future. However,
`MRB_QSYM` has been removed and changed as follows:

### Example

|       Type                | Symbol |  Previous Style  |   New Style    |
|---------------------------|--------|------------------|----------------|
| Operator                  | &      | MRB_QSYM(and)    | MRB_OPSYM(and) |
| Class Variable            | @@foo  | MRB_QSYM(00_foo) | MRB_CVSYM(foo) |
| Instance Variable         | @foo   | MRB_QSYM(0_foo)  | MRB_IVSYM(foo) |
| Method with Bang          | foo!   | MRB_QSYM(foo_b)  | MRB_SYM_B(foo) |
| Method with Question mark | foo?   | MRB_QSYM(foo_p)  | MRB_SYM_Q(foo) |
| Mmethod with Equal        | foo=   | MRB_QSYM(foo_e)  | MRB_SYM_E(foo) |

This change makes it possible to define, for example, `MRB_IVSYM(foo)` as
`mrb_intern_lit(mrb, "@" "foo")`, which is useful if we support building
without presym in the future.
2020-11-13 13:41:20 +09:00
KOBAYASHI Shuji 82a2a5657c Reduce recompilation when updating presym target files
Fix that updating presym target files would recompile almost all files even
if symbols did not change.
2020-11-03 16:03:46 +09:00
KOBAYASHI Shuji 9a5c78dc26 Fix parallel build
When `rake -m` and so on are used to build in parallel, building may be
started before presym files are generated. Then, for example, the following
error occurs and this issue is fixed.

```console
In file included from /Users/shuujii/mruby/mruby/include/mruby.h:92:
/mruby/mruby/include/mruby/presym.h:16:10: fatal error: '../build/presym.inc' file not found
#include <../build/presym.inc>
         ^~~~~~~~~~~~~~~~~~~~~
```
2020-11-02 14:13:33 +09:00
Yukihiro "Matz" Matsumoto e4d4192c3a Exclude symbols from mruby-test gem only used for tests. 2020-10-29 23:16:54 +09:00
KOBAYASHI Shuji 81c2108819 Revert MRUBY_TARGET mechanism; ref #5096
* In explanation of mruby, the expression `build_config.rb` is frequently
  used including official documents, so I think that it will not make sense
  if the file is no longer used.
* The `MRUBY_TARGET` mechanism seems to have little improvement, so I don't
  think it should be changed to avoid unnecessary confusion.
* `MRUBY_TARGET` and `MRuby.targets` represent somewhat different things,
  so using the same term "target" is a bit confusing.

The mechanism that can be written short when using a file under
`build_config` (renamed from `target`) directory remains
(`build_config/${MRUBY_CONFIG}.rb` is used if the path specified
in `MRUBY_CONFIG` doesn't exist).
2020-10-19 15:03:22 +09:00
Yukihiro "Matz" Matsumoto 5b40bb8d15 Avoid using C struct dump for test Ruby code.
Files under `test/t` and `mrbgem/*/test` are for tests, not for actual
execution. So symbols in those files need not to be pre-allocated. This
change slightly reduce the memory consumption.
2020-10-14 15:52:14 +09:00
Yukihiro "Matz" Matsumoto 7932523052 Remove host-debug internal target.
Target `host-debug` to use `host` internal target with debugging
configuration.
2020-10-12 18:20:23 +09:00
Kondo Uchio ed97c1779e Remove duplicated pattern 2020-10-12 18:20:21 +09:00
Uchio Kondo 0d8793047d Scan ruby files in directories 2020-10-12 18:20:20 +09:00
Kondo Uchio cbb46f6975 Exact match to allowed method/variable names 2020-10-12 18:20:20 +09:00
Uchio Kondo 0c242e278d Ensure exact match for symbols like foo!/foo?/foo=
e.g. symbols like "foo[]=" make invalid C codes
2020-10-12 18:20:20 +09:00
Yukihiro "Matz" Matsumoto 932b93fd12 Skip C comments from scan target; close #5072
The original PR was skipping Ruby comments as well, but caused some
issues in test suites.
2020-10-12 18:20:19 +09:00
Yukihiro "Matz" Matsumoto afcf19b727 Scan source code lines from downloaded mrbgems; fix #5071 2020-10-12 18:20:18 +09:00
Yukihiro "Matz" Matsumoto b7fe929232 Update Rakefile.
So that you can omit `host` target. Now `host-debug` works.
2020-10-12 16:21:51 +09:00
Yukihiro "Matz" Matsumoto ef9df5dc7d Allow MRUBY_CONFIG to specify target file out of source tree. 2020-10-12 16:21:39 +09:00
Yukihiro "Matz" Matsumoto 52507b1083 Generate C struct from irep instead of binary dump. 2020-10-12 16:21:10 +09:00
Yukihiro "Matz" Matsumoto f1c06eef7c Scan more symbols from Ruby files by rake gensym. 2020-10-12 16:21:09 +09:00
Yukihiro "Matz" Matsumoto a68f005fd7 Remove remaining chomp:true option from Rakefile. 2020-10-12 16:21:05 +09:00
Yukihiro "Matz" Matsumoto 26da07db15 Avoid using chomp:true option to File.readlines.
The option is not available in the old version of Ruby.
2020-10-12 16:21:04 +09:00
Yukihiro "Matz" Matsumoto 7ab7cadbbb Stringify non C identifier symbols to stop macro errors by old gcc. 2020-10-12 16:21:04 +09:00
Yukihiro "Matz" Matsumoto 312f54b580 Add Rakefile to build/presym dependency. 2020-10-12 16:20:55 +09:00
Yukihiro "Matz" Matsumoto 0881e34496 Add dependency from all to gensym.
You don't need to invoke `rake gensym` explicitly any longer.
2020-10-12 16:20:54 +09:00
Yukihiro "Matz" Matsumoto e2063fc267 Add operators to presym from Rakefile. 2020-10-12 16:20:53 +09:00
Yukihiro "Matz" Matsumoto 246c76e261 Rename MRB_OPSYM() to MRB_QSYM().
Where `QSYM` means quoted symbols, which cannot be represented C
symbols, so specify aliases instead.

- operators: name of the operation, e.g. add for `+`
- predicates: add `_p` suffix instead of `?`
- bang methods: add `_b` suffix instead of `!`
- instance variables: add `a_` prefix instead of `@`
- global variables: add `d_` prefix instead of `@`
- class variables: unsupported; don't use them
2020-10-12 16:20:53 +09:00
Yukihiro "Matz" Matsumoto 6128ae61a8 Simplified Rakefile. 2020-10-12 16:20:52 +09:00
Yukihiro "Matz" Matsumoto 1bcf6d9f3c Create MRB_OPSYM() macro to refer symbols corresponding operators.
For example, `MRB_OPSYM(add)` refers a symbol for `+`.
2020-10-12 16:20:51 +09:00
Yukihiro "Matz" Matsumoto acd5317fb1 Create build directory first before gensym. 2020-10-12 16:20:50 +09:00
Yukihiro "Matz" Matsumoto 8d90c7c0fa Remove unused variable from Rakefile. 2020-10-12 16:20:50 +09:00
Yukihiro "Matz" Matsumoto 5a88904fe8 Use File.open instead of File.write shortcut. 2020-10-12 16:20:49 +09:00
Yukihiro "Matz" Matsumoto b3db34c2a8 Remove presym files before writing just to make sure. 2020-10-12 16:20:49 +09:00
Yukihiro "Matz" Matsumoto fdbfeaf533 Build process updated:
You have to specify `TARGET` to specify a configuration, e.g.

```
rake TARGET=host-debug all test
```

When you port `mruby` to a new configuration:
1. copy an existing configuration under `target` directory
2. modify the new configuration file
3. build using the new configuration
4. send PR if you please
2020-10-12 16:20:47 +09:00
Yukihiro "Matz" Matsumoto 5738ed0474 Keep build/presym through rake clean. 2020-10-12 16:20:44 +09:00
Yukihiro "Matz" Matsumoto 3174061593 Prepend gensym rule to depfiles. 2020-10-12 16:20:44 +09:00
Yukihiro "Matz" Matsumoto a2fbb98fa6 Remove build/presym and build/presym.inc on rake clean. 2020-10-12 16:20:42 +09:00
Yukihiro "Matz" Matsumoto 721f934b9b Scan .rb files as well to generate presym table. 2020-10-12 16:20:42 +09:00
Yukihiro "Matz" Matsumoto f221f4e0fa Update Rakefile to generate presym. 2020-10-12 16:20:41 +09:00
Yukihiro "Matz" Matsumoto 1e156974d5 Generate a table for preallocated symbols (presym).
`presym` are symbols used in the C source files. `gensym` rake rule
scans the entire C source files and collect symbols referenced from
them.
2020-10-12 16:20:40 +09:00
KOBAYASHI Shuji bb98fd78c3 Use Rake.verbose instead of $verbose and $pp_show in build scripts
The incompatibility that the commands of `FileUtils` origin output verbose
by default due to the changes in d8a5163b and 26e6e75b is also fixed.
2019-12-28 17:03:37 +09:00