admin/mruby-mruby
1
0
Fork 0
mirror of https://github.com/mruby/mruby synced 2026-06-08 16:11:16 +00:00
Code Issues Packages Projects Releases Wiki Activity
11,990 Commits 6 Branches 38 Tags
af13cfb7d5b7ddbadada80931d2f0d856d72170b
Commit Graph

61 Commits

Author SHA1 Message Date
KOBAYASHI Shuji f2d8db39be Reduce memory usage of Hash object 
## Implementation Summary

* Change entry list from segmented list to flat array.
* Change value of hash bucket from pointer to entry to index of entry list,
  and represent it by variable length bits according to capacity of hash
  buckets.
* Store management information about entry list and hash table to `struct
  RHash` as much as possible.

## Benchmark Summary

Only the results of typical situations on 64-bit Word-boxing are present
here. For more detailed information, including consideration, see below
(although most of the body is written in Japanese).

* https://shuujii.github.io/mruby-hash-benchmark

### Memory Usage

Lower value is better.

| Hash Size |   Baseline    |      New      |   Factor   |
|----------:|--------------:|--------------:|-----------:|
|        16 |          344B |          256B |   0.74419x |
|        40 |        1,464B |          840B |   0.57377x |
|       200 |        8,056B |        3,784B |   0.46971x |
|       500 |       17,169B |        9,944B |   0.57949x |

### Performance

Higher value is better.

#### `mrb_hash_set`

| Hash Size |   Baseline    |      New      |   Factor   |
|----------:|--------------:|--------------:|-----------:|
|        16 |  1.41847M i/s |  1.36004M i/s |   0.95881x |
|        40 |  0.39224M i/s |  0.31888M i/s |   0.81296x |
|       200 |  0.03780M i/s |  0.04290M i/s |   1.13494x |
|       500 |  0.01225M i/s |  0.01314M i/s |   1.07275x |

#### `mrb_hash_get`

| Hash Size |   Baseline    |      New      |   Factor   |
|----------:|--------------:|--------------:|-----------:|
|        16 | 26.05920M i/s | 30.19543M i/s |   1.15872x |
|        40 | 44.26420M i/s | 32.75781M i/s |   0.74005x |
|       200 | 44.55171M i/s | 31.56926M i/s |   0.70860x |
|       500 | 39.19250M i/s | 29.73806M i/s |   0.75877x |

#### `mrb_hash_each`

| Hash Size |   Baseline    |      New      |   Factor   |
|----------:|--------------:|--------------:|-----------:|
|        16 | 25.11964M i/s | 30.34167M i/s |   1.20789x |
|        40 | 11.74253M i/s | 13.25539M i/s |   1.12884x |
|       200 |  2.01133M i/s |  2.97214M i/s |   1.47770x |
|       500 |  0.87411M i/s |  1.21178M i/s |   1.38631x |

#### `Hash#[]=`

| Hash Size |   Baseline    |      New      |   Factor   |
|----------:|--------------:|--------------:|-----------:|
|        16 |  0.50095M i/s |  0.56490M i/s |   1.12764x |
|        40 |  0.19132M i/s |  0.18392M i/s |   0.96129x |
|       200 |  0.03624M i/s |  0.03256M i/s |   0.89860x |
|       500 |  0.01527M i/s |  0.01236M i/s |   0.80935x |
#### `Hash#[]`

| Hash Size |   Baseline    |      New      |   Factor   |
|----------:|--------------:|--------------:|-----------:|
|        16 | 11.53211M i/s | 12.78806M i/s |   1.10891x |
|        40 | 15.26920M i/s | 13.37529M i/s |   0.87596x |
|       200 | 15.28550M i/s | 13.36410M i/s |   0.87430x |
|       500 | 14.57695M i/s | 12.75388M i/s |   0.87494x |

#### `Hash#each`

| Hash Size |   Baseline    |      New      |   Factor   |
|----------:|--------------:|--------------:|-----------:|
|        16 |  0.30462M i/s |  0.27080M i/s |   0.88898x |
|        40 |  0.12912M i/s |  0.11704M i/s |   0.90642x |
|       200 |  0.02638M i/s |  0.02402M i/s |   0.91071x |
|       500 |  0.01066M i/s |  0.00959M i/s |   0.89953x |

#### `Hash#delete`

| Hash Size |   Baseline    |      New      |   Factor   |
|----------:|--------------:|--------------:|-----------:|
|        16 |  7.84167M i/s |  6.96419M i/s |   0.88810x |
|        40 |  6.91292M i/s |  7.41427M i/s |   1.07252x |
|       200 |  3.75952M i/s |  7.32080M i/s |   1.94727x |
|       500 |  2.10754M i/s |  7.05963M i/s |   3.34970x |

#### `Hash#shift`

| Hash Size |   Baseline    |      New      |   Factor   |
|----------:|--------------:|--------------:|-----------:|
|        16 | 14.66444M i/s | 13.18876M i/s |   0.89937x |
|        40 | 11.95124M i/s | 11.10420M i/s |   0.92913x |
|       200 |  5.53681M i/s |  7.88155M i/s |   1.42348x |
|       500 |  2.96728M i/s |  5.40405M i/s |   1.82121x |

#### `Hash#dup`

| Hash Size |   Baseline    |      New      |   Factor   |
|----------:|--------------:|--------------:|-----------:|
|        16 |  0.15063M i/s |  5.37889M i/s |  35.71024x |
|        40 |  0.06515M i/s |  3.38196M i/s |  51.91279x |
|       200 |  0.01359M i/s |  1.46538M i/s | 107.84056x |
|       500 |  0.00559M i/s |  0.75411M i/s | 134.88057x |

### Binary Size

Lower value is better.

|    File    |   Baseline    |      New      |  Factor   |
|:-----------|--------------:|--------------:|----------:|
| mruby      |      730,408B |      734,176B |  1.00519x |
| libmruby.a |    1,068,134B |    1,072,846B |  1.00441x |

## Other Fixes

The following issues have also been fixed in the parts where there was some
change this time.

* [Heap use-after-free in `Hash#value?`](https://gist.github.com/shuujii/30e4fcd5844a4112a0ecd4a5b3483101#file-heap-use-after-free-in-hash-value-md)
* [Heap use-after-free in `ht_hash_equal`](https://gist.github.com/shuujii/30e4fcd5844a4112a0ecd4a5b3483101#file-heap-use-after-free-in-ht_hash_equal-md)
* [Heap use-after-free in `ht_hash_func`](https://gist.github.com/shuujii/30e4fcd5844a4112a0ecd4a5b3483101#file-heap-use-after-free-in-ht_hash_func-md)
* [Heap use-after-free in `mrb_hash_merge`](https://gist.github.com/shuujii/30e4fcd5844a4112a0ecd4a5b3483101#file-heap-use-after-free-in-mrb_hash_merge-md)
* [Self-replacement does not work for `Hash#replace`](https://gist.github.com/shuujii/30e4fcd5844a4112a0ecd4a5b3483101#file-self-replacement-does-not-work-for-hash-replace-md)
* [Repeated deletes and inserts increase memory usage of `Hash`](https://gist.github.com/shuujii/30e4fcd5844a4112a0ecd4a5b3483101#file-repeated-deletes-and-inserts-increase-memory-usage-of-hash-md)
* [`Hash#rehash` does not reindex completely](https://gist.github.com/shuujii/30e4fcd5844a4112a0ecd4a5b3483101#file-hash-rehash-does-not-reindex-completely-md)
* `mrb_hash_delete_key` does not cause an error for frozen object
* `mrb_hash_new_capa` does not allocate required space first
* [`mrb_os_memsize_of_hash_table` result is incorrect](https://github.com/mruby/mruby/pull/5032#discussion_r457994075)
 2020-11-10 15:21:49 +09:00
Yukihiro "Matz" Matsumoto eddd324979 Add MRB_SYM() for inline symbols. 2020-10-12 16:20:41 +09:00
Yukihiro "Matz" Matsumoto 5e55b61c44 Fix mrb_int and size_t combination warnings. 2020-08-11 15:06:51 +09:00
Rory O'Connell f74d370c15 mrb_ prefix convention 2020-07-15 19:57:22 -07:00
Rory OConnell e7bd7d0eaf Use size of hash's table in calculation 2020-07-13 15:58:50 -07:00
dearblue 40e8a90e89 Remove unnecessary type mrb_hash_value 
The type `mrb_hash_value` is no longer used by the segmented list
implementation (ref e8dcfe1 and e65d426).
 2019-09-29 23:55:38 +09:00
Yukihiro "Matz" Matsumoto bbfd64d01b Add an annotation of the return value from mrb_delete_key; #4737 
The return value from `mrb_delete_key` needs to be protected from GC in
some cases.
 2019-09-29 13:53:45 +09:00
Yukihiro "Matz" Matsumoto 28bbd3e7d9 Add argument names to C function prototypes. 2019-09-14 23:21:44 +09:00
David Siaw b5299b1c58 fix up documentation for values 2019-08-18 20:12:44 +09:00
Yukihiro "Matz" Matsumoto e65d42644a Remove #include <mruby/khash.h> from mruby/hash.h. 2018-12-17 15:59:42 +09:00
dearblue 62dd4d89fc Add mrb_hash_size() function. 2018-12-14 21:41:07 +09:00
Yukihiro "Matz" Matsumoto 265171a28c Rename ht_foreach_func to mrb_hash_foreach_func. 2018-12-11 10:41:26 +09:00
Yukihiro "Matz" Matsumoto c0d91a14e7 Add API function mrb_hash_foreach() to iterate over items in a hash. 2018-12-11 09:01:05 +09:00
Yukihiro "Matz" Matsumoto 25d390d6cc Improve Hash table using variable sized segments. 2018-11-19 09:30:15 +09:00
Yukihiro "Matz" Matsumoto e8dcfe1745 Use segmented list to implement Hash [Experimental] 
I know it's not hash at all, but reduce memory consumption.
 2018-09-26 12:55:22 +09:00
Yukihiro "Matz" Matsumoto f6564dd83e Add new function mrb_ensure_hash_type(); ref #4097 
Unlike `mrb_check_hash_type()` that returns `nil` if the argument is
not a `Hash`, `mrb_ensure_hash_type()` raises a `TypeError` exception.
 2018-08-30 22:30:36 +09:00
Yukihiro "Matz" Matsumoto 3554fc54de Add a new function mrb_hash_merge(). 2018-08-25 09:41:21 +09:00
Yukihiro "Matz" Matsumoto 8c9e712784 Keyword argument implemented. 2018-07-30 22:58:01 +09:00
Yukihiro "Matz" Matsumoto 667e788159 Use mrb_int instead of int as argument to mrb_hash_new_capa. 2017-06-19 09:21:07 +09:00
Herwin Weststrate 18d6994f78 Documented most methods in mruby/hash.h 2016-11-17 11:52:28 +01:00
William Light e02a88e031 make mrb_hash_values() a public API function 2016-08-01 11:05:46 +02:00
Yukihiro "Matz" Matsumoto 3c73c315f4 Hash: check flags before accessing ifnone; ref #980 2016-02-05 10:08:28 +09:00
Yukihiro "Matz" Matsumoto 725b3ca5a7 move KHASH_DECLARE(ht..) to mruby/hash.h; close #3073 2016-01-07 12:46:38 +09:00
Yukihiro "Matz" Matsumoto 065966dae4 common.h are supposed to be included from other header, so call it with quotes; ref #3032 2015-11-28 00:10:38 +09:00
Yukihiro "Matz" Matsumoto 5c405dea3d include changed from by quotes ("") to by brackets (<>); close #3032 2015-11-27 17:48:23 +09:00
Seba Gamboa 11fb10a21a Merge doc/api/mruby/hash.h.md docs 2015-10-08 13:29:57 -03:00
Seba Gamboa a0fe0c40b1 Remove old doxygen tags 2015-10-08 12:29:10 -03:00
Seba Gamboa 40bf7bde78 Sorting documentation grouping 2015-09-21 01:48:42 -03:00
Seba Gamboa c127614638 Setting up doxygen groups 2015-09-20 21:14:07 -03:00
Tatsuhiko Kubo 5fa30aeaea Fix mismatches for MRB_API declarations. 2014-08-29 01:06:22 +09:00
Yukihiro "Matz" Matsumoto 206f89e209 add MRB_API modifiers to mruby API functions 2014-08-04 00:47:08 +09:00
take_cheeze df780ae5e9 add mrb_intern_lit for creating symbol from string literal 2013-12-01 10:38:59 +09:00
Yukihiro "Matz" Matsumoto 9c6398a444 rename mrb_intern2() to mrb_intern(); huge API incompatibility; close #1513 2013-11-29 08:47:28 +09:00
Yukihiro "Matz" Matsumoto 40e6cdcb5c rename mrb_value_p() to mrb_ptr() since _p means predicate in mruby source 2013-08-07 17:39:28 +09:00
Yuichi Nishiwaki 8dc2fa3bc8 add read barrier to value.p 
API changes:
- value.p must be accessed via mrb_value_p macro
- value.p must be mutated via MRB_SET_VALUE_P macro
 2013-08-03 14:03:44 -07:00
Ryan Scott 0d4227ac04 Implemented ObjectSpace.count_objects to count the number of allocated objects for each type 2013-05-12 18:42:21 +10:00
Yukihiro "Matz" Matsumoto 443b2c2a2b rename hash related gc functions 2013-04-29 07:48:33 +09:00
Masaki Muranaka f8fa56d8c3 Use mrb_intern2() instead of mrb_intern(). This is for avoiding overhead by strlen(). 2013-03-22 14:22:45 +09:00
Cremno feaff10c58 removed declarations of undefined functions 2013-02-03 19:48:41 +01:00
skandhas 1a528da40b add mrb_check_hash_type 2013-01-10 17:02:41 +08:00
Yukihiro Matz Matsumoto 3eeef30618 make mrb_hash_keys() non static function 2012-11-14 14:42:21 +09:00
Masamitsu MURASE 42fa994e9b Publish mrb_hash_delete_key. 2012-08-26 04:17:56 +09:00
Yukihiro Matsumoto 97b7eb6096 rename MRUBY_OBJECT_HEADER to MRB_OBJECT_HEADER 2012-08-18 19:05:18 +09:00
Yukihiro Matsumoto 7d02df3016 NaN boxing 2012-08-14 13:16:34 +09:00
Yukihiro Matsumoto c9fe903fe1 now segmented list can be used as instance variable tables by -DMRB_USE_IV_SEGLIST; we still need to measure the performance and memory consumption 2012-08-12 01:59:18 +09:00
Masaki Muranaka 22b032fcdf mrb_hash_delete_key(), mrb_hash_keys() : Declare as static. 
mrb_check_hash_type(): Remove as no implementation.
 2012-07-29 20:11:43 +09:00
Yukihiro Matsumoto 766ae4ee5f dislose mrb_hash_keys; close #304 2012-06-21 16:27:16 +09:00
Yukihiro Matsumoto 1a369de489 remove src/gc.h 2012-06-07 01:18:17 +09:00
Yukihiro Matsumoto cc86250297 remove unsafe macros in hash.h 2012-06-03 00:23:59 +09:00
Yukihiro Matsumoto 110cdd82e0 naming convention consistency for hash.h 2012-06-02 21:47:27 +09:00