734e72ce8f
bit_util: Guarantee popcount's presence.
...
Implement popcount generically, so that we can rely on it being present.
2020-12-07 06:21:08 -08:00
d9f7e6c668
hpdata: Add a test.
...
We're about to make the functionality here more complicated; testing hpdata
directly (rather than relying on user's tests) will make debugging easier.
2020-12-07 06:21:08 -08:00
3ed0b4e8a3
HPA: Add an nevictions counter.
...
I.e. the number of times we've purged a hugepage-sized region.
2020-12-07 06:21:08 -08:00
fffcefed33
malloc_conf: Clarify HPA options.
2020-12-07 06:21:08 -08:00
f7cf23aa4d
psset: Relegate alloc/dalloc to test code.
...
This is no longer part of the "core" functionality; we only need the stub
implementations as an end-to-end test of hpdata + psset interactions when
metadata is being modified. Treat them accordingly.
2020-12-07 06:21:08 -08:00
f9299ca572
HPA: Use psset fit/insert/remove.
...
This will let us remove alloc_new and alloc_reuse functions from the psset.
2020-12-07 06:21:08 -08:00
0971e1e4e3
hpdata: Use addr/size instead of begin/npages.
...
This is easier for the users of the hpdata.
2020-12-07 06:21:08 -08:00
5228d869ee
psset: Use fit/insert/remove as basis functions.
...
All other functionality can be implemented in terms of these; doing so (while
retaining the same API) will be convenient for subsequent refactors.
2020-12-07 06:21:08 -08:00
089f8fa442
Move hpdata bitmap logic out of the psset.
2020-12-07 06:21:08 -08:00
ca30b5db2b
Introduce hpdata_t.
...
Using an edata_t both for hugepages and the allocations within those hugepages
was convenient at first, but has outlived its usefulness. Representing
hugepages explicitly, with their own data structure, will make future
development easier.
2020-12-07 06:21:08 -08:00
4a15008cfb
HPA unit test: skip if unsupported.
...
Previously, we replicated the logic in hpa_supported in the test as well.
2020-12-07 06:21:08 -08:00
43af63fff4
HPA: Manage whole hugepages at a time.
...
This redesigns the HPA implementation to allow us to manage hugepages all at
once, locally, without relying on a global fallback.
2020-12-07 06:21:08 -08:00
63677dde63
Pages: Statically detect if pages_huge may succeed
2020-12-07 06:21:08 -08:00
c1b2a77933
psset: Move in stats.
...
A later change will benefit from having these functions pulled into a
psset-module set of functions.
2020-12-07 06:21:08 -08:00
d0a991d47b
psset: Add insert/remove functions.
...
These will allow us to (for instance) move pageslabs from a psset dedicated to
not-yet-hugeified pages to one dedicated to hugeified ones.
2020-12-07 06:21:08 -08:00
d438296b1f
narenas_ratio: Accept fractional values.
...
With recent scalability improvements to the HPA, we're experimenting with much
lower arena counts; this gets annoying when trying to test across different
hardware configurations using only the narenas setting.
2020-12-04 23:48:19 -08:00
ecd39418ac
Add fxp: A fixed-point math library.
...
This will be used in the next commit to allow non-integer values for
narenas_ratio.
2020-12-04 23:48:19 -08:00
99c2d6c232
Backport jeprof --collapse for flamegraph generation
2020-12-04 10:48:21 -08:00
520b75fa2d
utrace support with label based signature.
2020-11-30 11:43:00 -08:00
92e189be8b
Add some comments to the batch allocation logic flow
2020-11-16 20:58:01 -08:00
d96e4525ad
Route batch allocation of small batch size to tcache
2020-11-16 20:58:01 -08:00
ac480136d7
Split out locality checking in batch allocation tests
2020-11-16 20:58:01 -08:00
be5e49f4fa
Add a batch mode for cache_bin_alloc()
2020-11-16 20:58:01 -08:00
4a65f34930
Fix a cache bin test
2020-11-16 20:58:01 -08:00
566c4a8594
Slight changes to cache bin internal functions
2020-11-16 20:58:01 -08:00
9545c2cd36
Add sample interval to prof last-N dump
2020-11-13 15:33:27 -08:00
cf2549a149
Add a per-arena oversize_threshold.
...
This can let manual arenas trade off memory and CPU the way auto arenas do.
2020-11-13 13:45:35 -08:00
4ca3d91e96
Rename geom_grow -> exp_grow.
...
This was promised in the review of the introduction of geom_grow, but would have
been painful to do there because of the series that introduced it. Now that
those are comitted, renaming is easier.
2020-11-13 13:42:33 -08:00
b4c37a6e81
Rename edata_tree_t -> edata_avail_t.
...
This isn't a tree any more, and it mildly irritates me any time I see it.
2020-11-13 13:42:11 -08:00
95f0a77fde
Detect pthread_getname_np explicitly.
...
At least one libc (musl) defines pthread_setname_np without defining
pthread_getname_np. Detect the presence of each individually, rather than
inferring both must be defined if set is.
2020-11-11 17:31:22 -08:00
b3c5690b7e
Update config.{guess,sub} to 2020-11-07@77632d9
2020-11-10 13:32:32 -08:00
589638182a
Use the edata_cache_small_t in the HPA.
2020-11-05 12:34:43 -08:00
03a6047111
Edata cache small: rewrite.
...
In previous designs, this was intended to be a sort of cache that couldn't fail.
In the current design, we want to use it just as a contention reduction
mechanism. Rewrite it with those goals in mind.
2020-11-05 12:34:43 -08:00
c9757d9e3b
HPA: Don't disable shards that were never started.
2020-11-05 12:34:43 -08:00
1b3ee75667
Add experimental.thread.activity_callback.
...
This (experimental, undocumented) functionality can be used by users to track
various statistics of interest at a finer level of granularity than the thread.
2020-11-05 12:33:25 -08:00
27ef02ca9a
Android build fix proposal.
...
These are detected at configure time while they are glibc
specifics. the bionic equivalent is not api compatible
and dlopen is restricted in this platform.
2020-11-02 13:38:44 -08:00
d2d941017b
MADV_DO[NOT]DUMP support equivalence on FreeBSD.
2020-11-02 09:15:15 -08:00
180b843159
Appveyor: fix 404 errors.
...
It looks like the mirrors we were using no longer carry this package, but that
it is installed by default and so no longer needs a remote mirror.
2020-10-27 15:28:20 -07:00
ef6d51ed44
DragonFlyBSD build support.
2020-10-27 12:35:19 -07:00
bf72188f80
Allow opt.tcache_max to accept small size classes.
...
Previously all the small size classes were cached. However this has downsides
-- particularly when page size is greater than 4K (e.g. iOS), which will result
in much higher SMALL_MAXCLASS.
This change allows tcache_max to be set to lower values, to better control
resources taken by tcache.
2020-10-24 20:43:44 -07:00
ea32060f9c
SEC: Implement thread affinity.
...
For now, just have every thread pick a shard once and stick with it.
2020-10-23 11:14:34 -07:00
d16849c91d
psset: Do first-fit based on slab age.
...
This functions more like the serial number strategy of the ecache and
hpa_central_t. Longer-lived slabs are more likely to continue to live for
longer in the future.
2020-10-23 11:14:34 -07:00
634ec6f50a
Edata: add an "age" field.
2020-10-23 11:14:34 -07:00
6599651aee
PA: Use an SEC in fron of the HPA shard.
2020-10-23 11:14:34 -07:00
ea51e97bb8
Add SEC module: a small extent cache.
...
This can be used to take pressure off a more centralized, worse-sharded
allocator without requiring a full break of the arena abstraction.
2020-10-23 11:14:34 -07:00
1964b08394
HPA: Add stats for the hpa_shard.
2020-10-23 11:14:34 -07:00
534504d4a7
HPA: add size-exclusion functionality.
...
I.e. only allowing allocations under or over certain sizes.
2020-10-23 11:14:34 -07:00
484f04733e
HPA: Add central mutex contention stats.
2020-10-23 11:14:34 -07:00
bf025d2ec8
HPA: Make slab sizes and maxes configurable.
...
This allows easy experimentation with them as tuning parameters.
2020-10-23 11:14:34 -07:00
1c7da33317
HPA: Tie components into a PAI implementation.
2020-10-23 11:14:34 -07:00
David Goldblatt