27 Commits

Author SHA1 Message Date
Bin Liu be2de8ccd8 Introduce pinned extents to contain unpurgeable pages
Some pages (e.g., hugetlb pages) cannot be purged, and should be
prioritized for reuse.  A custom extent_alloc hook signals this by
OR'ing EXTENT_ALLOC_FLAG_PINNED into the low bits of the returned
pointer; jemalloc strips the flag bits and caches pinned extents in
a dedicated ecache_pinned, separate from the dirty/muzzy decay
pipeline.

Pinned extents do not coalesce eagerly, except for ones larger than
SC_LARGE_MINCLASS.  A prefer-small policy reuses the smallest fitting
pinned extent, to avoid unnecessary split/fragmentation.
2026-05-05 10:44:28 -07:00
Slobodan Predolac 7638093c73 Improve const correctness in the repo 2026-05-04 11:56:17 -07:00
guangli-dai 6200e8987f Reformat the codebase with the clang-format 18. 2026-03-10 18:14:33 -07:00
Kevin Svetlitski 41e0b857be Make headers self-contained by fixing #includes
Header files are now self-contained, which makes the relationships
between the files clearer, and crucially allows LSP tools like `clangd`
to function correctly in all of our header files. I have verified that
the headers are self-contained (aside from the various Windows shims) by
compiling them as if they were C files – in a follow-up commit I plan to
add this to CI to ensure we don't regress on this front.
2023-07-14 09:06:32 -07:00
Qi Wang 602edd7566 Enabled -Wstrict-prototypes and fixed warnings. 2023-07-06 12:00:02 -07:00
Qi Wang deb8e62a83 Implement guard pages.
Adding guarded extents, which are regular extents surrounded by guard pages
(mprotected).  To reduce syscalls, small guarded extents are cached as a
separate eset in ecache, and decay through the dirty / muzzy / retained pipeline
as usual.
2021-09-26 16:30:15 -07:00
Qi Wang add636596a Stop checking head state in the merge hook.
Now that all merging go through try_acquire_edata_neighbor, the mergeablility
checks (including head state checking) are done before reaching the merge hook.
In other words, merge hook will never be called if the head state doesn't agree.
2021-03-29 17:19:53 -07:00
David T. Goldblatt 9f4fc27389 Ehooks: Fix a build warning.
We wrote `return some_void_func()` in a function returning void, which is
confusing and triggers warnings on MSVC.
2020-02-25 10:21:03 -08:00
David Goldblatt 6342da0970 Ehooks: Further optimize default merge case.
This avoids the cost of an iealloc in cases where the user uses the default
merge hook without using the default extent hooks.
2019-12-20 10:18:40 -08:00
David Goldblatt 9cad5639ff Ehooks: remove arena_ind parameter.
This lives within the ehooks_t now, so that callers don't need to know it.
2019-12-20 10:18:40 -08:00
David Goldblatt 57fe99d4be Move relevant index into the ehooks_t itself.
It's always passed into the ehooks; keeping it colocated lets us avoid passing
the arena everywhere.
2019-12-20 10:18:40 -08:00
David Goldblatt 0704516245 Ehooks: Add head tracking. 2019-12-20 10:18:40 -08:00
David Goldblatt 09475bf8ac extent_may_dalloc -> ehooks_dalloc_will_fail 2019-12-20 10:18:40 -08:00
David Goldblatt a738a66b5c Ehooks: Add some debug zero and addr checks.
These help make sure that the ehooks return properly zeroed memory when required
to.
2019-12-20 10:18:40 -08:00
David Goldblatt 4b2e5ee8b9 Ehooks: Add a "zero" ehook.
This is the first API expansion.  It lets the hooks pick where and how to purge
within themselves.
2019-12-20 10:18:40 -08:00
David Goldblatt 39fdc690a0 Ehooks comments and cleanup. 2019-12-20 10:18:40 -08:00
David Goldblatt c8dae890c8 Extent -> Ehooks: Move over default hooks. 2019-12-20 10:18:40 -08:00
David Goldblatt 2fe5108263 Extent -> Ehooks: Move merge hook. 2019-12-20 10:18:40 -08:00
David Goldblatt 1fff4d2ee3 Extent -> Ehooks: Move split hook. 2019-12-20 10:18:40 -08:00
David Goldblatt a5b42a1a10 Extent -> Ehooks: Move purge_forced hook. 2019-12-20 10:18:40 -08:00
David Goldblatt 368baa42ef Extent -> Ehooks: Move purge_lazy hook. 2019-12-20 10:18:40 -08:00
David Goldblatt d78fe241ac Extent -> Ehooks: Move commit and decommit hooks. 2019-12-20 10:18:40 -08:00
David Goldblatt 5459ec9dae Extent -> Ehooks: Move destroy hook. 2019-12-20 10:18:40 -08:00
David Goldblatt bac8e2e5a6 Extent -> Ehooks: Move dalloc hook. 2019-12-20 10:18:40 -08:00
David Goldblatt dc8b4e6e13 Extent -> Ehooks: Move alloc hook. 2019-12-20 10:18:40 -08:00
David Goldblatt ae0d8e8591 Move extent ehook calls into ehooks 2019-12-20 10:18:40 -08:00
David Goldblatt ba8b9ecbcb Add ehooks module 2019-12-20 10:18:40 -08:00