Add opt hpa_hugify_sync to hugify synchronously

Linux 6.1 introduced `MADV_COLLAPSE` flag to perform a best-effort
synchronous collapse of the native pages mapped by the memory range into
transparent huge pages.

Synchronous hugification might be beneficial for at least two reasons:
we are not relying on khugepaged anymore and get an instant feedback if
range wasn't hugified.

If `hpa_hugify_sync` option is on, we'll try to perform synchronously
collapse and if it wasn't successful, we'll fallback to asynchronous
behaviour.
This commit is contained in:
Dmitry Ilvokhin 2024-10-31 11:43:11 -07:00 committed by stanjo74
parent a361e886e2
commit 0ce13c6fb5
15 changed files with 141 additions and 8 deletions

View file

@ -567,6 +567,30 @@ pages_nohuge_unaligned(void *addr, size_t size) {
return pages_nohuge_impl(addr, size, false);
}
bool
pages_collapse(void *addr, size_t size) {
assert(PAGE_ADDR2BASE(addr) == addr);
assert(PAGE_CEILING(size) == size);
/*
* There is one more MADV_COLLAPSE precondition that is not easy to
* express with assert statement. In order to madvise(addr, size,
* MADV_COLLAPSE) call to be successful, at least one page in the range
* must currently be backed by physical memory. In particularly, this
* means we can't call pages_collapse on freshly mapped memory region.
* See madvise(2) man page for more details.
*/
#if defined(JEMALLOC_HAVE_MADVISE_COLLAPSE) && \
(defined(MADV_COLLAPSE) || defined(JEMALLOC_MADV_COLLAPSE))
# if defined(MADV_COLLAPSE)
return (madvise(addr, size, MADV_COLLAPSE) != 0);
# elif defined(JEMALLOC_MADV_COLLAPSE)
return (madvise(addr, size, JEMALLOC_MADV_COLLAPSE) != 0);
# endif
#else
return true;
#endif
}
bool
pages_dontdump(void *addr, size_t size) {
assert(PAGE_ADDR2BASE(addr) == addr);