Commit graph

3604 commits

Author SHA1 Message Date
lexprfuncall
7dcdafea00 Change the default page size to 64KiB on Aarch64 Linux
This updates the configuration script to set the default page size to
64KiB on Aarch64 Linux.  This is motivated by compatibility as a build
configured for a 64KiB page will work on kernels that use the smaller
4KiB or 16KiB pages, whereas the reverse is not true.

To make the configured page size setting more visible, the script now
displays the page size when printing the configuration results.

Users that want to override the page size in to choose a smaller value
can still do so with the --with-lg-pagesize configuration option.
2025-08-26 18:10:05 -07:00
Slobodan Predolac
88b29da00a [EASY][BUGFIX] Spelling and format 2025-08-23 21:47:21 -07:00
lexprfuncall
f890bbed4a Define malloc_{write,read}_fd as non-inline global functions
The static inline definition made more sense when these functions just
dispatched to a syscall wrapper.  Since they acquired a retry loop, a
non-inline definition makes more sense.
2025-08-23 11:42:23 -04:00
lexprfuncall
6ced4b5afb Handle interruptions and retries of read(2) and write(2) 2025-08-23 11:42:23 -04:00
lexprfuncall
5f06b14d1c Remove an orphaned comment
This was left behind when definitions of malloc_open and malloc_close
were abstracted from code that had followed.
2025-08-23 11:42:23 -04:00
Shirui Cheng
e2da7477f8 Revert PR #2608: Manually revert commits 70c94d..f9c0b5 2025-08-22 21:55:24 -07:00
lexprfuncall
9186700eb3 Fix the compilation check for process madvise
An include of unistd.h is needed to make the declaration of the
syscall function visible to the compiler.  The include of sys/mman.h
is not used at all.
2025-08-21 22:43:02 -07:00
Slobodan Predolac
4f126b47b3 Save and restore errno when calling process_madvise 2025-08-21 16:22:06 -07:00
lexprfuncall
acd85e5359 Remove an unused function and global variable
When the dehugify functionality was retired in an previous commit, a
dehugify-related function and global variable in a test was
accidentally left in-place causing builds that add -Werror to CFLAGS
to fail.
2025-08-21 08:59:37 -07:00
Slobodan Predolac
f9fae9f1f8 Experimental configuration option for fast path prefetch from cache_bin 2025-08-15 16:06:08 -07:00
lexprfuncall
c45b6223e5 Use relaxed atomics to access the process madvise pid fd
Relaxed atomics already provide sequentially consistent access to single
location data structures.
2025-08-13 18:33:27 -07:00
lexprfuncall
16c5abd1cd Do not dehugify when purging
Giving the advice MADV_DONTNEED to a range of virtual memory backed by
a transparent huge page already causes that range of virtual memory to
become backed by regular pages.
2025-08-13 18:31:50 -07:00
lexprfuncall
0d071a086b Fix several spelling errors in comments 2025-08-08 14:12:12 -07:00
Slobodan Predolac
97d25919c3 [process_madvise] Make init lazy so that python tests pass. Reset the pidfd on fork 2025-07-29 15:47:53 -07:00
Slobodan Predolac
fb52eac372 Add several USDT probes for hpa 2025-06-26 13:16:04 -07:00
Slobodan Predolac
9dd7b075b6 Add experimental support for usdt systemtap probes 2025-06-26 13:16:04 -07:00
guangli-dai
1c826c924a Ignore the clang-format changes in the git blame. 2025-06-22 09:21:44 -07:00
guangli-dai
34f359e0ca Reformat the codebase with the clang-format 18. 2025-06-20 14:35:15 -07:00
Shirui Cheng
ae4e729d15 Update the default value for opt_experimental_tcache_gc and opt_calloc_madvise_threshold 2025-06-17 13:25:20 -07:00
Guangli Dai
95f9f934ca Remove --enable-limit-usize-gap for cirrus CI since the config-time option is removed. 2025-06-16 11:16:40 -07:00
guangli-dai
f9e9e0986a Update appveyor settings. 2025-06-16 11:16:40 -07:00
dzhao.ampere
d58e311320 test/unit/psset.c: fix SIGSEGV when PAGESIZE is large
When hugepage is enabled and PAGESIZE is large, the test could
ask for a stack size larger than user limit. Allocating the
memory instead can avoid the failure.
2025-06-13 17:08:38 -07:00
Slobodan Predolac
390e70c840 [thread_event] Add support for user events in thread events when stats are enabled 2025-06-11 15:37:03 -07:00
Slobodan Predolac
b2a35a905f [thread_event] Remove macros from thread_event and replace with dynamic event objects 2025-06-11 15:37:03 -07:00
Qi Wang
1972241cd2 Remove unused options in the batched madvise unit tests. 2025-06-02 11:25:37 -07:00
Jason Evans
27d7960cf9 Revert "Extend purging algorithm with peak demand tracking"
This reverts commit ad108d50f1.
2025-06-02 10:44:37 -07:00
guangli-dai
edaab8b3ad Turn clang-format off for codes with multi-line commands in macros 2025-05-28 19:22:21 -07:00
guangli-dai
4531411abe Modify .clang-format to have declarations aligned 2025-05-28 19:22:21 -07:00
guangli-dai
1818170c8d Fix binshard.sh by specifying bin_shards for all sizes. 2025-05-28 19:21:49 -07:00
guangli-dai
fd60645260 Add one more check to double free validation. 2025-05-28 19:21:49 -07:00
Xin Yang
5e460bfea2 Refactor: use the cache_bin_sz_t typedef instead of direct uint16_t
any future changes to the underlying data type for bin sizes
(such as upgrading from `uint16_t` to `uint32_t`) can be achieved
by modifying only the `cache_bin_sz_t` definition.

Signed-off-by: Xin Yang <yangxin.dev@bytedance.com>
2025-05-22 10:43:33 -07:00
Xin Yang
9169e9272a Fix: Adjust CACHE_BIN_NFLUSH_BATCH_MAX size to prevent assert failures
The maximum allowed value for `nflush_batch` is
`CACHE_BIN_NFLUSH_BATCH_MAX`. However, `tcache_bin_flush_impl_small`
could potentially declare an array of `emap_batch_lookup_result_t`
of size `CACHE_BIN_NFLUSH_BATCH_MAX + 1`. leads to a `VARIABLE_ARRAY`
assertion failure, observed when `tcache_nslots_small_max` is
configured to 2048. This patch ensures the array size does not exceed
the allowed maximum.

Signed-off-by: Xin Yang <yangxin.dev@bytedance.com>
2025-05-22 10:27:09 -07:00
guangli-dai
f19a569216 Ignore formatting commit in blame. 2025-05-20 14:21:08 -07:00
Slobodan Predolac
b6338c4ff6 EASY - be explicit in non-vectorized hpa tests 2025-05-19 16:31:04 -07:00
guangli-dai
554185356b Sample format on tcache_max test 2025-05-19 15:06:13 -07:00
guangli-dai
3cee771cfa Modify .clang-format to make it more aligned with current freebsd style 2025-05-19 15:06:13 -07:00
Jiebin Sun
3c14707b01 To improve reuse efficiency, the maximum coalesced size for large extents
in the dirty ecache has been limited. This patch was tested with real
workloads using ClickHouse (Clickbench Q35) on a system with 2x240 vCPUs.
The results showed a 2X in query per second (QPS) performance and
a reduction in page faults to 29% of the previous rate. Additionally,
microbenchmark testing involved 256 memory reallocations resizing
from 4KB to 16KB in one arena, which demonstrated a 5X performance
improvement.

Signed-off-by: Jiebin Sun <jiebin.sun@intel.com>
2025-05-12 15:45:36 -07:00
guangli-dai
37bf846cc3 Fixes to prevent static analysis warnings. 2025-05-06 14:47:35 -07:00
guangli-dai
8347f1045a Renaming limit_usize_gap to disable_large_size_classes 2025-05-06 14:47:35 -07:00
Guangli Dai
01e9ecbeb2 Remove build-time configuration 'config_limit_usize_gap' 2025-05-06 14:47:35 -07:00
Slobodan Predolac
852da1be15 Add experimental option force using SYS_process_madvise 2025-04-28 18:45:30 -07:00
Slobodan Predolac
1956a54a43 [process_madvise] Use process_madvise across multiple huge_pages 2025-04-25 19:19:03 -07:00
Slobodan Predolac
0dfb4a5a1a Add output argument to hpa_purge_begin to count dirty ranges 2025-04-25 19:19:03 -07:00
Slobodan Predolac
cfa90dfd80 Refactor hpa purging to prepare for vectorized call across multiple pages 2025-04-25 19:19:03 -07:00
Qi Wang
a3910b9802 Avoid forced purging during thread-arena migration when bg thd is on. 2025-04-25 19:18:20 -07:00
guangli-dai
c23a6bfdf6 Add opt.limit_usize_gap to stats 2025-04-16 10:38:10 -07:00
guangli-dai
c20a63a765 Silence the uninitialized warning from clang. 2025-04-16 10:38:10 -07:00
Qi Wang
f81fb92a89 Remove Travis CI macOS configs (not supported anymore). 2025-04-14 15:27:38 -07:00
Slobodan Predolac
f19f49ef3e if process_madvise is supported, call it when purging hpa 2025-04-04 13:57:42 -07:00
Kaspar M. Rohrer
80e9001af3 Move `extern "C" specifications for C++ to where they are needed
This should fix errors when compiling C++ code with modules enabled on clang.
2025-03-31 10:41:51 -07:00