Prepare pac and hpa psset for size to grow by PAGE over GROUP*PAGE

For PAC, to avoid having too many bins, arena bins still have the same
layout.  This means some extra search is needed for a page-level request that
is not aligned with the orginal size class: it should also search the heap
before the current index since the previous heap might also be able to
have some allocations satisfying it.  The same changes apply to HPA's
psset.

This search relies on the enumeration of the heap because not all allocs in
the previous heap are guaranteed to satisfy the request.  To balance the
memory and CPU overhead, we currently enumerate at most a fixed number
of nodes before concluding none can satisfy the request during an
enumeration.
This commit is contained in:
Guangli Dai 2025-01-31 14:14:46 -08:00
parent 205ba7b223
commit bffe921ba0
6 changed files with 298 additions and 12 deletions

View file

@ -2,8 +2,9 @@
#include "jemalloc/internal/ph.h"
#define BFS_ENUMERATE_MAX 30
typedef struct node_s node_t;
ph_structs(heap, node_t);
ph_structs(heap, node_t, BFS_ENUMERATE_MAX);
struct node_s {
#define NODE_MAGIC 0x9823af7e
@ -239,6 +240,22 @@ TEST_BEGIN(test_ph_random) {
expect_false(heap_empty(&heap),
"Heap should not be empty");
/* Enumerate nodes. */
heap_enumerate_helper_t helper;
uint16_t max_queue_size = sizeof(helper.bfs_queue)
/ sizeof(void *);
expect_u_eq(max_queue_size, BFS_ENUMERATE_MAX,
"Incorrect bfs queue length initialized");
assert(max_queue_size == BFS_ENUMERATE_MAX);
heap_enumerate_prepare(&heap, &helper,
BFS_ENUMERATE_MAX, max_queue_size);
size_t node_count = 0;
while(heap_enumerate_next(&heap, &helper)) {
node_count ++;
}
expect_lu_eq(node_count, j,
"Unexpected enumeration results.");
/* Remove nodes. */
switch (i % 6) {
case 0: