mirror of
https://github.com/jemalloc/jemalloc.git
synced 2026-05-18 10:56:20 +03:00
118 lines
3.5 KiB
Markdown
118 lines
3.5 KiB
Markdown
# Page Allocator (PA) Microbenchmark Suite
|
|
|
|
This directory contains a comprehensive microbenchmark suite for testing and analyzing jemalloc's Page Allocator (PA) system, including the Hugepage-aware Page Allocator (HPA) and Slab Extent Cache (SEC) components.
|
|
|
|
## Overview
|
|
|
|
The PA microbenchmark suite consists of two main programs designed to preprocess allocation traces and replay them against jemalloc's internal PA system to measure performance, memory usage, and allocation patterns.
|
|
|
|
To summarize how to run it, assume we have a file `test/stress/pa/data/hpa.csv` collected from a real application using USDT, the simulation can be run as follows:
|
|
```
|
|
make tests_pa
|
|
./test/stress/pa/pa_data_preprocessor hpa test/stress/pa/data/hpa.csv test/stress/pa/data/sample_hpa_output.csv
|
|
./test/stress/pa/pa_microbench -p -o test/stress/pa/data/sample_hpa_stats.csv test/stress/pa/data/sample_hpa_output.csv
|
|
```
|
|
|
|
If it's sec, simply replace the first parameter passed to `pa_data_preprocessor` with sec.
|
|
|
|
## Architecture
|
|
|
|
### PA System Components
|
|
|
|
The Page Allocator sits at the core of jemalloc's memory management hierarchy:
|
|
|
|
```
|
|
Application
|
|
↓
|
|
Arena (tcache, bins)
|
|
↓
|
|
PA (Page Allocator) ← This is what we benchmark
|
|
├── HPA (Hugepage-aware Page Allocator)
|
|
└── SEC (Slab Extent Cache)
|
|
↓
|
|
Extent Management (emap, edata)
|
|
↓
|
|
Base Allocator
|
|
↓
|
|
OS (mmap/munmap)
|
|
```
|
|
|
|
### Microbenchmark Architecture
|
|
|
|
```
|
|
Raw Allocation Traces
|
|
↓
|
|
[pa_data_preprocessor] ← Preprocesses and filters traces
|
|
↓
|
|
CSV alloc/dalloc Files
|
|
↓
|
|
[pa_microbench] ← Replays against real PA system
|
|
↓
|
|
Performance Statistics & Analysis
|
|
```
|
|
|
|
## Programs
|
|
|
|
### 1. pa_data_preprocessor
|
|
|
|
A C++ data preprocessing tool that converts raw allocation traces into a standardized CSV format suitable for microbenchmarking.
|
|
|
|
**Purpose:**
|
|
- Parse and filter raw allocation trace data
|
|
- Convert various trace formats to standardized CSV
|
|
- Filter by process ID, thread ID, or other criteria
|
|
- Validate and clean allocation/deallocation sequences
|
|
|
|
### 2. pa_microbench
|
|
|
|
A C microbenchmark that replays allocation traces against jemalloc's actual PA system to measure performance and behavior with HPA statistics collection.
|
|
|
|
**Purpose:**
|
|
- Initialize real PA infrastructure (HPA, SEC, base allocators, emaps)
|
|
- Replay allocation/deallocation sequences from CSV traces
|
|
- Measure allocation latency, memory usage, and fragmentation
|
|
- Test different PA configurations (HPA-only vs HPA+SEC)
|
|
- Generate detailed HPA internal statistics
|
|
|
|
**Key Features:**
|
|
- **Real PA Integration**: Uses jemalloc's actual PA implementation, not simulation
|
|
- **Multi-shard Support**: Tests allocation patterns across multiple PA shards
|
|
- **Configurable Modes**: Supports HPA-only mode (`-p`) and HPA+SEC mode (`-s`)
|
|
- **Statistics Output**: Detailed per-shard statistics and timing data
|
|
- **Configurable Intervals**: Customizable statistics output frequency (`-i/--interval`)
|
|
|
|
## Building
|
|
|
|
### Compilation
|
|
|
|
```bash
|
|
# Build both PA microbenchmark tools
|
|
cd /path/to/jemalloc
|
|
make tests_pa
|
|
```
|
|
|
|
This creates:
|
|
- `test/stress/pa/pa_data_preprocessor` - Data preprocessing tool
|
|
- `test/stress/pa/pa_microbench` - PA microbenchmark
|
|
|
|
## Usage
|
|
|
|
### Data Preprocessing
|
|
|
|
```bash
|
|
# Basic preprocessing
|
|
./test/stress/pa/pa_data_preprocessor <hpa/sec> input_trace.txt output.csv
|
|
```
|
|
|
|
### Microbenchmark Execution
|
|
|
|
```bash
|
|
# Run with HPA + SEC (default mode)
|
|
./test/stress/pa/pa_microbench -s -o stats.csv trace.csv
|
|
|
|
# Run with HPA-only (no SEC)
|
|
./test/stress/pa/pa_microbench -p -o stats.csv trace.csv
|
|
|
|
# Show help
|
|
./test/stress/pa/pa_microbench -h
|
|
```
|