mirror of
https://github.com/jemalloc/jemalloc.git
synced 2026-07-20 15:47:17 +03:00
7aa77c8e2a
The noinline accessor puts an opaque call on the malloc/free fastpath.
test/stress/microbench (malloc(1)/free pairs, pinned core, clang):
unpatched accessor this commit
malloc, no LTO 7.0 ns/op 9.8 (+40%) 7.0 (+-0%)
free, no LTO 6.5 ns/op 9.2 (+40%) 6.5 (+-0%)
malloc, ThinLTO 7.0 ns/op 9.0 (+28%) 7.6 (~+8%)
free, ThinLTO 6.5 ns/op 8.4 (+28%) 7.0 (~+8%)
Under a static TLS model the tsd address is `thread_pointer + offset` with a
thread-independent offset. Capture the offset once (a noinline helper, lazy and
sentinel-initialized) and re-read the thread pointer per call with a `volatile`
asm the optimizer may not hoist or CSE (per-arch reads from mimalloc's
mi_prim_tls_slot). Both inputs are hoist-proof: the tp read is volatile, the
offset is thread-independent.
The offset global is read and written with relaxed atomics: threads racing
their first allocation init it concurrently with the same value, so a plain
access would be a benign-but-UB data race (TSan-reported). Relaxed adds no
fence -- it compiles to the same load/store on the fast path.
The offset must NOT be computed inline as `&tsd_tls - __builtin_thread_pointer()`:
the terms hoist independently, and clang ThinLTO keeps the stale `&tsd_tls` in a
callee-saved register while re-reading the thread pointer, so `fresh_tp +
(stale_addr - fresh_tp)` cancels back to the stale address. Capturing behind a
call boundary evaluates both terms at one point on one thread.
Gated on JEMALLOC_TLS_MODEL_INITIAL_EXEC (a new configure define, set whenever
jemalloc applies its default initial-exec model), GNU asm, a known arch, and
!_WIN32 (MinGW's thread pointer lives in the TEB, not fs/gs:0); everything else
keeps the noinline accessor. Verified with the fiber-migration reproducer under
clang ThinLTO.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|---|---|---|
| .. | ||
| jemalloc | ||
| msvc_compat | ||