mirror of
https://github.com/jemalloc/jemalloc.git
synced 2026-08-04 17:36:23 +03:00
Add an inline fast path to JEMALLOC_TLS_ADDR for static TLS
The noinline accessor puts an opaque call on the malloc/free fastpath.
test/stress/microbench (malloc(1)/free pairs, pinned core, clang):
unpatched accessor this commit
malloc, no LTO 7.0 ns/op 9.8 (+40%) 7.0 (+-0%)
free, no LTO 6.5 ns/op 9.2 (+40%) 6.5 (+-0%)
malloc, ThinLTO 7.0 ns/op 9.0 (+28%) 7.6 (~+8%)
free, ThinLTO 6.5 ns/op 8.4 (+28%) 7.0 (~+8%)
Under a static TLS model the tsd address is `thread_pointer + offset` with a
thread-independent offset. Capture the offset once (a noinline helper, lazy and
sentinel-initialized) and re-read the thread pointer per call with a `volatile`
asm the optimizer may not hoist or CSE (per-arch reads from mimalloc's
mi_prim_tls_slot). Both inputs are hoist-proof: the tp read is volatile, the
offset is thread-independent.
The offset global is read and written with relaxed atomics: threads racing
their first allocation init it concurrently with the same value, so a plain
access would be a benign-but-UB data race (TSan-reported). Relaxed adds no
fence -- it compiles to the same load/store on the fast path.
The offset must NOT be computed inline as `&tsd_tls - __builtin_thread_pointer()`:
the terms hoist independently, and clang ThinLTO keeps the stale `&tsd_tls` in a
callee-saved register while re-reading the thread pointer, so `fresh_tp +
(stale_addr - fresh_tp)` cancels back to the stale address. Capturing behind a
call boundary evaluates both terms at one point on one thread.
Gated on JEMALLOC_TLS_MODEL_INITIAL_EXEC (a new configure define, set whenever
jemalloc applies its default initial-exec model), GNU asm, a known arch, and
!_WIN32 (MinGW's thread pointer lives in the TEB, not fs/gs:0); everything else
keeps the noinline accessor. Verified with the fiber-migration reproducer under
clang ThinLTO.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
7b9853c4a1
commit
7aa77c8e2a
3 changed files with 79 additions and 13 deletions
|
|
@ -2841,6 +2841,8 @@ if test "x${je_cv_tls_model}" = "xyes" -a \
|
|||
AC_DEFINE([JEMALLOC_TLS_MODEL],
|
||||
[__attribute__((tls_model("initial-exec")))],
|
||||
[ ])
|
||||
AC_DEFINE([JEMALLOC_TLS_MODEL_INITIAL_EXEC], [ ],
|
||||
[Defined when the TSD thread-locals use the initial-exec (static) TLS model, i.e. live at a fixed offset from the thread pointer.])
|
||||
else
|
||||
AC_DEFINE([JEMALLOC_TLS_MODEL], [ ], [ ])
|
||||
fi
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue