`tsd_get`/`tsd_set` took the tsd address as a plain `&tsd_tls` -- `thread_pointer
+ const_offset`, which the compiler treats as loop-invariant. Under
whole-program LTO it is hoisted out of the inlined malloc/free and kept in a
callee-saved register across a user-space context switch (swapcontext,
boost::context fibers); resumed on a different OS thread, the cached tsd/tcache
still belongs to the previous thread and the two race -- heap corruption that
reproduces only under LTO and is invisible to sanitizers. The write side is
worse: tsd_set (from tsd_fetch_slow, on a thread's first allocation after
migration) would copy the new thread's tsd over the previous thread's live one
and register the wrong cleanup key.
Route tsd_get/tsd_set in every GNU backend through JEMALLOC_TLS_ADDR(tsd_tls): a
per-variable accessor that takes the address inside a noinline `memory`-barrier
function, opaque to the optimizer. Correct for every TLS model, one call per
access; MSVC keeps the plain address.
The accessor is declared in every TU (JEMALLOC_TLS_ADDR_DECLARE) but defined
once in src/tsd.c (JEMALLOC_TLS_ADDR_DEFINE, under JEMALLOC_TSD_C_): a per-TU
`static` body is emitted at -O0 even when unused, pulling an undefined reference
to the internal thread-local into the integration-test util objects, which link
only the public shared library. DECLARE/DEFINE are invoked directly, never
forwarded, so `##tlsvar` pastes the literal name rather than the expanded
`je_tsd_tls`.
arena_types.h + arena_structs.h + arena_externs.h merged into arena.h,
keeping the three logical sections (TYPES / STRUCTS / EXTERNS) with
explicit dividers. arena_inlines_a.h and arena_inlines_b.h stay
separate; arena_inlines_b.h now carries a comment explaining why
merging the two would reintroduce a real #include cycle through
tcache_inlines.h -> arena_choose (the asymmetric cycle-breaker).
arena_decay_constants.h (new): minimal header for
ARENA_DECAY_NTICKS_PER_UPDATE.
Both components had a four-way split (_types, _structs, _externs,
_inlines) that predates explicit per-file includes. With the edata
<-> prof_types coupling broken in the prior commit, merging _types +
_structs + _externs no longer risks an include cycle.
- prof.h replaces prof_types.h + prof_structs.h + prof_externs.h.
- tcache.h replaces tcache_types.h + tcache_structs.h + tcache_externs.h.
prof_inlines.h and tcache_inlines.h stay separate: prof_inlines.h
sits at the bottom of the dependency layering, and tcache_inlines.h's
include of arena_externs.h is the asymmetric cycle-breaker that keeps
the arena <-> tcache symbol cycle from becoming an include cycle.
Fold historical *_types/_structs/_externs/_inlines splits where the
layering is no longer load-bearing.
- large_externs.h -> large.h: rename; it was a single-purpose
function-prototype file.
- background_thread_structs.h + background_thread_externs.h ->
background_thread.h: merge. background_thread_inlines.h stays
separate because it depends on arena_inlines_a.h.
- bin_types.h -> bin.h: BIN_SHARDS_MAX / N_BIN_SHARDS_DEFAULT join the
bin_t struct and the bin_dalloc_locked_info_t type. bin_inlines.h
stays separate so TUs that only need the bin_t type don't pull in
the tcache-flush inline closure. The two bin_stats_* helpers move
from bin.h into bin_inlines.h so all bin inlines live together.
- tsd_binshards.h (new): houses tsd_binshards_t and its zero
initializer. Lets tsd_internals.h pull it in for X-macro expansion
without dragging mutex.h -- mutex.h depends on TSD machinery, which
would form a cycle through bin.h.
jemalloc_internal_includes.h drops the references to the deleted/
merged headers.
Header files are now self-contained, which makes the relationships
between the files clearer, and crucially allows LSP tools like `clangd`
to function correctly in all of our header files. I have verified that
the headers are self-contained (aside from the various Windows shims) by
compiling them as if they were C files – in a follow-up commit I plan to
add this to CI to ensure we don't regress on this front.
This is a prerequisite to achieving self-contained headers. Previously,
the various tsd implementation headers (`tsd_generic.h`,
`tsd_tls.h`, `tsd_malloc_thread_cleanup.h`, and `tsd_win.h`) relied
implicitly on being included in `tsd.h` after a variety of dependencies
had been defined above them. This commit instead makes these
dependencies explicit by splitting them out into a separate file,
`tsd_internals.h`, which each of the tsd implementation headers includes
directly.
