Document the adverse effects of delayed memory return. See #2688.

Also document the surprising behaviour that the times configured
`dirty_decay_ms`/`muzzy_decay_ms` have no effect unless
`background_thread:true` is given if the process is sleeping.

I was very surprised when I had a process waiting for user
input sitting around at 100 GB RES memory,
and `dirty_decay_ms` not returning the memory after the default
10 seconds, but `dirty_decay_ms:0` fixing the problem immediately.
I was even more confused that changing the time configured by
`dirty_decay_ms` seemed to be completely ignored by jemalloc,
until I got the explanation from
https://github.com/jemalloc/jemalloc/issues/2688#issuecomment-2464775694

Personally I think the default-on of `dirty_decay_ms` is questionable,
as I encountered the problem documented here with
"This can lead to out-of-memory crashes ..."
where my program needed 100 GB for some processing, freed it,
and then invoked a child process that also needed 100 GB RAM;
the parent program continued to sit at 100 GB RES usage
and thus was OOM-killed, creating a bug where none should be
(as the parent process freed all memory exactly right).

This commit at least documents the impact of this questionable default.
This commit is contained in:
Niklas Hambüchen 2025-03-26 23:20:46 +01:00
parent 3688dfb5c3
commit 7b01e60b98
2 changed files with 77 additions and 3 deletions

View file

@ -43,6 +43,11 @@ Runtime options can be set via
between CPU and memory usage. Shorter decay time purges unused pages faster
to reduces memory usage (usually at the cost of more CPU cycles spent on
purging), and vice versa.
Be aware that decay times `> 0` will not be honored until the next relevant call
into jemalloc, unless you also enable `background_thread:true`.
Without `background_thread:true`, processes that are sleeping
(e.g. because they call `sleep()`, block on user input, network/file activity,
or run subprocesses) will not purge memory.
Suggested: tune the values based on the desired trade-offs.