Feature: benchmark memory usage with jemalloc

[romanz]

Following https://twitter.com/dpc_pw/status/1856972589140152700

[romanz]

@dpc would it please be possible to share the commits being used in https://twitter.com/dpc_pw/status/1857090601042198921?

[dpc]

I'm still working and investigating, but I'm currently testing: fedimint/fedimint#6354

[romanz]

When running with jemalloc (#1115), there is a ~10% memory usage improvement (right-side plot vs. left-side):

Tested with Electrum client loading the 1BitcoinEaterAddressDontSendf59kuE address (with >3.8k txs).

[antonilol]

10% seems far off from only using half the memory, sure, electrs itself also uses memory, but does #1115 also change the allocator used by rocksdb? Setting the global allocator in Rust will use it for Rust's heap allocated data structures (Vec, HashMap, BTreeMap, etc), but i am unsure about if this will cause rocksdb to also use it. (see facebook/rocksdb#12364, rust-rocksdb/rust-rocksdb#863)

[dpc]

@antonilol I think you're right and actually enabling jemalloc for rocksdb requires some work. In Fedimint it seems we're hitting some weird patterns around iterator usage on the Rust side where jemalloc helps significantly. I need to investigate enabling jemalloc for rocksdb itself and will share and follow this thread with anyone else trying the same thing.