Alekos Filini

Reducing Tor's memory usage

I recently migrated my Tor relay from a Fedora 32 container to a NixOS container. For some reason, it magically stared using a lot less CPU, but also a bit more memory.

[archived]

After a few days it was clear that it was struggling with the memory, I could see a few big drops in memory usage where the kernel swapped out a lot of pages.

In the official docs1 docs they recommend recompiling with a different malloc implementation (which they also use in the official release tarballs), but being lazy I didn’t really want to spend time configuring NixOS to compile the package locally with non-standard flags. So I decided to try the quick route first: changing the malloc implementation at runtime.

NixOS supports a few different memory allocators2. Most of them are designed to aid development, by trying to help finding undefined behaviors in the usage of allocated memory. There is one provider that can help us, though: jemalloc.

jemalloc3 is designed to reduce memory fragmentation, which is very likely the cause of the high memory usage of my relay, since it doesn’t seem to be connected to the bandwidth provided but just slowly grows over time.

So, I tried adding environment.memoryAllocator.provider = "jemalloc" to my configuration, restarting Tor, and…

First 24 hours of Tor running with the default libc (glibc)
First 24 hours of Tor running with the default libc (glibc)
First 24 hours of Tor running with jemalloc
First 24 hours of Tor running with jemalloc

  1. Why is my Tor relay using so much memory? | Tor Project | Support [original] [archived] ↩︎

  2. nixpkgs/malloc.nix at nixos-21.05 · NixOS/nixpkgs · GitHub [original] [archived] ↩︎

  3. jemalloc [original] [archived] ↩︎

#jemalloc #libc #relay #tor