System cache constantly grows when stress testing backend app

Question

I've been stress testing my .NET application that uses PostgreSQL to find memory leaks. The app is very stable memory-wise — memory usage only depends on the number of concurrent connections.

However, I’ve noticed that the RAM usage marked as cache is slowly growing over time until it fills up, causing a RAM shortage (the cache is not being released).

I’m testing my app by making 1000 concurrent POST requests to the backend, which is configured with a pool of up to 100 PostgreSQL connections. PostgreSQL has shared_buffers set to 128MB and a maximum of 100 connections.

I haven’t found any high memory usage from the .NET process or from my simple Node app that I’m using for the tests.

The test simply makes many concurrent HTTPS connections: The test is just a simple https client trying to make many concurrent connections.

while (true) {
  await stressREST(token, url, msg, 1000);
}

const httpsAgent = new https.Agent({
  keepAlive: true,
  maxSockets: 100,
  maxTotalSockets: 100,
});

const client = axios.create({
  httpsAgent,
  timeout: 10_000,
});

export default async function stressREST(
  token: string,
  url: string,
  content: object,
  times: number
) {
  const promises = [];

  for (let i = 0; i < times; i++) {
    promises.push(
      client.post(url, content, {
        headers: { Authorization: `Bearer ${token}` },
      })
    );
  }

  await Promise.all(promises);
  console.log(times, "requests done");
}

I added the HTTPS client because I thought the backend wasn’t keeping up with the connections and Node or the server was caching the requests, but it didn’t help. When I stop the Node process, the high cache usage still persists in RAM even after stopping my backend. Using SetInterval() instead of while true loop didn't help either.

I also set the kernel shmmax value to 8GB, but it didn’t make any difference.

I use Fedora 43 with 32GB of RAM.

You have a potentially memory consumption problem in .NET app but the only code we see is JS one. I would argue that this is far from being a minimal reproducible example =) Have you tried profiling your .NET app? — Guru Stron
– Guru Stron, Commented Nov 1 at 14:19
@GuruStron Yes, I've been profiling my .NET app everytime I ran the tests, the memory stays flat at around 400MB with the setup I showed, memory usage scales with more conns but it's being actively GCed and it stays flat. .NET process itself capped at 4.5GB but has also been shrinking when the memory shortage was imminent. There is no info in Fedora's task manager what is consuming so much memory aside of saying that cache takes like 20GB. It's hard to provide minimal reproducible example with my backend but the route of this POST request is just a basic CRUD with EF and DI'ed services — pkrzysiek
– pkrzysiek, Commented Nov 1 at 16:06

Mina Golzari Dalir · Accepted Answer · 2025-11-15 19:22:31Z

0

This behavior is normal Linux kernel optimization, not a memory leak the kernel aggressively caches I/O operations from your stress test in unused RAM to improve performance, and under sustained artificial load this cache accumulates until it consumes most available memory. To resolve this, you can either manually clear the cache using echo 3 > /proc/sys/vm/drop_caches as a temporary measure, or permanently adjust the kernel's cache reclaim aggressiveness by setting vm.vfs_cache_pressure = 1000in /etc/sysctl.conf to make the system more responsive to memory pressure from applications.

answered Nov 15 at 19:22

Mina Golzari Dalir

676 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

System cache constantly grows when stress testing backend app

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related