Java service memory usage increasing

Question

Our backend services are developed with java springboot framework. One of the large services when kept running, memory utilization increases slowly as shown in the below snapshot. When it reaches 95%, the service goes down without any error logging. It's 32GB memory.

Memory utilization

Heap and GC stats seems to be fine.

Heap Usage

GC Usage

I also looked at the memory dump and didn't find any native code leak.

The landing page of the application is a dashboard for which concurrent requests are sent to the backend. The backend has to do quite a bit of processing for the same. The sudden spike in memory happens when the data for processing is more and it never comes down after that. I've looked at heap dumps and couldn't find any leak suspects.

What else could be causing the spike in memory utilization?

"the service goes down without any error logging" You mean that the JVM exits? What's the exit status? Is there a core file? Have you tried -XX:+HeapDumpOnOutOfMemoryError? — tgdavies
– tgdavies, Commented Oct 22, 2021 at 6:44
you need to analyze what data structures or collections occupy or consume more memory space during the spike and before the spike. try taking a heap dump before spike happens and take another heap dump after spike. — vaibhavsahu
– vaibhavsahu, Commented Oct 22, 2021 at 6:50
@ScaryWombat the memory utilization reaches 95% and the service becomes unresponsive. — mohan p
– mohan p, Commented Oct 22, 2021 at 7:07
@vaibhavsahu from my observation type:byte[] object is only taking some space. But it doesn't look like a reason for the increase in memory usage. — mohan p
– mohan p, Commented Oct 22, 2021 at 7:17

mohan p · Accepted Answer · 2021-11-19 13:17:50Z

1

Solution: Moving from G1 GC to Shenandoah GC (low pause GC) has helped us to address the memory issue for now. After the change. memory utilization looks like this.

Command usage e.g. java -XX:+UseShenandoahGC -XX:+UnlockExperimentalVMOptions -XX:ShenandoahUncommitDelay=1000 -XX:ShenandoahGuaranteedGCInterval=10000 -jar app.jar

and, you can read about how this works here https://dzone.com/articles/java-garbage-collection-3 or https://wiki.openjdk.java.net/display/shenandoah/Main

edited Nov 19, 2021 at 13:17

answered Nov 19, 2021 at 7:29

mohan p

411 silver badge4 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

josejuan · Accepted Answer · 2021-10-22 06:37:12Z

0

If the application is working properly it is not a problem, only that the garbage collector is not performing until 80% is reached. In fact, the graph with "GC New Gen Size" shows the typical "sawtooth" which is completely normal. In any case, you can adjust how the garbage collector will work.

For example: https://sematext.com/blog/java-garbage-collection-tuning/

In this chart, you can see something called “shark tooth”. Usually, it is a sign of a healthy JVM heap.

answered Oct 22, 2021 at 6:37

josejuan

9,60628 silver badges32 bronze badges

Collectives™ on Stack Overflow

Java service memory usage increasing

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related