Memory spikes in Druid pods (especially Historical)

Apache Druid Q&A Ingestion
#1

Hello,

We monitor CPU and Memory usage with Prom/Grafana and we noticed some spikes in the JVM Memory Used. I’m trying to figure out what is the root cause of these spikes…

It goes sometimes over my -xmxlimit so I increased it but I want to be sure that that won’t increase again and again in the future.

Exemple of 4 spikes (~6Gb), while the memory is ~100/200Mb in normal time

image
image1235×404 86.1 KB

HOW can we explain that and secure these increases ?
Thks

#2

Have you tuned? You might also look at cleaning up your metadata.

#3

No, so far no tuning and concerning the cleaning metadata, we set up it yes.
@Mark_Herrera do you think that the +UseG1GC can solve the issue ?

#4

I’d certainly give it a try. Let us know how it goes.

#5

Using G1GC is absolutely a recommended best practice.

1 Like