Monitoring JVM usage with Graphite

Hi,

We are using Druid cluster in production. On seeing lot of OutOfMemory errors on broker nodes, we turned on monitoring through Graphite and Grafana.

Here is the broker jvm config -

-server

-Xmx15g

-Xms15g

-XX:NewSize=6g

-XX:MaxNewSize=6g

-XX:MaxDirectMemorySize=20g

-XX:+UseConcMarkSweepGC

-XX:+PrintGCDetails

-XX:+PrintGCTimeStamps

-XX:+HeapDumpOnOutOfMemoryError

-Duser.timezone=UTC

-Dfile.encoding=UTF-8

-Djava.util.logging.manager=org.apache.logging.log4j.jul.LogManager

-Djava.io.tmpdir=/tmp/druid

-Dcom.sun.management.jmxremote

-Dcom.sun.management.jmxremote.port=17071

-Dcom.sun.management.jmxremote.local.only=false

-Dcom.sun.management.jmxremote.authenticate=false

-Dcom.sun.management.jmxremote.ssl=false

But the charts in Graphite display completely different values -

JMX monitor seems to show correct values though.

I dont know if i am using wrong configuration or wrong metrics. I want metrics emitted by Broker through graphite to be same as JMX values. This will help us schedule right alerts and take action on it.

Thanks,

Chaitanya