JVM Metrics for Peons

Hello everyone,

we are currently using Druid 0.12.3 and the statsd-emitter for our metrics. I was wondering if there is a possibility to show jvm metrics for every single peon? I can only find one metric for “jvm heap max” and “jvm heap used” in Graphite (see attached image). Is this an average over every Peon?

Our configuration for Middlemanager:

runtime.properties:

General settings

druid.service=druid/middleManager

druid.port=8091

MiddleManager-specific settings

druid.worker.capacity=5

Resources for peons

druid.indexer.task.baseTaskDir=var/druid/task

druid.indexer.task.restoreTasksOnRestart=true

druid.indexer.runner.javaOpts=-server -Xmx3g -XX:+UseG1GC -XX:MaxGCPauseMillis=100

Peon Properties

druid.indexer.fork.property.druid.processing.buffer.sizeBytes=536870912

druid.indexer.fork.property.druid.processing.numThreads=2

#Monitoring

druid.monitoring.monitors=[“io.druid.java.util.metrics.JvmMonitor”]

``

jvm.config:

-server

-Xms64m

-Xmx64m

-XX:+UseConcMarkSweepGC

-Duser.timezone=UTC

-Dfile.encoding=UTF-8

-Djava.io.tmpdir=var/tmp

-Djava.util.logging.manager=org.apache.logging.log4j.jul.LogManager

``

Thanks,

Stephan

They all should be emitting under different “host” keys (same ip, different ports). I’m not sure if something specific to the statsd emitter is filtering them out (I’m not familiar with it) but in general they do all emit metrics individually.

Gian

Hi Gian,

thanks for the fast reply.

Its good to know that the metrics should be there usually. I will try to tweak the statsd-emitter a little bit. If i find out the solution I will post it here. So others could use it as well.

Have a nice day and thanks again,

Stephan

Hi Gian,

i tried to tweak the the statsd emitter by adding more dimensions but without success.

For testing I switched to the logging-emitter but the peons dont get their own jvm metrics (see logging.txt).

Settings for the logging-emitter:

Monitoring

druid.emitter=logging

druid.emitter.logging.logLevel=info

``

After that I switched to graphite-emitter but with the same outcome. No jvm metrics for my three peons are collected (see graphite.png).

Settings for the graphite-emitter:

Monitoring

druid.emitter=graphite

druid.emitter.logging.logLevel=info

druid.emitter.graphite.hostname=graphite

druid.emitter.graphite.port=2004

druid.emitter.graphite.eventConverter={“type”:“all”, “namespacePrefix”: “druid.test”, “ignoreHostname”:false, “ignoreServiceName”:false}

``

Which kind of emitter are you using to get the metrics?

Best wishes,

Stephan

logging.txt (21.7 KB)

Hi,
Stephan, did you manage to send graphite metrics?
I experience the same issue the graphite emitter doesn’t send metrics at all.
I’m running with Druid 0.14

Thx!