Monitor and log broker queries

My team is currently looking for a way to monitor query frequency to the broker, and log the queries made so we know which metrics and queries are “hot”, and which are “not”.

Question 1: Monitoring

We are using the graphite-emitter extension (which is community-maintained), so it may be a limitation of the extension currently, but does anyone know of a way to easily monitor these metrics using graphite and grafana?

Question 2: Grafana with Graphite and Broker query monitoring

We do have some logs being shipped, and this may extend question 1 a bit. Does anyone know how to not have these raw queries populate the dropdown on grafana, or somehow aggregate them to get metrics about query times (maybe using graphite aggregation)?

Question 3: Logging

We are also using the ELK stack for logging, and are wondering if there is a way to log the broker’s raw queries for further analysis. We currently have the indexer’s logs being send to Logstash correctly, but don’t see any documentation or way to log the broker’s queries.

For graphite emitter, I believe you can configure it to emit more dimensions by including more dimensions about the query by specifying them in the whitelist configuration as specified here - http://druid.io/docs/latest/development/extensions-contrib/graphite.html

Apart from the metrics, druid also supports emitting request logs which contains complete query and the total time it took on each node,

related configs for request logs are documented here -

http://druid.io/docs/latest/configuration/index.html

After enabling request logs, you can ingest them in the ELK stack for further analysis.