Measuring Druid Ingestion Latency with Kafka connector

Relates to Apache Druid 0.21.0

We are considering Druid as a datastore for storing transaction events in our company. These events are currently being stored in Kafka as JSON objects. We want to consume and ingest these Kafka messages into Druid using the native Kafka connector. Is there a way to measure the write performance of the Druid Kafka consumer in terms of -

  • bytes ingested per second
  • messages ingested per second
  • average/percentile latency of Kafka message ingestion

Thank you!

The rule of thumb is that Druid can ingest 10K records / sec / core. I think you can get a lot of what you are looking for here: