Druid Kafka indexing service

Hi All,

I am new to Druid. We have druid installed on single machine with default configurations, and trying to load data to Druid from Kafka. We are able to load data to druid using the supervisor spec submitted, but we face intermittent task failures with following errors

  1. Error while loading multiple records(one at a time, but trying to load many records in a timeframe to asses the performance) org.apache.druid.indexing.seekablestream.SeekableStreamIndexTaskRunner - Encountered exception while running task.
    org.apache.kafka.common.errors.InterruptException: java.lang.InterruptedException

  2. Caused by: java.lang.OutOfMemoryError’. This error happens very frequently and we could see plenty of failed index_kafka tasks in overlord console.

Could you please share some insight on how to fix these errors. Please advise if additional information that will help the analysis

What is your MiddleManager heap size? This means that there’s not enough heap to keep the records being index in memory.

Rommel Garcia

Hi Rommel,

Thank you for the quick reply. Are you looking for jvm config? I am sharing the jvm prperties for all Service

middleManager: jvm.config

If your peons are inheriting the heap config from MM, then the 64MB is small. Try increasing it to 1GB since i’m not sure how big your single node cluster is.

Rommel Garcia

Seconded.

Also, this may be helpful: https://druid.apache.org/docs/latest/operations/recommendations.html

Particularly the FAQs about memory: https://druid.apache.org/docs/latest/operations/performance-faq.html

Hi Rommel,

RAM info

free -m

total used free shared buff/cache available

Mem: 31993 8719 3851 2187 19422 20629

Swap: 4095 708 3387

Cool. You have about 32GB of RAM total. Lots of room to increase more heap if needed for MM.

Rommel Garcia

Thank you. We will try allocating more heap.