Druid Kafka indexing service

Apache Druid Q&A Google Groups
#1

Hi All,

I am new to Druid. We have druid installed on single machine with default configurations, and trying to load data to Druid from Kafka. We are able to load data to druid using the supervisor spec submitted, but we face intermittent task failures with following errors

  1. Error while loading multiple records(one at a time, but trying to load many records in a timeframe to asses the performance) org.apache.druid.indexing.seekablestream.SeekableStreamIndexTaskRunner - Encountered exception while running task.
    org.apache.kafka.common.errors.InterruptException: java.lang.InterruptedException

  2. Caused by: java.lang.OutOfMemoryError’. This error happens very frequently and we could see plenty of failed index_kafka tasks in overlord console.

Could you please share some insight on how to fix these errors. Please advise if additional information that will help the analysis

#2

What is your MiddleManager heap size? This means that there’s not enough heap to keep the records being index in memory.

Rommel Garcia

#3

Hi Rommel,

Thank you for the quick reply. Are you looking for jvm config? I am sharing the jvm prperties for all Service

middleManager: jvm.config

#4

If your peons are inheriting the heap config from MM, then the 64MB is small. Try increasing it to 1GB since i’m not sure how big your single node cluster is.

Rommel Garcia

#5

Seconded.

Also, this may be helpful: https://druid.apache.org/docs/latest/operations/recommendations.html

Particularly the FAQs about memory: https://druid.apache.org/docs/latest/operations/performance-faq.html

#6

Hi Rommel,

RAM info

free -m

total used free shared buff/cache available

Mem: 31993 8719 3851 2187 19422 20629

Swap: 4095 708 3387

#7

Cool. You have about 32GB of RAM total. Lots of room to increase more heap if needed for MM.

Rommel Garcia

#8

Thank you. We will try allocating more heap.