LAG Between Kafka and Realtime

Hi All

i have a version of Druid 0.9.2 with a different topic

i have a particular topic that gets more traffic of the others

when this topic starts to get more traffic i have a huge LAG between Kafka and Realtime.

i tried to increase also the hardware as CPU and RAM but nothing change

also, the load average and performance of the server seems fine

i don’t see the server under stress before of the update and now

i don’t understand what’s going on

Any help?

Thanks in advance

Is this the Kafka indexing service? You might need to increase your “taskCount” a bit.

Will also depend on the number of partitions in the topic ? for example if there is a single partition task count increase will not help ?

Yeah, increasing taskCount beyond the number of Kafka partitions won’t help.