Kafka indexing service - how to troubleshoot?

I have a problem using Kafka indexing service , and need at least a general advice how too look for a clue what is wrong.

So here is the problem, it is pretty simple:

I have loaded kafka-indexinxing-service extension, submitted Supervisor spec and successfully created a Supervisor.

As far as I can conclude from coordinator and overlord logs the extension was loaded, Kafka consumer created, has discovered and connected to the right topic (please see at the bottom of the message). I have verified twice (using Kafka command-line consumer) that messages arrive to the topic.

However nothing happens - there are no running tasks and (obviously) no data source created, just nothing happens. I do not see any errors or warnings in the logs, and in general my cluster does work - it can ingest data in batch mode using same data schema with no problem.

So everything look just fine but does not work, and I am out of directions for troubleshooting. I am new with Druid and almost for sure doing something wrong, but have no idea how to debug it… Any advice, hint, etc. will be highly appreciated.

Thanks in advance,

Lev

2016-08-04T15:05:47,817 INFO [Curator-LeaderSelector-0] org.apache.kafka.clients.consumer.ConsumerConfig - ConsumerConfig values:
value.deserializer = class org.apache.kafka.common.serialization.ByteArrayDeserializer
group.id = kafka-supervisor-hjdjccdf
partition.assignment.strategy = [org.apache.kafka.clients.consumer.RangeAssignor]
key.deserializer = class org.apache.kafka.common.serialization.ByteArrayDeserializer
2016-08-04T15:05:47,882 INFO [Curator-LeaderSelector-0] org.apache.kafka.common.utils.AppInfoParser - Kafka version : 0.9.0.1
2016-08-04T15:05:47,882 INFO [Curator-LeaderSelector-0] org.apache.kafka.common.utils.AppInfoParser - Kafka commitId : 23c69d62a0cabf06
2016-08-04T15:05:47,884 INFO [Curator-LeaderSelector-0] io.druid.indexing.kafka.supervisor.KafkaSupervisor - Started KafkaSupervisor[adjesty_st1], first run in [PT5S]
2016-08-04T15:05:53,031 INFO [KafkaSupervisor-adjesty_st1-0] io.druid.indexing.kafka.supervisor.KafkaSupervisor - New partition [0] discovered for topic [aggregator], adding to task group [0]
2016-08-04T15:05:53,031 INFO [KafkaSupervisor-adjesty_st1-0] io.druid.indexing.kafka.supervisor.KafkaSupervisor - New partition [1] discovered for topic [aggregator], adding to task group [0]

2016-08-04T15:05:53,032 INFO [KafkaSupervisor-adjesty_st1-0] io.druid.indexing.kafka.supervisor.KafkaSupervisor - New partition [15] discovered for topic [aggregator], adding to task group [0]

2016-08-04T15:05:53,036 INFO [KafkaSupervisor-adjesty_st1-0] io.druid.indexing.kafka.supervisor.KafkaSupervisor - Creating new task group [0] for partitions [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]

``

I do not work on Druid, but I have been playing with the Kafka Indexing Service recently. If you look at the overlord console do you see if you have available slots on middle managers to run the indexing tasks? Also look in the overlord logs for state transitions of tasks being created - they may be going into PENDING instead of RUNNING if there aren’t any slots available.

If there are available slots you are going to need to include more of your overlord log and possibly the middle manager log.

Regards,

–Ben

Hey Lev,

Have you been able to get this working?