Druid Kafka Indexing service: load data to druid from kafka installed in sifferent server

Hi Team,

We are working on the Druid Kafka indexing service and were successful in loading data from kafka installed on same linux server as that of Druid(Druid and Kafka running in same machine)

Now we have a requirement to load data to Druid, from others kafka servers. Assumed it would be as simple as changing the bootstrap server ip in supervisor config, but it doesn’t work as expected. NO error is thrown, but aftre submitting the supervisor spec, no indexing tasks are created.

Could you please advise what are the changes required in order to have the indexing service up and running to load data from kafka installed on separate server.

“ioConfig”: {

“topic”: “cms_test”,

“consumerProperties”: {

“bootstrap.servers”: “:9092”

},

Thanks

Soumya

Hi Soumya,
Are you able to verify some basic connectivity from your druid host to external Kafka bootstrap server?

Can you try kafka command line tool from druid host