How can I set Druid to ingest streaming data from Kerberized Kafka?

I have Kerberized Kafka broker.

I am trying to setup Druid to ingest streaming data from that Kafka:

  • by submitting supervisor spec to Druid overlord
    I do see that there is field for Kafka server URL(bootstrap.servers) but I don’t see any field for Kerberos information.
    How can I set up Druid so that it can ingest streaming data from Kerberized Kafka?

You’ll want to setup a JAAS conf file for Druid as a KafkaClient, such as the following keytab-based example:


KafkaClient {

com.sun.security.auth.module.Krb5LoginModule required

useTicketCache=false

useKeyTab=true

storeKey=true

keyTab=<path of keytab for druid>

principal=<principal for druid>

};

Then add -Djava.security.auth.login.config=<JAAS conf file> to the JVM properties for each Druid service. Additionally, you’ll need to add that flag to druid.indexer.runner.javaOpts in your middleManager runtime properties so that the indexing tasks pick this up too.

Thanks,

Jon