[druid-user] How to post a druid kaka indexer supervisor in kerberos environment

The Overlord endpoint is expecting credentials to be passed when you submit an indexing job or this URL is incorrect http://host:8081/druid/inexer/v1/supervisor/ it should be http://host:8081/druid/indexer/v1/supervisor/ .

Rommel Garcia

Thank you Rommel for your advice. I have updated the url as you suggested but still getting 401 error. Can you please elaborate on “**The **Overlord endpoint is expecting credentials to be passed when you submit an indexing job” ?

Thank you Rommel for your advice. I have updated the url as you suggested but still getting 401 error. Can you please elaborate on “**The **Overlord endpoint is expecting credentials to be passed when you submit an indexing job” ?

Hi Imran,

If authentication and authorization is enabled on the Druid cluster, Druid APIs expect authentication headers to be passed. Can you check if authentication and authorization is enabled for your cluster ?

Thanks,

Sashi

Thanks Sashidhar. I am able to do a post request. Did couple of changes as of below:

added below properties in the druid kafka indexer json file in “consumerProperties” section

“security.protocol”: “SASL_PLAINTEXT”,
“group.id”:"",
“sasl.kerberos.service.name”: “kafka”

``

Below curl command used to post the druid kafka indexer json:

curl -ikv --negotiate -u : -X ‘POST’ -H ‘Content-Type:application/json’ -d @druid_file.json http:/host:8090/druid/indexer/v1/supervisor/

``

After above changes I am able to submit to the supervisor but now getting the kafka consumer constuction exception…Need help on this if i need to change any configurations on druid??

2019-04-29T15:25:19,488 ERROR [task-runner-0-priority-0] io.druid.indexing.overlord.ThreadPoolTaskRunner - Exception while running task[KafkaIndexTask{id=index_kafka__***_8c0724ccdb495bc_gmaihmej, type=index_kafka, dataSource=***_}]
org.apache.kafka.common.KafkaException: Failed to construct kafka consumer
at org.apache.kafka.clients.consumer.KafkaConsumer.(KafkaConsumer.java:717) ~[?:?]
at org.apache.kafka.clients.consumer.KafkaConsumer.(KafkaConsumer.java:597) ~[?:?]
at org.apache.kafka.clients.consumer.KafkaConsumer.(KafkaConsumer.java:579) ~[?:?]
at io.druid.indexing.kafka.KafkaIndexTask.newConsumer(KafkaIndexTask.java:916) ~[?:?]
at io.druid.indexing.kafka.KafkaIndexTask.run(KafkaIndexTask.java:295) ~[?:?]
at io.druid.indexing.overlord.ThreadPoolTaskRunner$ThreadPoolTaskRunnerCallable.call(ThreadPoolTaskRunner.java:436) [druid-indexing-service-0.10.1.2.6.5.0-292.jar:0.10.1.2.6.5.0-292]
at io.druid.indexing.overlord.ThreadPoolTaskRunner$ThreadPoolTaskRunnerCallable.call(ThreadPoolTaskRunner.java:408) [druid-indexing-service-0.10.1.2.6.5.0-292.jar:0.10.1.2.6.5.0-292]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_112]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_112]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_112]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112]
Suppressed: java.lang.NullPointerException
at io.druid.segment.realtime.plumber.CoordinatorBasedSegmentHandoffNotifier.close(CoordinatorBasedSegmentHandoffNotifier.java:164) ~[druid-server-0.10.1.2.6.5.0-292.jar:0.10.1.2.6.5.0-292]
at io.druid.segment.realtime.appenderator.AppenderatorDriver.close(AppenderatorDriver.java:351) ~[druid-server-0.10.1.2.6.5.0-292.jar:0.10.1.2.6.5.0-292]
at io.druid.indexing.kafka.KafkaIndexTask.run(KafkaIndexTask.java:576) ~[?:?]
at io.druid.indexing.overlord.ThreadPoolTaskRunner$ThreadPoolTaskRunnerCallable.call(ThreadPoolTaskRunner.java:436) [druid-indexing-service-0.10.1.2.6.5.0-292.jar:0.10.1.2.6.5.0-292]
at io.druid.indexing.overlord.ThreadPoolTaskRunner$ThreadPoolTaskRunnerCallable.call(ThreadPoolTaskRunner.java:408) [druid-indexing-service-0.10.1.2.6.5.0-292.jar:0.10.1.2.6.5.0-292]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_112]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_112]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_112]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112]
Suppressed: java.lang.NullPointerException
at io.druid.segment.realtime.appenderator.AppenderatorImpl.shutdownExecutors(AppenderatorImpl.java:691) ~[druid-server-0.10.1.2.6.5.0-292.jar:0.10.1.2.6.5.0-292]
at io.druid.segment.realtime.appenderator.AppenderatorImpl.close(AppenderatorImpl.java:619) ~[druid-server-0.10.1.2.6.5.0-292.jar:0.10.1.2.6.5.0-292]
at io.druid.indexing.kafka.KafkaIndexTask.run(KafkaIndexTask.java:576) ~[?:?]
at io.druid.indexing.overlord.ThreadPoolTaskRunner$ThreadPoolTaskRunnerCallable.call(ThreadPoolTaskRunner.java:436) [druid-indexing-service-0.10.1.2.6.5.0-292.jar:0.10.1.2.6.5.0-292]
at io.druid.indexing.overlord.ThreadPoolTaskRunner$ThreadPoolTaskRunnerCallable.call(ThreadPoolTaskRunner.java:408) [druid-indexing-service-0.10.1.2.6.5.0-292.jar:0.10.1.2.6.5.0-292]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_112]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_112]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_112]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112]
Caused by: org.apache.kafka.common.KafkaException: java.lang.IllegalArgumentException: Could not find a ‘KafkaClient’ entry in the JAAS configuration. System property ‘java.security.auth.login.config’ is not set
at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:94) ~[?:?]
at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:93) ~[?:?]
at org.apache.kafka.common.network.ChannelBuilders.clientChannelBuilder(ChannelBuilders.java:51) ~[?:?]
at org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:84) ~[?:?]
at org.apache.kafka.clients.consumer.KafkaConsumer.(KafkaConsumer.java:657) ~[?:?]
… 10 more
Caused by: java.lang.IllegalArgumentException: Could not find a ‘KafkaClient’ entry in the JAAS configuration. System property ‘java.security.auth.login.config’ is not set
at org.apache.kafka.common.security.JaasUtils.defaultJaasConfig(JaasUtils.java:85) ~[?:?]
at org.apache.kafka.common.security.JaasUtils.jaasConfig(JaasUtils.java:67) ~[?:?]
at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:85) ~[?:?]

``

See if this helps - https://stackoverflow.com/questions/50684956/kafka-could-not-find-a-kafkaclient-entry-in-the-jaas-configuration-java.