Cannot get Kafka indexing service running with Secured Kafka with PLAINTEXTSASL in Hortonworks

I have been trying to get Kafka indexing service running with with Secured Kafka with Kerberos (PLAINTEXTSASL) for some time.

I have running Kerberos secured Kafka broker, producer and consumer, they are all tested .

consumer properties (“security.protocol”: “PLAINTEXTSASL”, “”: “kafka” ) related to security in json file, but still getting below error with curl command

[root@devupgnew4 ~]# curl -ikv --negotiate -X ‘POST’ -H ‘Content-Type:application/json’ -d @/tmp/examples/wikipedia-kafka-supervisor.json

POST /druid/indexer/v1/supervisor HTTP/1.1

User-Agent: curl/7.29.0


Accept: /


Content-Length: 1457

Expect: 100-continue

< HTTP/1.1 100 Continue

HTTP/1.1 100 Continue

< HTTP/1.1 500 Server Error

HTTP/1.1 500 Server Error

< Date: Thu, 31 Jan 2019 15:02:24 GMT

< Cache-Control: must-revalidate,no-cache,no-store

Cache-Control: must-revalidate,no-cache,no-store

< Content-Type: text/html;charset=iso-8859-1

Content-Type: text/html;charset=iso-8859-1

< Content-Length: 409

Content-Length: 409

< Connection: close

Connection: close

< Server: Jetty(9.3.19.v20170502)

Server: Jetty(9.3.19.v20170502)


Error 500


Problem accessing /druid/indexer/v1/supervisor. Reason:

    org.apache.kafka.common.KafkaException: Failed to construct kafka consumer

Powered by Jetty:// 9.3.19.v20170502
  • Closing connection 0

My json schema for Kafka indexing

[root@devupgnew4 ~]# cat /tmp/examples/wikipedia-kafka-supervisor.json


“type”: “kafka”,

“dataSchema”: {

“dataSource”: “kafkawikipedia”,

“parser”: {

“type”: “string”,

“parseSpec”: {

“format”: “json”,

“timestampSpec”: {

“column”: “time”,

“format”: “auto”


“dimensionsSpec”: {

“dimensions”: [

















{ “name”: “added”, “type”: “long” },

{ “name”: “deleted”, “type”: “long” },

{ “name”: “delta”, “type”: “long” }





“metricsSpec” : ,

“granularitySpec”: {

“type”: “uniform”,

“segmentGranularity”: “DAY”,

“queryGranularity”: “NONE”,

“rollup”: false



“tuningConfig”: {

“type”: “kafka”,

“reportParseExceptions”: false


“ioConfig”: {

“topic”: “kafkawiki”,

“replicas”: 1,

“taskDuration”: “PT10M”,

“completionTimeout”: “PT20M”,

“consumerProperties”: {

“bootstrap.servers”: “,,”,

“security.protocol”: “PLAINTEXTSASL”,

“”: “kafka”




Could you please help me in getting Kafka indexing service running with with Secured Kafka ?

overlord.log attached.

overload.log (120 KB)

Hi Team,

Can any one help me on this. We have HDP kerberos cluster.


Mohammad Layeeq

Team can any one help me on this.