Failed to send supervisor for ingestion from Kafka & Confluent Schema-Registry

Hi All,
I am pretty new to the system and for few days I am trying to ingest data from Confluent Kafka and get schema from confluent schema-registry
We are writing sending messages to the Kafka to topic mcapi.avro.test-topic1 in avro format and the schema is being managed by the schema-registry.

When trying to send the supervisor from the GUI I am getting the following error:
Failed to submit supervisor: Cannot construct instance of org.apache.druid.indexing.kafka.supervisor.KafkaSupervisorSpec, problem: ioConfig at [Source: (org.eclipse.jetty.server.HttpInputOverHTTP); line: 1, column: 700]

Any Ideas?

I have build the following supervisor json:

{
“type”: “kafka”,
“dataSchema”: {
“dataSource”: “TPriEvent”,
“parser”: {
“type”: “avro_stream”,
“avroBytesDecoder”: {
“type”: “schema_registry”,
“url”: “http://SCHEMA_IP_ADDRESS:8081”,
},
“parseSpec”: {
“format”: “avro”,
“timestampSpec”: {
“column”: “timestamp”,
“format”: “auto”
},
“dimensionSpec”: {
“dimensions”: [
{ “name” : “schema_name”, “type” : “string” }
],
“dimensionExclusions”:
},
},
“metricsSpec”: [
],
“granularitySpec”: {
“type”: “uniform”,
“segmentGranularity”: “hour”,
“queryGranularity”: “NONE”,
“rollup”: false
}
},
“ioConfig”: {
“topic”: “mcapi.avro.test-topic1”,
“inputFormat”: “avro_stream”,
“useEarliestOffset”: true,
“consumerProperties”: {
“bootstrap.servers”: “KAFKA_SERVER_IP_ADDR:9092”
}
},
“tuningConfig”: {
“type”: “kafka”,
“reportParseExceptions”: true
}
}
}

Can you access the Kafka server from your Druid servers? It looks like it can’t connect.

Hi,
Yes I am able to do that
It’s doesn’t let me to send the supervisor at all, at this part its still doesn’t trying to connect to the Kafka

Have you added the avro extension?
https://druid.apache.org/docs/latest/ingestion/data-formats.html#avro-stream-parser

Yes, I did :frowning: