Realtime data from rabbitmq to Druid

Hi all,

I need a help to ingest***realtime data from rabbitmq to Druid ***

[root@dmse02lx1567c ~]# curl -X POST -H ‘Content-Type: application/json’ -d @Druid_Ingestion/rabbitmq-index.json http://localhost:8081/druid/indexer/v1/supervisor

{“error”:“Could not resolve type id ‘realtime’ into a subtype of [simple type, class io.druid.indexing.overlord.supervisor.SupervisorSpec]\n at [Source: HttpInputOverHTTP@4ff1a0dd[c=1749,q=1,[0]=EOF,s=STREAM]; line: 1, column: 4]”}

{

“type”: “realtime”,

“spec”: {

“dataSchema” : {

“dataSource” : “rabbitmq_test”,

“parser” : {

“type” : “base64”,

“parseSpec” : {

“format” : “json”,

“timestampSpec” : {

“column” : “submit_date”,

“format” : “auto”

},

“dimensionsSpec” : {

“dimensions”: [ “request_type”,“origin”,“description” ],

“dimensionExclusions” : ,

“spatialDimensions” :

}

}

},

“metricsSpec” : [ {

“type” : “count”,

“name” : “count”

} ],

“granularitySpec” : {

“type” : “uniform”,

“segmentGranularity” : “hour”,

“queryGranularity” : “none”

}

},

“ioConfig” : {

“type” : “realtime”,

“firehose”: {

“type”: “rabbitmq”,

“connection” : {

“host”: “172.29.242.173”,

“port”: “5672”,

“username”: “guest”,

“password”: “guest”,

“virtualHost”: “/”,

“uri”: “amqp://172.29.242.173/”

},

“config” : {

“exchange”: “NiFi_Exchanges_test_1”,

“queue” : “NiFi_Queues_test_1”,

“routingKey”: “NiFi_Routing_key_test_1”,

“durable”: “true”,

“exclusive”: “false”,

“autoDelete”: “false”,

“maxRetries”: “10”,

“retryIntervalSeconds”: “1”,

“maxDurationSeconds”: “300”

},

“plumber”: {

“type”: “realtime”

}

}

},

“tuningConfig”: {

“type” : “realtime”,

“maxRowsInMemory”: 7500000,

“intermediatePersistPeriod”: “PT10m”,

“windowPeriod”: “PT10m”,

“basePersistDirectory”: “/tmp/realtime/basePersist”,

“rejectionPolicy”: {

“type”: “serverTime”

}

}

}

}

you need to submit a task instead of hitting the supervisor end point.
Try -

curl -X POST -H ‘Content-Type: application/json’ -d @Druid_Ingestion/rabbitmq-index.json http://localhost:8081/druid/indexer/v1/task

Hi Bangarwa,

http://localhost:8081/druid/indexer/v1/task

For batch files. This is Streaming Data from rabbitmq. We want to see, use and analysis data during streaming time