Druid Ingestion

Hi All,

I am trying to ingest data in druid from a kafka topic. I am working on hortonworks single node cluster.

Metadata Storage : mysql

Deep Storage : local

This is how my data looks in kafka topic.

{“Machine”:“CMGG-12”,“Time_Stamp”:“1522233364000”,“Z2Temp”:463020.0,“XTemp”:630.0,“YTemp”:23370.0,“ZTemp”:30.0,“ATemp”:101580.0,“CTemp”:251010.0}

This is my ingestion schema :

{

“type”: “kafka”,

“dataSchema”: {

“dataSource”: “testMemRegV14”,

“parser”: {

“type”: “string”,

“parseSpec”: {

“timestampSpec”: {

“column”: “Time_Stamp”,

“format”: “millis”

},

“dimensionsSpec”: {

“dimensions”: [“Machine”, “Time_Stamp”, “Z2Temp”, “XTemp”]

},

“format”: “json”

}

},

“metricsSpec”: [{

“type”: “count”,

“name”: “count”

}],

“granularitySpec”: {

“type”: “uniform”,

“segmentGranularity”: “minute”,

“queryGranularity”: “minute”

}

},

“tuningConfig”: {

“type”: “kafka”,

“maxRowsPerSegment”: 5000000

},

“ioConfig”: {

“topic”: “MemRegDataV5”,

“consumerProperties”: {

“bootstrap.servers”: “localhost:6667”

},

“taskCount”: 1,

“replicas”: 1,

“taskDuration”: “PT60S”

}

}

I can see the tasks in druid console it display status SUCCESS. But I cant see data in druid.

Please HELP.

Hey,
Your tasks are getting successfully completed but since your data is out of the windowPeriod, it is getting dropped i.e. not getting indexed. The record's timestamp is 03/28/2018 @ 10:36am (UTC). Can you try the same activity for the current timestamp in UTC only?
Read this
http://druid.io/docs/latest/ingestion/stream-push.html