Tranquility not sending data to Druid

I am evaluating Druid for my use case which ingest csv data through tranquility in real time. Following is the server configuration:-

{ "dataSources" : { "audience" : { "spec" : { "dataSchema" : { "dataSource" : "audience", "parser" : { "type" : "string", "parseSpec":{ "format" : "csv", "timestampSpec" : { "column" : "timestamp" }, "columns" : ["timestamp","partner_id","event_id","product_id","device_id","count"], "dimensionsSpec" : { "dimensions" : ["partner_id","event_id","product_id","device_id"] } } }, "metricsSpec" : [{ "type" : "longSum", "name" : total, "fieldName" : "count" }], "granularitySpec" : { "segmentGranularity" : "HOUR", "queryGranularity" : "HOUR", "intervals" : [ "2013-08-31/2013-09-01" ] } }, "ioConfig" : { "type" : "realtime" }, "tuningConfig" : { "type" : "realtime", "maxRowsInMemory" : "100000", "intermediatePersistPeriod" : "PT10M", "windowPeriod" : "PT10M" } }, "properties" : { "task.partitions" : "1", "task.replicants" : "1" } } }, "properties" : { "zookeeper.connect" : "localhost", "druid.discovery.curator.path" : "/druid/discovery", "druid.selectors.indexing.serviceName" : "druid/overlord", "http.port" : "8200", "http.threads" : "8" } }

data is generated randomly by a python script as:-

1471336991,1,960,136,3ZLA7,1 1471336991,1,369,367,8MP2B,1 1471336991,2,544,550,C9ZG8,1 1471336991,1,135,394,XFX31,1 1471336991,2,590,552,VXMTL,1 1471336991,1,493,615,0C2HR,1 1471336991,2,435,710,HKYP0,1 1471336991,1,394,483,V2HP9,1 1471336991,2,441,376,J1LYO,1

Following commands submits the data and returns

{“result”:{“received”:1000,“sent”:0}}

``

python createData.py |curl -XPOST -H'Content-Type: text/plain' --data-binary @- http://localhost:8200/v1/post/audience.

hi Mangat,
As you are using tranquility, you will need to define timestamp too.

Hi Vrinda,

Thank you for the reply. Could you please explain where do I need to define timestamp?

In the server configuration I have already specified timestamp column as:-

"timestampSpec" : { "column" : "timestamp" }, "columns" : ["timestamp","partner_id","event_id","product_id","device_id","count"],

``

My data also contains latest timestamp from the script, (as the druid default window in 10 minutes only!).

Finally able to solve the problem. Actually I was sending time to Druid in Epoch time format, but it expect ISO-8601 format. In python one can easily get so by :-

datetime.datetime.utcnow().isoformat()

Mangat,
You can also send epoch but it needs to be in milliseconds.