Index task failure in 0.7.1-rc1 | Unable to to find a matching interval

Hi,

I am trying local firehose index task on 0.7.1-rc1 with the below index spec but the task is failing with the error

com.metamx.common.ISE: Unable to to find a matching interval for [2015-03-01T07:59:59.977Z]

at io.druid.indexing.common.task.IndexTask.getDataIntervals(IndexTask.java:214) ~[druid-indexing-service-0.7.1-rc1.jar:0.7.1-rc1]

at io.druid.indexing.common.task.IndexTask.run(IndexTask.java:165) ~[druid-indexing-service-0.7.1-rc1.jar:0.7.1-rc1]

at io.druid.indexing.overlord.ThreadPoolTaskRunner$ThreadPoolTaskRunnerCallable.call(ThreadPoolTaskRunner.java:235) [druid-indexing-service-0.7.1-rc1.jar:0.7.1-rc1]

at io.druid.indexing.overlord.ThreadPoolTaskRunner$ThreadPoolTaskRunnerCallable.call(ThreadPoolTaskRunner.java:214) [druid-indexing-service-0.7.1-rc1.jar:0.7.1-rc1]

at java.util.concurrent.FutureTask.run(FutureTask.java:262) [?:1.7.0_65]

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [?:1.7.0_65]

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [?:1.7.0_65]

at java.lang.Thread.run(Thread.java:745) [?:1.7.0_65]

{

“type” : “index”,

“spec” : {

“dataSchema” : {

“dataSource” : “mors-1”,

“parser” : {

“type” : “string”,

“parseSpec” : {

“format” : “json”,

“timestampSpec” : {

“column” : “ts”,

“format” : “millis”

},

“dimensionsSpec” : {

“dimensions”: [“channel”, “e”, “page”, “ppid”, “oid”, “mpid”, “mktgcaids”, “incc”],

“dimensionExclusions” : ,

“spatialDimensions” :

}

}

},

“metricsSpec” : [

{

“type” : “count”,

“name” : “rowcount”

}

],

“granularitySpec” : {

“type” : “uniform”,

“segmentGranularity” : “HOUR”,

“queryGranularity” : “HOUR”,

“intervals”: [ “2015-03-01T08:00:00Z/2015-03-01T09:00:00Z” ]

}

},

“ioConfig” : {

“type” : “index”,

“firehose” : {

“type” : “local”,

“baseDir” : “/home/dgdeploy/mors_hadoop/2015/03/01/00/”,

“filter” : “part-r-00000”

}

},

“tuningConfig” : {

“type” : “index”,

“targetPartitionSize” : 5000000,

“rowFlushBoundary” : 0,

“numShards”: -1

}

}

}

Basically the input data file has data for the hour 2015-03-01/08 and it has some records from the hour 2015-03-01/07. In the index spec I have specified granularitySpec.intervals = 2015-03-01T08:00:00Z/2015-03-01T09:00:00Z to account for 1 hour.

In 0.6.160, this used to work fine where Indexer ignores any records in the input data which doesn’t fall under granularitySpec.intervals. With the latest code, I am seeing this error

Unable to to find a matching interval for [2015-03-01T07:59:59.977Z]

I tried the same with the stable version 0.7.0 and the issue didn’t appear. Index job ran fine.

Hi Venkatesh, thanks for this catch. This is a bug that was introduced while trying to support arbitrary granularity spec. Do you mind filing an issue so we can fix it before declaring 0.7.1 stable?

Hi, I added https://github.com/druid-io/druid/pull/1264/files to fix this issue. Hopefully this matches the case you saw.