Configured timestamp column doesn't work

**hadoop_index configuration fragment:**

 "parser" : {
        "type" : "orc",
        "parseSpec" : {
          "format" : "timeAndDims",
          "timestampSpec" : {
            "column" : "import_time",
            "format" : "yyyyMMdd HH:mm:ss"
          },
          "dimensionsSpec" : {
            "dimensions": ["doc_id","title","url"]
          }
        }
      }



from following task log, I think  time column is not assigned as my configuration.
**task error log:**



18:38:40.687 [task-runner-0-priority-0] INFO  org.apache.hadoop.mapreduce.Job - Task Id : attempt_1559809729032_5069641_m_000009_0, Status : FAILED
Error: [org.apache.druid.java.util.common.RE](http://www.google.com/url?q=http%3A%2F%2Forg.apache.druid.java.util.common.RE&sa=D&sntz=1&usg=AFQjCNGVdilUAl0NeDp5Bn_wqMKdCeK6Xw): Failure on row[{V_02PfpTdI, è¿·ä½ ä¸–ç•Œï¼šè€ å©†ï¼Œä½ è§‰å¾—å« ç»™æˆ‘æ˜¯ä¸­äº†å‡ ç­‰å¥–ï¼Ÿ, [https://oppo.yidianzixun.com/article/V_02PfpTdI?s=oppobrowser&appid=oppobrowser](https://oppo.yidianzixun.com/article/V_02PfpTdI?s=oppobrowser&appid=oppobrowser), 视频, 动漫, 其他, 2, null, å° é›¯ä¸–ç•Œè§‚, 4, 20190312 14:17:39, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, null, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 20190714 07:37:47, 0, 0}]
	at org.apache.druid.indexer.HadoopDruidIndexerMapper.map(HadoopDruidIndexerMapper.java:103)
	at org.apache.druid.indexer.DetermineHashedPartitionsJob$DetermineCardinalityMapper.run(DetermineHashedPartitionsJob.java:327)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.lang.IllegalArgumentException: Invalid format: "V_02PfpTdI"
	at org.joda.time.format.DateTimeFormatter.parseDateTime(DateTimeFormatter.java:899)
	at org.apache.druid.java.util.common.DateTimes$UtcFormatter.parse(DateTimes.java:73)
	at org.apache.druid.java.util.common.parsers.TimestampParser.lambda$createTimestampParser$4(TimestampParser.java:93)
	at org.apache.druid.java.util.common.parsers.TimestampParser.lambda$createObjectTimestampParser$10(TimestampParser.java:132)
	at org.apache.druid.data.input.impl.TimestampSpec.parseDateTime(TimestampSpec.java:106)
	at org.apache.druid.data.input.impl.TimestampSpec.extractTimestamp(TimestampSpec.java:94)
	at org.apache.druid.data.input.orc.OrcHadoopInputRowParser.parseBatch(OrcHadoopInputRowParser.java:130)
	at org.apache.druid.data.input.orc.OrcHadoopInputRowParser.parseBatch(OrcHadoopInputRowParser.java:59)
	at org.apache.druid.segment.transform.TransformingInputRowParser.parseBatch(TransformingInputRowParser.java:50)
	at org.apache.druid.indexer.HadoopDruidIndexerMapper.parseInputRow(HadoopDruidIndexerMapper.java:144)
	at org.apache.druid.indexer.HadoopDruidIndexerMapper.map(HadoopDruidIndexerMapper.java:67)

hi 黄峰 , the parser section of the ingestion spec looks legit. Can you convert your orc to json , and make sure the time column name is correct?

Thanks

I find orc written by hive don’t include real column name, just col_0,col_1 so on. so it can’t match my task configuration. so sad!

在 2019年7月15日星期一 UTC+8下午11:48:33,Ming F写道:

hi 黄峰, I ran into a similar issue probably the same as yours, where the timestamp column: import_time in your ingestion spec needed to be column: _col0, or whatever that _col id was.

And rename the columns use "format": "orc" instead of timeAndDims which supports flattenSpec


"format": "orc",

"flattenSpec": {

"useFieldDiscovery": true,

"fields": [

{

"type": "path",

"name": "doc_id",

"expr": "$._col1"

},{

"type": "path",

"name": "title",

"expr": "$._col2"

}

]

},

"timestampSpec": {

"column": "_col0",

"format": "auto"

},

"dimensionsSpec": {

"dimensions": [

],

"dimensionExclusions": [],

"spatialDimensions": []

}

}

},```

This is because the latest orc extension in  is a total rewrite of the old contrib extension

Hope this works out for you.