Need help: Spatial dimension spec does work properly for TSV parse spec

Hello community,

Is is pretty painful issue that I have after migration on druid 0.14.2 version.

Description is here:

https://github.com/apache/incubator-druid/issues/7948

In short for template:

“parser”: {

``
“parseSpec”: {

“columns”: [

“ts”,

“ip_address”,

“latitude”,

“longitude”,

“radius”,

“confidence”

],

“dimensionsSpec”: {

“dimensionExclusions”: ,

“dimensions”: [

“ip_address”,

“radius”,

“confidence”

],

“spatialDimensions”: [

{

“dimName”: “geo”,

“dims”: [

“latitude”,

“longitude”

]

}

]

},

“format”: “tsv”,

“timestampSpec”: {

“column”: “ts”,

“format”: “millis”

}

},

“type”: “lzo”

I got an error

Caused by: shade.com.fasterxml.jackson.databind.JsonMappingException: Instantiation of [simple type, class org.apache.druid.data.input.impl.DelimitedParseSpec] value failed: column[geo] not in columns. (through reference chain: org.apache.druid.data.input.impl.StringInputRowParser[“parseSpec”])

at shade.com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.wrapException(StdValueInstantiator.java:399)

at shade.com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.createFromObjectWith(StdValueInstantiator.java:231)

at shade.com.fasterxml.jackson.databind.deser.impl.PropertyBasedCreator.build(PropertyBasedCreator.java:135)

at shade.com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:442)

at shade.com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1099)

at shade.com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:296)

at shade.com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeOther(BeanDeserializer.java:166)

at shade.com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:136)

at shade.com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId(AsPropertyTypeDeserializer.java:122)

at shade.com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:93)

at shade.com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType(AbstractDeserializer.java:131)

at shade.com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:518)

at shade.com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping(BeanDeserializer.java:463)

at shade.com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:378)

at shade.com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1099)

at shade.com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:296)

at shade.com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeOther(BeanDeserializer.java:166)

at shade.com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:136)

at shade.com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId(AsPropertyTypeDeserializer.java:122)

at shade.com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:93)

at shade.com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType(AbstractDeserializer.java:131)

at shade.com.fasterxml.jackson.databind.deser.impl.TypeWrappedDeserializer.deserialize(TypeWrappedDeserializer.java:42)

at shade.com.fasterxml.jackson.databind.ObjectMapper._convert(ObjectMapper.java:3454)

… 12 more

Caused by: java.lang.IllegalArgumentException: column[geo] not in columns.

at shade.com.google.common.base.Preconditions.checkArgument(Preconditions.java:148)

at org.apache.druid.data.input.impl.DelimitedParseSpec.verify(DelimitedParseSpec.java:119)

at org.apache.druid.data.input.impl.DelimitedParseSpec.(DelimitedParseSpec.java:63)

at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

``

According to a code I see that DelimitedParseSpec call #verify method with all dimensions.

And one of that dimensions is spatial dimensions. Looks like getDimensionNames returns names of dimensions, but for spatial one it is a name that dosn’t exist in columns list.

Is this a bug or behavior is correct and I need to fix my template somehow ?

Thanks