Hadoop Indexer Storing segments in storage/$dataSource/$dataSource

I am using a hadoop indexer (via druid indexer job) on HDFS json data to update segments for a data source. The mapreduce works properly with one exception, the segments are not being stored in the proper folder.

My current segments for datasource ‘events’ are stored in: s3://druid/storage/events

However the hadoop indexer is putting the segments in: **** s3://druid/storage/events/events

I cannot see anything wrong in my spec file, as it defines the same stuff as my tranquility realtime indexes (which work properly)

Is there a setting of some sort I am missing?

My spec file for the indexer is like such:

{

“type” : “index_hadoop”,

“spec” : {

“selectStrategy”: {

“type”: “fillCapacityWithAffinity”,

“affinityConfig”: {

“affinity”: {

“events”: [“druid-worker:8080”]

}

}

},

“dataSchema” : {

“parser” : {

“type” : “hadoopyString”,

“parseSpec” : {

“timestampSpec” : {

“format” : “auto”,

“column” : “ts”

},

“dimensionsSpec” : {

“spatialDimensions” : [

{

}

],

“dimensions” : ,

“dimensionExclusions” : [“duration”, “ts”, “id”]

},

“format” : “json”

}

},

“dataSource” : “events”,

“granularitySpec” : {

“type” : “uniform”,

“segmentGranularity” : “HOUR”,

“queryGranularity” : “MINUTE”,

“intervals” : [ “2016-09-24T00:00:00.000Z/2016-09-24T01:00:00.000Z” ]

},

“metricsSpec” : [{

“type”:“hyperUnique”,

“name”:“unique_users”,

“fieldName”:“uid”

},

{

“type”:“count”,

“name”:“count”

}

]

},

“tuningConfig” : {

“type” : “hadoop”,

“jobProperties” : {

“fs.s3.awsAccessKeyId” : “”,

“fs.s3.awsSecretAccessKey” : “”,

“fs.s3.impl” : “org.apache.hadoop.fs.s3native.NativeS3FileSystem”,

“fs.s3n.awsAccessKeyId” : “”,

“fs.s3n.awsSecretAccessKey” : “”,

“fs.s3n.impl” : “org.apache.hadoop.fs.s3native.NativeS3FileSystem”,

“io.compression.codecs” : “org.apache.hadoop.io.compress.GzipCodec,org.apache.hadoop.io.compress.DefaultCodec,org.apache.hadoop.io.compress.BZip2Codec,org.apache.hadoop.io.compress.SnappyCodec”

}

},

“ioConfig” : {

“type” : “hadoop”,

“inputSpec” : {

“type” : “static”,

“paths” : “hdfs://tmp/export”

}

}

},

“hadoopDependencyCoordinates”: [“org.apache.hadoop:hadoop-client:2.7.2”]

}

``

Hi Pere, can you file an issue here? https://github.com/druid-io/druid/issues

This is a known issue that we should track and fix.

Can you also share your common.runtime.properties?

Just want to make sure there’s nothing wrong there.