NativeS3FileSystem - Renaming s3n - returning false as src does not exist

Hello everyone!

I am trying to write test information to s3 bucket and i am getting an error:

017-04-18T15:07:48,095 DEBUG [pool-23-thread-1] org.apache.hadoop.fs.s3native.NativeS3FileSystem - getFileStatus could not find key ‘druid/segments/wikiticker-s3-new/2015-09-12T00:00:00.000Z_2015-09-13T00:00:00.000Z/2017-04-18T15:07:24.820Z/0/index.zip.0’
2017-04-18T15:07:48,095 DEBUG [pool-23-thread-1] org.apache.hadoop.fs.s3native.NativeS3FileSystem - Renaming ‘s3n://BUCKET_NAME/druid/segments/wikiticker-s3-new/2015-09-12T00:00:00.000Z_2015-09-13T00:00:00.000Z/2017-04-18T15:07:24.820Z/0/index.zip.0’ to ‘s3n://BUCKET_NAME/druid/segments/wikiticker-s3-new/2015-09-12T00:00:00.000Z_2015-09-13T00:00:00.000Z/2017-04-18T15:07:24.820Z/0/index.zip’ - returning false as src does not exist
2017-04-18T15:07:48,102 INFO [Thread-61] org.apache.hadoop.mapred.LocalJobRunner - reduce task executor complete.
2017-04-18T15:07:48,106 WARN [Thread-61] org.apache.hadoop.mapred.LocalJobRunner - job_local182715154_0002
java.lang.Exception: java.io.IOException: Unable to rename [s3n://`BUCKET_NAME`/druid/segments/wikiticker-s3-new/2015-09-12T00:00:00.000Z_2015-09-13T00:00:00.000Z/2017-04-18T15:07:24.820Z/0/index.zip.0] to [s3n://`BUCKET_NAME`/druid/segments/wikiticker-s3-new/2015-09-12T00:00:00.000Z_2015-09-13T00:00:00.000Z/2017-04-18T15:07:24.820Z/0/index.zip]
at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462) ~[hadoop-mapreduce-client-common-2.3.0.jar:?]
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:529) [hadoop-mapreduce-client-common-2.3.0.jar:?]
Caused by: java.io.IOException: Unable to rename [s3n://`BUCKET_NAME`/druid/segments/wikiticker-s3-new/2015-09-12T00:00:00.000Z_2015-09-13T00:00:00.000Z/2017-04-18T15:07:24.820Z/0/index.zip.0] to [s3n://`BUCKET_NAME`/druid/segments/wikiticker-s3-new/2015-09-12T00:00:00.000Z_2015-09-13T00:00:00.000Z/2017-04-18T15:07:24.820Z/0/index.zip]
at io.druid.indexer.JobHelper.serializeOutIndex(JobHelper.java:452) ~[druid-indexing-hadoop-0.9.2.jar:0.9.2]
at io.druid.indexer.IndexGeneratorJob$IndexGeneratorReducer.reduce(IndexGeneratorJob.java:727) ~[druid-indexing-hadoop-0.9.2.jar:0.9.2]
at io.druid.indexer.IndexGeneratorJob$IndexGeneratorReducer.reduce(IndexGeneratorJob.java:478) ~[druid-indexing-hadoop-0.9.2.jar:0.9.2]
at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:171) ~[hadoop-mapreduce-client-core-2.3.0.jar:?]
at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:627) ~[hadoop-mapreduce-client-core-2.3.0.jar:?]
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:389) ~[hadoop-mapreduce-client-core-2.3.0.jar:?]
at org.apache.hadoop.mapred.LocalJobRunner$Job$ReduceTaskRunnable.run(LocalJobRunner.java:319) ~[hadoop-mapreduce-client-common-2.3.0.jar:?]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:473) ~[?:1.7.0_131]
at java.util.concurrent.FutureTask.run(FutureTask.java:262) ~[?:1.7.0_131]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) ~[?:1.7.0_131]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) ~[?:1.7.0_131]
at java.lang.Thread.run(Thread.java:745) ~[?:1.7.0_131]

``

My task:
{
“type” : “index_hadoop”,
“spec” : {
“ioConfig” : {
“type” : “hadoop”,
“inputSpec” : {
“type” : “static”,
“paths” : “quickstart/wikiticker-2015-09-12-sampled.json.gz”
}
},
“dataSchema” : {
“dataSource” : “wikiticker-s3-new”,
“granularitySpec” : {
“type” : “uniform”,
“segmentGranularity” : “day”,
“queryGranularity” : “none”,
“intervals” : [“2015-09-12/2015-09-13”]
},
“parser” : {
“type” : “hadoopyString”,
“parseSpec” : {
“format” : “json”,
“dimensionsSpec” : {
“dimensions” : [
“channel”,
“cityName”,
“comment”,
“countryIsoCode”,
“countryName”,
“isAnonymous”,
“isMinor”,
“isNew”,
“isRobot”,
“isUnpatrolled”,
“metroCode”,
“namespace”,
“page”,
“regionIsoCode”,
“regionName”,
“user”
]
},
“timestampSpec” : {
“format” : “auto”,
“column” : “time”
}
}
},
“metricsSpec” : [
{
“name” : “count”,
“type” : “count”
},
{
“name” : “added”,
“type” : “longSum”,
“fieldName” : “added”
},
{
“name” : “deleted”,
“type” : “longSum”,
“fieldName” : “deleted”
},
{
“name” : “delta”,
“type” : “longSum”,
“fieldName” : “delta”
},
{
“name” : “user_unique”,
“type” : “hyperUnique”,
“fieldName” : “user”
}
]
},
“tuningConfig” : {
“type” : “hadoop”,
“partitionsSpec” : {
“type” : “hashed”,
“targetPartitionSize” : 5000000
},
“jobProperties” : {
“fs.s3.awsAccessKeyId” : “",
“fs.s3n.awsAccessKeyId” : "
”,
“fs.s3.awsSecretAccessKey” : “",
“fs.s3n.awsSecretAccessKey” : "
”,
“fs.s3.impl” : “org.apache.hadoop.fs.s3native.NativeS3FileSystem”,
“fs.s3n.impl” : “org.apache.hadoop.fs.s3native.NativeS3FileSystem”,
“io.compression.codecs” : “org.apache.hadoop.io.compress.GzipCodec,org.apache.hadoop.io.compress.DefaultCodec,org.apache.hadoop.io.compress.BZip2Codec,org.apache.hadoop.io.compress.SnappyCodec”
}
}
}
}

``

The file `s3n://BUCKET_NAME/druid/segments/wikiticker-s3-new/2015-09-12T00:00:00.000Z_2015-09-13T00:00:00.000Z/2017-04-18T15:07:24.820Z/0/index.zip.0 exists and i can see it in s3 bucket.

Best regards,
Oleksandr.
`

Also i found similar issue https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!searchin/druid-user/Renaming$20s3n|sort:relevance/druid-user/U9smlqQc4yc/znKjjBUzAQAJ
Currently i am using 0.9.2 version.

Have updated druid, to druid-0.10.0 and problem gone.

Nope, problem was not resolved yet :frowning:

So, please feel free to help with this stuff.
It can be the problem with some permissions, but i am not sure, please help.

Best regards,
Oleksandr.

Now, problem is resolved, was set incorrect permissions for user keys.