Upgraded to 0.10.1-rc3

Seems like running into some issues here. I am getting this issue with my ingestion spec:

Class org.apache.hadoop.fs.s3native.NativeS3FileSystem not found


However, I use the same configuration I have always used when upgrading Druid, except this time I include the -rc at the end of the build file to match the download / extraction path:

If you specify druid.extensions.loadList=[], Druid won’t load any extension from file system.

If you don’t specify druid.extensions.loadList, Druid will load all the extensions under root extension directory.

More info: http://druid.io/docs/latest/operations/including-extensions.html

druid.extensions.loadList=[“druid-kafka-eight”, “druid-s3-extensions”, “druid-histogram”, “druid-datasketches”, “druid-lookups-cached-global”, “postgresql-metadata-storage”]

druid.extensions.directory=/opt/druid/druid-{{getv “/docker-druid/environment/DRUID_VERSION”}}/extensions

If you have a different version of Hadoop, place your Hadoop client jar files in your hadoop-dependencies directory

and uncomment the line below to point to your directory.

druid.extensions.hadoopDependenciesDir=/opt/druid/druid-{{getv “/docker-druid/environment/DRUID_VERSION”}}/hadoop-dependencies

This looks to be the same issue as https://groups.google.com/forum/?#!msg/druid-user/HhcMkkbKRXI/6BYEs8HDAgAJ

Also @Johnson, you should be able to solve your problems with s3n by running pull-deps (see here) to pull in org.apache.hadoop:hadoop-aws:2.7.3 and including that in your hadoopoDependencyCoordinates

Thanks, Robert - I am attempting re-install. I’m going to specifiy “hadoop-aws-2.7.3” in my extensions load list it “appears” that it is loading. Hopefully this will get this going. Thanks for your help!

Hey, Robert -

2017-09-05T18:26:51,697 ERROR [task-runner-0-priority-0] io.druid.indexing.overlord.ThreadPoolTaskRunner - Exception while running task[HadoopIndexTask{id=index_ip_queries-10, type=index_hadoop, dataSource=ip_queries}]
io.druid.java.util.common.ISE: Hadoop dependency [/opt/druid/druid-0.10.1-rc3/hadoop-dependencies/hadoop-aws/2.7.3] didn't exist!?
        at io.druid.initialization.Initialization.getHadoopDependencyFilesToLoad(Initialization.java:274) ~[druid-server-0.10.1-rc3.jar:0.10.1-rc3]
        at io.druid.indexing.common.task.HadoopTask.buildClassLoader(HadoopTask.java:156) ~[druid-indexing-service-0.10.1-rc3.jar:0.10.1-rc3]

``

Adding “hadoop-aws” to my extensions it still fails with this error? I also inspected, and I do in fact see hadoop-aws under my extensions directory…?

Robert, actually I correct the issue by removing the extension loading in my ingestion spec. However, I do now get this:

2017-09-05T18:34:45,469 INFO [task-runner-0-priority-0] org.apache.hadoop.mapreduce.JobSubmitter - Cleaning up the staging area file:/tmp/hadoop-root/mapred/staging/root697478582/.staging/job_local697478582_0001
2017-09-05T18:34:45,473 ERROR [task-runner-0-priority-0] io.druid.indexing.overlord.ThreadPoolTaskRunner - Exception while running task[HadoopIndexTask{id=index_ip_queries-11, type=index_hadoop, dataSource=ip_queries}]
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
        at com.google.common.base.Throwables.propagate(Throwables.java:160) ~[guava-16.0.1.jar:?]
        at io.druid.indexing.common.task.HadoopTask.invokeForeignLoader(HadoopTask.java:218) ~[druid-indexing-service-0.10.1-rc3.jar:0.10.1-rc3]
        at io.druid.indexing.common.task.HadoopIndexTask.run(HadoopIndexTask.java:177) ~[druid-indexing-service-0.10.1-rc3.jar:0.10.1-rc3]
        at io.druid.indexing.overlord.ThreadPoolTaskRunner$ThreadPoolTaskRunnerCallable.call(ThreadPoolTaskRunner.java:436) [druid-indexing-service-0.10.1-rc3.jar:0.10.1-rc3]
        at io.druid.indexing.overlord.ThreadPoolTaskRunner$ThreadPoolTaskRunnerCallable.call(ThreadPoolTaskRunner.java:408) [druid-indexing-service-0.10.1-rc3.jar:0.10.1-rc3]
        at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_144]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_144]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_144]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_144]
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_144]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_144]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_144]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_144]
        at io.druid.indexing.common.task.HadoopTask.invokeForeignLoader(HadoopTask.java:215) ~[druid-indexing-service-0.10.1-rc3.jar:0.10.1-rc3]
        ... 7 more
Caused by: java.lang.NoSuchMethodError: com.amazonaws.AmazonWebServiceRequest.copyPrivateRequestParameters()Ljava/util/Map;
        at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3506) ~[?:?]
        at com.amazonaws.services.s3.AmazonS3Client.headBucket(AmazonS3Client.java:1031) ~[?:?]
        at com.amazonaws.services.s3.AmazonS3Client.doesBucketExist(AmazonS3Client.java:994) ~[?:?]
        at org.apache.hadoop.fs.s3a.S3AFileSystem.initialize(S3AFileSystem.java:297) ~[?:?]

``