io.druid.cli.Main tools pull-deps - Missing default hadoop coordinates.

I attempted to download all my Druid dependencies directly via the “pull-deps” command (http://druid.io/docs/latest/tutorials/firewall.html), but unfortunately some of my dependencies are missing.
My setup:


wget http:``//static``.druid.io``/artifacts/releases/druid-0``.8.3-bin.``tar``.gz

tar -zxvf druid-0.8.3-bin.``tar``.gz

cd druid-0.8.3


# Configured Druid *.properties files.

# Windows
# java -classpath "config\_common;lib\*" io.druid.cli.Main tools pull-deps

java -classpath "config/_common:lib/*" io.druid.cli.Main tools pull-deps

``

``

My Druid configuration (config/_common/common.runtime.properties):

druid.extensions.coordinates=[“io.druid.extensions:postgresql-metadata-storage”,“io.druid.extensions:druid-kafka-eight”,“io.druid.extensions:druid-hdfs-storage”,“io.druid.extensions:druid-histogram”]
druid.extensions.localRepository=extensions-repo

druid.indexer.task.defaultHadoopCoordinates=[“org.apache.hadoop:hadoop-client:2.7.1”,“org.apache.hadoop:hadoop-hdfs:2.7.1”]

``

My hadoop-client 2.7.1 and hadoop-hdfs 2.7.1 dependencies are missing (Only 2.3.0 versions exist):

java -classpath “config/_common:lib/*” io.druid.cli.Main tools pull-deps | grep -e “hadoop-[ch]”

2016-02-11T01:22:13,745 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.3-test/extensions-repo/org/apache/hadoop/hadoop-client/2.3.0/hadoop-client-2.3.0.jar]
2016-02-11T01:22:13,745 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.3-test/extensions-repo/org/apache/hadoop/hadoop-common/2.3.0/hadoop-common-2.3.0.jar]
2016-02-11T01:22:13,747 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.3-test/extensions-repo/org/apache/hadoop/hadoop-hdfs/2.3.0/hadoop-hdfs-2.3.0.jar]
2016-02-11T01:22:14,247 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.3-test/extensions-repo/org/apache/hadoop/hadoop-client/2.3.0/hadoop-client-2.3.0.jar]
2016-02-11T01:22:14,248 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.3-test/extensions-repo/org/apache/hadoop/hadoop-common/2.3.0/hadoop-common-2.3.0.jar]
2016-02-11T01:22:14,251 INFO [main] io.druid.initialization.Initialization - Added URL[file:/opt/druid-0.8.3-test/extensions-repo/org/apache/hadoop/hadoop-hdfs/2.3.0/hadoop-hdfs-2.3.0.jar]

``

**Question: ** Is there some setting I am missing to pull both
these hadoop-client 2.7.1 and hadoop-hdfs 2.7.1 dependencies down, or is
this an issue with Druid?

Hi Mark, if you are starting out Druid for the first time, you might want to consider just using 0.9.0-RC, as it has a completely different setup for dealing with hadoop dependencies:

http://druid.io/docs/0.9.0-rc1/operations/other-hadoop.html

Unfortunately, I think I will have to stay with the Druid 0.8.x releases, as my product is going into production very soon, and I don’t have the time to wait for Druid 0.9.0 to be production ready.

Hey Mark,

Try adding: --no-default-hadoop --hadoop-coordinate org.apache.hadoop:hadoop-client:2.7.1 --hadoop-coordinate org.apache.hadoop:hadoop-hdfs:2.7.1

Hi Mark,

Has this issue been resolved?

Unfortunately I am using Druid 0.8.3 and the newer options such as “–hadoop-coordinate” are not present.

See https://github.com/druid-io/druid/blob/0.8.3/services/src/main/java/io/druid/cli/PullDependencies.java

So for now, only version 2.3.0 of the hadoop-client and hadoop-hdfs libraries are downloaded, which means I have to manually add these libraries to my Druid classpath.

ls extensions-repo/org/apache/hadoop/hadoop-client/

2.3.0
ls extensions-repo/org/apache/hadoop/hadoop-hdfs/

2.3.0

``