Azure storage account path for druid.storage.storageDirectory

Hey guys,

We’re trying to set up deep storage for druid under azure and keep running into:

2017-10-16T17:13:59,623 INFO [task-runner-0-priority-0] io.druid.indexing.common.task.HadoopTask - Hadoop Container Druid Classpath is set to [file:/opt/druid/hadoop-dependencies/hadoop-client/2.3.0/]
2017-10-16T17:13:59,644 ERROR [task-runner-0-priority-0] io.druid.indexing.overlord.ThreadPoolTaskRunner - Uncaught Throwable while running task[HadoopIndexTask{id=index_hadoop_wikiticker_2017-10-16T17:13:52.197Z, type=index_hadoop, dataSource=wikiticker}]
java.lang.AbstractMethodError: io.druid.storage.azure.AzureDataSegmentPusher.getPathForHadoop()Ljava/lang/String;
	at io.druid.indexing.common.task.HadoopIndexTask.run(HadoopIndexTask.java:181) ~[druid-indexing-service-0.10.0.jar:0.10.0]
	at io.druid.indexing.overlord.ThreadPoolTaskRunner$ThreadPoolTaskRunnerCallable.call(ThreadPoolTaskRunner.java:436) [druid-indexing-service-0.10.0.jar:0.10.0]
	at io.druid.indexing.overlord.ThreadPoolTaskRunner$ThreadPoolTaskRunnerCallable.call(ThreadPoolTaskRunner.java:408) [druid-indexing-service-0.10.0.jar:0.10.0]
	at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_144]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_144]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_144]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_144]

I am finding no documentation for what to set the druid.storage.storageDirectory property to and when I remove it entirely it throws an exception that it can't parse an empty string.

Does anyone have any insight as to what this property should be?

Thanks,
Gregory

It turns out that my azure jar files were a little out of date. Though now I am getting a

java.lang.Exception: java.lang.NullPointerException: segmentOutputPath

at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462) ~[hadoop-mapreduce-client-common-2.3.0.jar:?]

at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:522) [hadoop-mapreduce-client-common-2.3.0.jar:?]

Caused by: java.lang.NullPointerException: segmentOutputPath

at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:229) ~[guava-16.0.1.jar:?]

at io.druid.indexer.HadoopDruidIndexerConfig.verify(HadoopDruidIndexerConfig.java:592) ~[druid-indexing-hadoop-0.10.0.jar:0.10.0]

at io.druid.indexer.HadoopDruidIndexerConfig.fromConfiguration(HadoopDruidIndexerConfig.java:212) ~[druid-indexing-hadoop-0.10.0.jar:0.10.0]

at io.druid.indexer.HadoopDruidIndexerMapper.setup(HadoopDruidIndexerMapper.java:48) ~[druid-indexing-hadoop-0.10.0.jar:0.10.0]

at io.druid.indexer.DetermineHashedPartitionsJob$DetermineCardinalityMapper.setup(DetermineHashedPartitionsJob.java:224) ~[druid-indexing-hadoop-0.10.0.jar:0.10.0]

at io.druid.indexer.DetermineHashedPartitionsJob$DetermineCardinalityMapper.run(DetermineHashedPartitionsJob.java:282) ~[druid-indexing-hadoop-0.10.0.jar:0.10.0]

at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) ~[hadoop-mapreduce-client-core-2.3.0.jar:?]

at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340) ~[hadoop-mapreduce-client-core-2.3.0.jar:?]

at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243) ~[hadoop-mapreduce-client-common-2.3.0.jar:?]

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_144]

at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_144]

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_144]

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_144]

at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_144]