Druid GCS deep storage integration issue

Hi

After setting up druid instances(0.13.0-incubating) on Google Compute Engine and using Google Cloud Storage as Deep Storage, when I am trying to load file using: bin/post-index-task --file quickstart/tutorial/wikipedia-index.json

The job fails with below error in middleManager logs:

""2019-02-01T13:18:10,729 INFO [main] org.apache.druid.guice.JsonConfigurator - Loaded class[class org.apache.druid.query.lookup.LookupListeningAnnouncerConfig] from props[druid.lookup.] as [ListeningAnnouncerConfig{listenersPath=’/druid/listeners’}]

""2019-02-01T13:18:10,730 ERROR [main] org.apache.druid.cli.CliPeon - Error when starting up. Failing.

" com.google.inject.ProvisionException: Unable to provision, see the following errors:

  1. Unknown provider[google] of Key[type=org.apache.druid.segment.loading.DataSegmentPusher, annotation=[none]], known options[[local]]

at org.apache.druid.guice.PolyBind.createChoice(PolyBind.java:70) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> org.apache.druid.guice.LocalDataStorageDruidModule)

while locating org.apache.druid.segment.loading.DataSegmentPusher

for the 4th parameter of org.apache.druid.indexing.common.TaskToolboxFactory.(TaskToolboxFactory.java:113)

at org.apache.druid.cli.CliPeon$1.configure(CliPeon.java:200) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> org.apache.druid.cli.CliPeon$1)

while locating org.apache.druid.indexing.common.TaskToolboxFactory

for the 1st parameter of org.apache.druid.indexing.overlord.SingleTaskBackgroundRunner.(SingleTaskBackgroundRunner.java:95)

at org.apache.druid.cli.CliPeon$1.configure(CliPeon.java:239) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> org.apache.druid.cli.CliPeon$1)

while locating org.apache.druid.indexing.overlord.SingleTaskBackgroundRunner

while locating org.apache.druid.indexing.overlord.TaskRunner

for the 4th parameter of org.apache.druid.indexing.worker.executor.ExecutorLifecycle.(ExecutorLifecycle.java:79)

at org.apache.druid.cli.CliPeon$1.configure(CliPeon.java:223) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> org.apache.druid.cli.CliPeon$1)

while locating org.apache.druid.indexing.worker.executor.ExecutorLifecycle

1 error

at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:1028) ~[guice-4.1.0.jar:?]

at com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1050) ~[guice-4.1.0.jar:?]

at org.apache.druid.guice.LifecycleModule$2.start(LifecycleModule.java:132) ~[druid-api-0.13.0-incubating.jar:0.13.0-incubating]

at org.apache.druid.cli.GuiceRunnable.initLifecycle(GuiceRunnable.java:107) [druid-services-0.13.0-incubating.jar:0.13.0-incubating]

at org.apache.druid.cli.CliPeon.run(CliPeon.java:348) [druid-services-0.13.0-incubating.jar:0.13.0-incubating]

at org.apache.druid.cli.Main.main(Main.java:118) [druid-services-0.13.0-incubating.jar:0.13.0-incubating]

"2019-02-01T13:18:11,077 INFO [forking-task-runner-2-[index_wikipedia_2019-02-01T13:18:06.842Z]] org.apache.druid.indexing.overlord.ForkingTaskRunner - Process exited with status[1] for task: index_wikipedia_2019-02-01T13:18:06.842Z

I have attached the common.runtime.properties. I can see the google extension loading in the logs.

INFO [main] org.apache.druid.guice.JsonConfigurator - Loaded class[class org.apache.druid.guice.ExtensionsConfig] from props[druid.extensions.] as [ExtensionsConfig{searchCurrentClassloader=true, directory=’/druid/apache-druid-0.13.0-incubating/extensions’, useExtensionClassloaderFirst=false, hadoopDependenciesDir=‘hadoop-dependencies’, hadoopContainerDruidClasspath=‘null’, addExtensionsToHadoopContainer=false, loadList=[druid-google-extensions, druid-kafka-indexing-service, postgresql-metadata-storage]}]

""2019-02-01T13:18:08,285 INFO [main] org.apache.druid.initialization.Initialization - Loading extension [druid-google-extensions] for class [interface org.apache.druid.cli.CliCommandCreator]

Kindly help.

common.runtime.properties.j2 (3.19 KB)

Can you attach a more full startup log?
Specifically, do you get these lines?

Note that the first line below is a different “Loading extension” from the one you mentioned — it’s the one looking for DruidModule not CliCommandCreator.

INFO [main] org.apache.druid.initialization.Initialization - Loading extension [druid-google-extensions] for class [interface org.apache.druid.initialization.DruidModule]

INFO [main] org.apache.druid.initialization.Initialization - Adding implementation [org.apache.druid.storage.google.GoogleStorageDruidModule] for class [interface org.apache.druid.initialization.DruidModule] from local file system extension

INFO [main] org.apache.druid.storage.google.GoogleStorageDruidModule - Getting jackson modules…

INFO [main] org.apache.druid.storage.google.GoogleStorageDruidModule - Getting jackson modules…

Hi David

Based on your suggestion, realised that google extension was not working.

It was due to the maven extension. I was using io.druid instead of****org.apache.druid.extensions.contrib.

Thanks a lot.