Error injecting constructor, java.io.IOException: No FileSystem for scheme: http

Hi All,

Can any one help on this issue ?

Setup : HDFS — Cluster (1-master ,2 Slaves)

Druid — Cluster (1-(zk,coordinator,overload),1-(historical,middile manager),1-(broker))

Version using for Druid : imply-2.4.3

HDFS : 2.7.4

common.runtime.properties :

For HDFS:

druid.storage.type=hdfs

druid.storage.storageDirectory=http://hadoopmachince:9000/druid/segments

i am not able to figure out the problem.Please any one can suggest possible issue ?

2018-01-16T09:56:55,122 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[interface io.druid.client.cache.CacheProvider] from props[druid.cache.] as [io.druid.client.cache.CaffeineCacheProvider@782fd504]

2018-01-16T09:56:55,165 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.client.cache.CacheConfig] from props[druid.realtime.cache.] as [io.druid.client.cache.CacheConfig@10e9a5fe]

2018-01-16T09:56:55,168 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.query.lookup.LookupListeningAnnouncerConfig] from props[druid.lookup.] as [ListeningAnnouncerConfig{listenersPath=’/druid/listeners’}]

2018-01-16T09:56:55,169 ERROR [main] io.druid.cli.CliPeon - Error when starting up. Failing.

com.google.inject.ProvisionException: Unable to provision, see the following errors:

  1. Error injecting constructor, java.io.IOException: No FileSystem for scheme: http

at io.druid.storage.hdfs.HdfsDataSegmentPusher.(HdfsDataSegmentPusher.java:65)

while locating io.druid.storage.hdfs.HdfsDataSegmentPusher

at io.druid.storage.hdfs.HdfsStorageDruidModule.configure(HdfsStorageDruidModule.java:97) (via modules: com.google.inject.util.Modules$OverrideModule -> io.druid.storage.hdfs.HdfsStorageDruidModule)

while locating io.druid.segment.loading.DataSegmentPusher annotated with @com.google.inject.multibindings.Element(setName=,uniqueId=182, type=MAPBINDER, keyType=java.lang.String)

at io.druid.guice.PolyBind.createChoice(PolyBind.java:70) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> io.druid.guice.LocalDataStorageDruidModule)

while locating io.druid.segment.loading.DataSegmentPusher

for the 4th parameter of io.druid.indexing.common.TaskToolboxFactory.(TaskToolboxFactory.java:108)

at io.druid.cli.CliPeon$1.configure(CliPeon.java:162) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> io.druid.cli.CliPeon$1)

while locating io.druid.indexing.common.TaskToolboxFactory

for the 1st parameter of io.druid.indexing.overlord.ThreadPoolTaskRunner.(ThreadPoolTaskRunner.java:96)

at io.druid.cli.CliPeon$1.configure(CliPeon.java:189) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> io.druid.cli.CliPeon$1)

while locating io.druid.indexing.overlord.ThreadPoolTaskRunner

while locating io.druid.indexing.overlord.TaskRunner

for the 4th parameter of io.druid.indexing.worker.executor.ExecutorLifecycle.(ExecutorLifecycle.java:78)

at io.druid.cli.CliPeon$1.configure(CliPeon.java:179) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> io.druid.cli.CliPeon$1)

while locating io.druid.indexing.worker.executor.ExecutorLifecycle

1 error

at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:1028) ~[guice-4.1.0.jar:?]

at com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1050) ~[guice-4.1.0.jar:?]

at io.druid.guice.LifecycleModule$2.start(LifecycleModule.java:154) ~[druid-api-0.11.0-iap2.jar:0.11.0-iap2]

at io.druid.cli.GuiceRunnable.initLifecycle(GuiceRunnable.java:101) [druid-services-0.11.0-iap2.jar:0.11.0-iap2]

at io.druid.cli.CliPeon.run(CliPeon.java:283) [druid-services-0.11.0-iap2.jar:0.11.0-iap2]

at io.druid.cli.Main.main(Main.java:108) [druid-services-0.11.0-iap2.jar:0.11.0-iap2]

Caused by: java.io.IOException: No FileSystem for scheme: http

at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2660) ~[?:?]

at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2667) ~[?:?]

at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:94) ~[?:?]

at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2703) ~[?:?]

at org.apache.hadoop.fs.FileSystem$Cache.getUnique(FileSystem.java:2691) ~[?:?]

at org.apache.hadoop.fs.FileSystem.newInstance(FileSystem.java:420) ~[?:?]

at io.druid.storage.hdfs.HdfsDataSegmentPusher.(HdfsDataSegmentPusher.java:70) ~[?:?]

at io.druid.storage.hdfs.HdfsDataSegmentPusher$$FastClassByGuice$$36c2ed09.newInstance() ~[?:?]

at com.google.inject.internal.DefaultConstructionProxyFactory$FastClassProxy.newInstance(DefaultConstructionProxyFactory.java:89) ~[guice-4.1.0.jar:?]

at com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:111) ~[guice-4.1.0.jar:?]

at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:90) ~[guice-4.1.0.jar:?]

at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:268) ~[guice-4.1.0.jar:?]

at com.google.inject.internal.FactoryProxy.get(FactoryProxy.java:56) ~[guice-4.1.0.jar:?]

at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46) ~[guice-4.1.0.jar:?]

at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092) ~[guice-4.1.0.jar:?]

at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40) ~[guice-4.1.0.jar:?]

at com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:194) ~[guice-4.1.0.jar:?]

at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:41) ~[guice-4.1.0.jar:?]

at com.google.inject.internal.InjectorImpl$2$1.call(InjectorImpl.java:1019) ~[guice-4.1.0.jar:?]

at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092) ~[guice-4.1.0.jar:?]

at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:1015) ~[guice-4.1.0.jar:?]

at com.google.inject.spi.ProviderLookup$1.get(ProviderLookup.java:104) ~[guice-4.1.0.jar:?]

at com.google.inject.spi.ProviderLookup$1.get(ProviderLookup.java:104) ~[guice-4.1.0.jar:?]

at com.google.inject.multibindings.MapBinder$RealMapBinder$ValueProvider.get(MapBinder.java:821) ~[guice-multibindings-4.1.0.jar:?]

at io.druid.guice.PolyBind$ConfiggedProvider.get(PolyBind.java:204) ~[druid-api-0.11.0-iap2.jar:0.11.0-iap2]

at com.google.inject.internal.ProviderInternalFactory.provision(ProviderInternalFactory.java:81) ~[guice-4.1.0.jar:?]

at com.google.inject.internal.InternalFactoryToInitializableAdapter.provision(InternalFactoryToInitializableAdapter.java:53) ~[guice-4.1.0.jar:?]

at com.google.inject.internal.ProviderInternalFactory.circularGet(ProviderInternalFactory.java:61) ~[guice-4.1.0.jar:?]

at com.google.inject.internal.InternalFactoryToInitializableAdapter.get(InternalFactoryToInitializableAdapter.java:45) ~[guice-4.1.0.jar:?]

at com.google.inject.internal.SingleParameterInjector.inject(SingleParameterInjector.java:38) ~[guice-4.1.0.jar:?]

at com.google.inject.internal.SingleParameterInjector.getAll(SingleParameterInjector.java:62) ~[guice-4.1.0.jar:?]

at com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:110) ~[guice-4.1.0.jar:?]

at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:90) ~[guice-4.1.0.jar:?]

at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:268) ~[guice-4.1.0.jar:?]

at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46) ~[guice-4.1.0.jar:?]

at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092) ~[guice-4.1.0.jar:?]

at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40) ~[guice-4.1.0.jar:?]

at com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:194) ~[guice-4.1.0.jar:?]

at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:41) ~[guice-4.1.0.jar:?]

at com.google.inject.internal.SingleParameterInjector.inject(SingleParameterInjector.java:38) ~[guice-4.1.0.jar:?]

at com.google.inject.internal.SingleParameterInjector.getAll(SingleParameterInjector.java:62) ~[guice-4.1.0.jar:?]

at com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:110) ~[guice-4.1.0.jar:?]

at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:90) ~[guice-4.1.0.jar:?]

at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:268) ~[guice-4.1.0.jar:?]

at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46) ~[guice-4.1.0.jar:?]

at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092) ~[guice-4.1.0.jar:?]

at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40) ~[guice-4.1.0.jar:?]

at io.druid.guice.LifecycleScope$1.get(LifecycleScope.java:68) ~[druid-api-0.11.0-iap2.jar:0.11.0-iap2]

at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:41) ~[guice-4.1.0.jar:?]

at com.google.inject.internal.FactoryProxy.get(FactoryProxy.java:56) ~[guice-4.1.0.jar:?]

at com.google.inject.internal.SingleParameterInjector.inject(SingleParameterInjector.java:38) ~[guice-4.1.0.jar:?]

at com.google.inject.internal.SingleParameterInjector.getAll(SingleParameterInjector.java:62) ~[guice-4.1.0.jar:?]

at com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:110) ~[guice-4.1.0.jar:?]

at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:90) ~[guice-4.1.0.jar:?]

at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:268) ~[guice-4.1.0.jar:?]

at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46) ~[guice-4.1.0.jar:?]

at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092) ~[guice-4.1.0.jar:?]

at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40) ~[guice-4.1.0.jar:?]

at io.druid.guice.LifecycleScope$1.get(LifecycleScope.java:68) ~[druid-api-0.11.0-iap2.jar:0.11.0-iap2]

at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:41) ~[guice-4.1.0.jar:?]

at com.google.inject.internal.InjectorImpl$2$1.call(InjectorImpl.java:1019) ~[guice-4.1.0.jar:?]

at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1085) ~[guice-4.1.0.jar:?]

at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:1015) ~[guice-4.1.0.jar:?]

regards,

sudhanshu lenka

Is the following an HDFS location?

druid.storage.storageDirectory=http://hadoopmachince:9000/druid/segments

If so, I think you’ll need to use a hdfs:// url instead of http://

Thnks, Jonathan. I change to hdfs for druid.storage.storageDirectory .It works.

Regards,

Sudhanshu Lenka