Druid 0.11.0 Version upgrade facing issue related to Zookeeper

Hi All,

After Upgrade to druid 0.11.0 facing below issue.Is there anythings missing ?

When i started middle manager and historical .

2018-01-17T14:53:13,983 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.storage.hdfs.HdfsKerberosConfig] from props[druid.hadoop.security.kerberos.] as [io.druid.storage.hdfs.HdfsKerberosConfig@0]

2018-01-17T14:53:13,986 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.curator.CuratorConfig] from props[druid.zk.service.] as [io.druid.curator.CuratorConfig@2b0b7e5a]

2018-01-17T14:53:13,998 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.curator.ExhibitorConfig] from props[druid.exhibitor.service.] as [io.druid.curator.ExhibitorConfig@5f14761c]

2018-01-17T14:53:14,066 INFO [main] org.apache.curator.utils.Compatibility - Running in ZooKeeper 3.4.x compatibility mode

2018-01-17T14:53:14,067 WARN [main] org.apache.curator.retry.ExponentialBackoffRetry - maxRetries too large (30). Pinning to 29

2018-01-17T14:53:14,088 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.initialization.ZkPathsConfig] from props[druid.zk.paths.] as [io.druid.server.initialization.ZkPathsConfig@22e2266d]

2018-01-17T14:53:14,090 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.server.initialization.IndexerZkConfig] from props[druid.zk.paths.indexer.] as [io.druid.server.initialization.IndexerZkConfig@ff5f27ed]

2018-01-17T14:53:14,114 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.indexing.overlord.config.RemoteTaskRunnerConfig] from props[druid.indexer.runner.] as [RemoteTaskRunnerConfig{taskAssignmentTimeout=PT5M, taskCleanupTimeout=PT15M, minWorkerVersion=‘0’, maxZnodeBytes=524288, taskShutdownLinkTimeout=PT1M, pendingTasksRunnerNumThreads=1, maxRetriesBeforeBlacklist=5, taskBlackListBackoffTimeMillis=PT15M, taskBlackListCleanupPeriod=PT5M, maxPercentageBlacklistWorkers= 20}]

2018-01-17T14:53:14,115 WARN [main] org.apache.curator.retry.ExponentialBackoffRetry - maxRetries too large (30). Pinning to 29

2018-01-17T14:53:14,119 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.indexing.worker.config.WorkerConfig] from props[druid.worker.] as [io.druid.indexing.worker.config.WorkerConfig@a83495d]

2018-01-17T14:53:14,128 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.indexing.overlord.config.ForkingTaskRunnerConfig] from props[druid.indexer.runner.] as [io.druid.indexing.overlord.config.ForkingTaskRunnerConfig@6d33a66e]

2018-01-17T14:53:14,139 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.indexing.common.config.TaskConfig] from props[druid.indexer.task.] as [io.druid.indexing.common.config.TaskConfig@3b2553d9]

2018-01-17T14:53:14,141 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.storage.hdfs.tasklog.HdfsTaskLogsConfig] from props[druid.indexer.logs.] as [io.druid.storage.hdfs.tasklog.HdfsTaskLogsConfig@5f36c8e3]

2018-01-17T14:53:14,143 ERROR [main] io.druid.cli.CliMiddleManager - Error when starting up. Failing.

com.google.inject.ProvisionException: Unable to provision, see the following errors:

  1. Error in custom provider, java.lang.NoSuchMethodError: io.druid.indexing.worker.Worker.(Ljava/lang/String;Ljava/lang/String;ILjava/lang/String;)V

at io.druid.cli.CliMiddleManager$1.getWorker(CliMiddleManager.java:106) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> io.druid.cli.CliMiddleManager$1)

at io.druid.cli.CliMiddleManager$1.getWorker(CliMiddleManager.java:106) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> io.druid.cli.CliMiddleManager$1)

while locating io.druid.indexing.worker.Worker

for the 5th parameter of io.druid.indexing.worker.WorkerCuratorCoordinator.(WorkerCuratorCoordinator.java:74)

at io.druid.cli.CliMiddleManager$1.configure(CliMiddleManager.java:93) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> io.druid.cli.CliMiddleManager$1)

while locating io.druid.indexing.worker.WorkerCuratorCoordinator

for the 3rd parameter of io.druid.indexing.worker.WorkerTaskMonitor.(WorkerTaskMonitor.java:85)

at io.druid.cli.CliMiddleManager$1.configure(CliMiddleManager.java:92) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> io.druid.cli.CliMiddleManager$1)

while locating io.druid.indexing.worker.WorkerTaskMonitor

Caused by: java.lang.NoSuchMethodError: io.druid.indexing.worker.Worker.(Ljava/lang/String;Ljava/lang/String;ILjava/lang/String;)V

at io.druid.cli.CliMiddleManager$1.getWorker(CliMiddleManager.java:110)

at io.druid.cli.CliMiddleManager$1$$FastClassByGuice$$6739c387.invoke()

at com.google.inject.internal.ProviderMethod$FastClassProviderMethod.doProvision(ProviderMethod.java:264)

at com.google.inject.internal.ProviderMethod$Factory.provision(ProviderMethod.java:401)

at com.google.inject.internal.ProviderMethod$Factory.get(ProviderMethod.java:376)

at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46)

at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092)

at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)

at com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:194)

at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:41)

at com.google.inject.internal.SingleParameterInjector.inject(SingleParameterInjector.java:38)

at com.google.inject.internal.SingleParameterInjector.getAll(SingleParameterInjector.java:62)

at com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:110)

at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:90)

at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:268)

at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46)

at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092)

at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)

at io.druid.guice.LifecycleScope$1.get(LifecycleScope.java:68)

at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:41)

at com.google.inject.internal.SingleParameterInjector.inject(SingleParameterInjector.java:38)

at com.google.inject.internal.SingleParameterInjector.getAll(SingleParameterInjector.java:62)

at com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:110)

at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:90)

at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:268)

at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46)

at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092)

at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)

at io.druid.guice.LifecycleScope$1.get(LifecycleScope.java:68)

at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:41)

at com.google.inject.internal.InjectorImpl$2$1.call(InjectorImpl.java:1019)

at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1085)

at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:1015)

at com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1050)

at io.druid.guice.LifecycleModule$2.start(LifecycleModule.java:154)

at io.druid.cli.GuiceRunnable.initLifecycle(GuiceRunnable.java:103)

at io.druid.cli.ServerRunnable.run(ServerRunnable.java:41)

at io.druid.cli.Main.main(Main.java:108)

  1. Error in custom provider, java.lang.NoSuchMethodError: org.apache.curator.CuratorZookeeperClient.(Lorg/apache/curator/utils/ZookeeperFactory;Lorg/apache/curator/ensemble/EnsembleProvider;IILorg/apache/zookeeper/Watcher;Lorg/apache/curator/RetryPolicy;ZLorg/apache/curator/connection/ConnectionHandlingPolicy;)V

at io.druid.curator.CuratorModule.makeCurator(CuratorModule.java:75) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> io.druid.curator.CuratorModule)

at io.druid.curator.CuratorModule.makeCurator(CuratorModule.java:75) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> io.druid.curator.CuratorModule)

while locating org.apache.curator.framework.CuratorFramework

for the 2nd parameter of io.druid.indexing.worker.WorkerTaskMonitor.(WorkerTaskMonitor.java:85)

at io.druid.cli.CliMiddleManager$1.configure(CliMiddleManager.java:92) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> io.druid.cli.CliMiddleManager$1)

while locating io.druid.indexing.worker.WorkerTaskMonitor

Caused by: java.lang.NoSuchMethodError: org.apache.curator.CuratorZookeeperClient.(Lorg/apache/curator/utils/ZookeeperFactory;Lorg/apache/curator/ensemble/EnsembleProvider;IILorg/apache/zookeeper/Watcher;Lorg/apache/curator/RetryPolicy;ZLorg/apache/curator/connection/ConnectionHandlingPolicy;)V

at org.apache.curator.framework.imps.CuratorFrameworkImpl.(CuratorFrameworkImpl.java:132)

at org.apache.curator.framework.CuratorFrameworkFactory$Builder.build(CuratorFrameworkFactory.java:158)

at io.druid.curator.CuratorModule.makeCurator(CuratorModule.java:82)

at io.druid.curator.CuratorModule$$FastClassByGuice$$7ca37876.invoke()

at com.google.inject.internal.ProviderMethod$FastClassProviderMethod.doProvision(ProviderMethod.java:264)

at com.google.inject.internal.ProviderMethod$Factory.provision(ProviderMethod.java:401)

at com.google.inject.internal.ProviderMethod$Factory.get(ProviderMethod.java:376)

at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46)

at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092)

at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)

at com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:194)

at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:41)

at com.google.inject.internal.SingleParameterInjector.inject(SingleParameterInjector.java:38)

at com.google.inject.internal.SingleParameterInjector.getAll(SingleParameterInjector.java:62)

at com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:110)

at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:90)

at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:268)

at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46)

at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092)

at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)

at io.druid.guice.LifecycleScope$1.get(LifecycleScope.java:68)

at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:41)

at com.google.inject.internal.InjectorImpl$2$1.call(InjectorImpl.java:1019)

at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1085)

at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:1015)

at com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1050)

at io.druid.guice.LifecycleModule$2.start(LifecycleModule.java:154)

at io.druid.cli.GuiceRunnable.initLifecycle(GuiceRunnable.java:103)

at io.druid.cli.ServerRunnable.run(ServerRunnable.java:41)

at io.druid.cli.Main.main(Main.java:108)

  1. Error in custom provider, java.lang.NoSuchMethodError: org.apache.curator.CuratorZookeeperClient.(Lorg/apache/curator/utils/ZookeeperFactory;Lorg/apache/curator/ensemble/EnsembleProvider;IILorg/apache/zookeeper/Watcher;Lorg/apache/curator/RetryPolicy;ZLorg/apache/curator/connection/ConnectionHandlingPolicy;)V

at io.druid.curator.CuratorModule.makeCurator(CuratorModule.java:75) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> io.druid.curator.CuratorModule)

at io.druid.curator.CuratorModule.makeCurator(CuratorModule.java:75) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> io.druid.curator.CuratorModule)

while locating org.apache.curator.framework.CuratorFramework

for the 4th parameter of io.druid.indexing.worker.WorkerCuratorCoordinator.(WorkerCuratorCoordinator.java:74)

at io.druid.cli.CliMiddleManager$1.configure(CliMiddleManager.java:93) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> io.druid.cli.CliMiddleManager$1)

while locating io.druid.indexing.worker.WorkerCuratorCoordinator

for the 3rd parameter of io.druid.indexing.worker.WorkerTaskMonitor.(WorkerTaskMonitor.java:85)

at io.druid.cli.CliMiddleManager$1.configure(CliMiddleManager.java:92) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> io.druid.cli.CliMiddleManager$1)

while locating io.druid.indexing.worker.WorkerTaskMonitor

Caused by: java.lang.NoSuchMethodError: org.apache.curator.CuratorZookeeperClient.(Lorg/apache/curator/utils/ZookeeperFactory;Lorg/apache/curator/ensemble/EnsembleProvider;IILorg/apache/zookeeper/Watcher;Lorg/apache/curator/RetryPolicy;ZLorg/apache/curator/connection/ConnectionHandlingPolicy;)V

at org.apache.curator.framework.imps.CuratorFrameworkImpl.(CuratorFrameworkImpl.java:132)

at org.apache.curator.framework.CuratorFrameworkFactory$Builder.build(CuratorFrameworkFactory.java:158)

at io.druid.curator.CuratorModule.makeCurator(CuratorModule.java:82)

at io.druid.curator.CuratorModule$$FastClassByGuice$$7ca37876.invoke()

at com.google.inject.internal.ProviderMethod$FastClassProviderMethod.doProvision(ProviderMethod.java:264)

at com.google.inject.internal.ProviderMethod$Factory.provision(ProviderMethod.java:401)

at com.google.inject.internal.ProviderMethod$Factory.get(ProviderMethod.java:376)

at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46)

at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092)

at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)

at com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:194)

at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:41)

at com.google.inject.internal.SingleParameterInjector.inject(SingleParameterInjector.java:38)

at com.google.inject.internal.SingleParameterInjector.getAll(SingleParameterInjector.java:62)

at com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:110)

at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:90)

at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:268)

at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46)

at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092)

at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)

at io.druid.guice.LifecycleScope$1.get(LifecycleScope.java:68)

at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:41)

at com.google.inject.internal.SingleParameterInjector.inject(SingleParameterInjector.java:38)

at com.google.inject.internal.SingleParameterInjector.getAll(SingleParameterInjector.java:62)

at com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:110)

at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:90)

at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:268)

at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46)

at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092)

at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)

at io.druid.guice.LifecycleScope$1.get(LifecycleScope.java:68)

at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:41)

at com.google.inject.internal.InjectorImpl$2$1.call(InjectorImpl.java:1019)

at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1085)

at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:1015)

at com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1050)

at io.druid.guice.LifecycleModule$2.start(LifecycleModule.java:154)

at io.druid.cli.GuiceRunnable.initLifecycle(GuiceRunnable.java:103)

at io.druid.cli.ServerRunnable.run(ServerRunnable.java:41)

at io.druid.cli.Main.main(Main.java:108)

3 errors

at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:1028) ~[guice-4.1.0.jar:?]

at com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1050) ~[guice-4.1.0.jar:?]

at io.druid.guice.LifecycleModule$2.start(LifecycleModule.java:154) ~[druid-api-0.11.0.jar:0.10.1-iap6]

at io.druid.cli.GuiceRunnable.initLifecycle(GuiceRunnable.java:103) [druid-services-0.10.1-iap6.jar:0.10.1-iap6]

at io.druid.cli.ServerRunnable.run(ServerRunnable.java:41) [druid-services-0.10.1-iap6.jar:0.10.1-iap6]

at io.druid.cli.Main.main(Main.java:108) [druid-services-0.10.1-iap6.jar:0.10.1-iap6]

I saw Apache Curator is playing a role in this exception. Is it really installed in the same cluster ? Have you tried disabling it and try again ?

How i can disable Apache Curator ? Do i need to remove from pom file and build druid jar again or something else.

Regards,

Sudhanshu Lenka