What's the correct step to use druid kafka-emitter extension in druid-0.16.1?

Hi all, i download kafka-emitter jars by “java -classpath “lib/*” org.apache.druid.cli.Main tools pull-deps --clean -c io.druid.extensions.contrib:kafka-emitter:0.12.1” and configure kafka-emitter as https://druid.apache.org/docs/latest/development/extensions-contrib/kafka-emitter.html#introduction supposed.

Here is my related configuration.

#common.runtime.properties
druid.monitoring.monitors=[“org.apache.druid.java.util.metrics.JvmMonitor”]

druid.emitter=composing

druid.emitter.composing.emitters=[“logging”,“kafka”]

druid.emitter.logging.logLevel=info

druid.emitter.kafka.bootstrap.servers=tsdb2:9092

druid.emitter.kafka.metric.topic=druid-metric-test

druid.emitter.kafka.alert.topic=druid-alter-test

``

luochong@tsdb2:~/druid-current/extensions/kafka-emitter$ ls
kafka-clients-0.10.2.0.jar lz4-1.3.0.jar snappy-java-1.1.2.6.jar

kafka-emitter-0.12.1.jar slf4j-api-1.6.4.jar

``

The Exception is thrown at the time i startup druid:

2020-04-24T16:49:36,638 INFO [main] org.apache.druid.guice.JsonConfigurator - Loaded class[class org.apache.druid.java.util.emitter.core.LoggingEmitterConfig] from props[druid.emitter.logging.] as [LoggingEmitterConfig{loggerClass=‘org.apache.druid.java.util.emitter.core.LoggingEmitter’, logLevel=‘info’}]

Exception in thread “main” java.lang.RuntimeException: com.google.inject.CreationException: Unable to create injector, see the following errors:

  1. No implementation for org.apache.druid.java.util.emitter.core.Emitter annotated with @com.google.inject.name.Named(value=kafka) was bound.

while locating org.apache.druid.java.util.emitter.core.Emitter annotated with @com.google.inject.name.Named(value=kafka)

at org.apache.druid.server.emitter.EmitterModule.getServiceEmitter(EmitterModule.java:103) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> org.apache.druid.server.emitter.EmitterModule)

at org.apache.druid.server.emitter.EmitterModule.getServiceEmitter(EmitterModule.java:103) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> org.apache.druid.server.emitter.EmitterModule)

while locating org.apache.druid.java.util.emitter.service.ServiceEmitter

for the 4th parameter of org.apache.druid.server.metrics.MetricsModule.getMonitorScheduler(MetricsModule.java:92)

at org.apache.druid.server.metrics.MetricsModule.getMonitorScheduler(MetricsModule.java:92) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> org.apache.druid.server.metrics.MetricsModule)

while locating org.apache.druid.java.util.metrics.MonitorScheduler

at org.apache.druid.server.metrics.MetricsModule.configure(MetricsModule.java:77) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> org.apache.druid.server.metrics.MetricsModule)

while locating org.apache.druid.java.util.metrics.MonitorScheduler annotated with @com.google.inject.name.Named(value=ForTheEagerness)

1 error

at org.apache.druid.cli.GuiceRunnable.makeInjector(GuiceRunnable.java:72)

at org.apache.druid.cli.ServerRunnable.run(ServerRunnable.java:56)

at org.apache.druid.cli.Main.main(Main.java:113)

Caused by: com.google.inject.CreationException: Unable to create injector, see the followi:[2020-04-24 16:50:46,967] INFO [GroupMetadataManager brokerId=2] Removed 0 expired offsets in 1 milliseconds. (kafka.coordinator.group.GroupMetadataManager)

ng errors:

  1. No implementation for org.apache.druid.java.util.emitter.core.Emitter annotated with @com.google.inject.name.Named(value=kafka) was bound.

while locating org.apache.druid.java.util.emitter.core.Emitter annotated with @com.google.inject.name.Named(value=kafka)

at org.apache.druid.server.emitter.EmitterModule.getServiceEmitter(EmitterModule.java:103) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> org.apache.druid.server.emitter.EmitterModule)

at org.apache.druid.server.emitter.EmitterModule.getServiceEmitter(EmitterModule.java:103) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> org.apache.druid.server.emitter.EmitterModule)

while locating org.apache.druid.java.util.emitter.service.ServiceEmitter

for the 4th parameter of org.apache.druid.server.metrics.MetricsModule.getMonitorScheduler(MetricsModule.java:92)

at org.apache.druid.server.metrics.MetricsModule.getMonitorScheduler(MetricsModule.java:92) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> org.apache.druid.server.metrics.MetricsModule)

while locating org.apache.druid.java.util.metrics.MonitorScheduler

at org.apache.druid.server.metrics.MetricsModule.configure(MetricsModule.java:77) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> org.apache.druid.server.metrics.MetricsModule)

while locating org.apache.druid.java.util.metrics.MonitorScheduler annotated with @com.google.inject.name.Named(value=ForTheEagerness)

1 error

    at com.google.inject.internal.Errors.throwCreationExceptionIfErrorsExist(Errors.java:470)

at com.google.inject.internal.InternalInjectorCreator.injectDynamically(InternalInjectorCreator.java:184)

at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:110)

at com.google.inject.Guice.createInjector(Guice.java:99)

at com.google.inject.Guice.createInjector(Guice.java:73)

at com.google.inject.Guice.createInjector(Guice.java:62)

at org.apache.druid.initialization.Initialization.makeInjectorWithModules(Initialization.java:419)

at org.apache.druid.cli.GuiceRunnable.makeInjector(GuiceRunnable.java:69)

… 2 more

``

It seems druid cant find kafka-emitter plugin.

Does anyone give me some suggestions on configuring kafka-emitter? Thank you.

在 2020年4月24日星期五 UTC+8下午4:54:05,Chong Luo写道:

Here is my related configuration.

#common.runtime.properties
druid.monitoring.monitors=[“org.apache.druid.java.util.metrics.JvmMonitor”]

druid.emitter=composing

druid.emitter.composing.emitters=[“logging”,“kafka”]

druid.emitter.logging.logLevel=info

druid.emitter.kafka.bootstrap.servers=tsdb2:9092

druid.emitter.kafka.metric.topic=druid-metric-test

druid.emitter.kafka.alert.topic=druid-alter-test

``

kafka-clients-0.10.2.0.jar lz4-1.3.0.jar snappy-java-1.1.2.6.jar

kafka-emitter-0.12.1.jar slf4j-api-1.6.4.jar

luochong@tsdb2:~/druid-current/extensions/kafka-emitter$ ls

``

The Exception is thrown at the time i startup druid:

2020-04-24T16:49:36,638 INFO [main] org.apache.druid.guice.JsonConfigurator - Loaded class[class org.apache.druid.java.util.emitter.core.LoggingEmitterConfig] from props[druid.emitter.logging.] as [LoggingEmitterConfig{loggerClass=‘org.apache.druid.java.util.emitter.core.LoggingEmitter’, logLevel=‘info’}]

Exception in thread “main” java.lang.RuntimeException: com.google.inject.CreationException: Unable to create injector, see the following errors:

  1. No implementation for org.apache.druid.java.util.emitter.core.Emitter annotated with @com.google.inject.name.Named(value=kafka) was bound.

while locating org.apache.druid.java.util.emitter.core.Emitter annotated with @com.google.inject.name.Named(value=kafka)

at org.apache.druid.server.emitter.EmitterModule.getServiceEmitter(EmitterModule.java:103) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> org.apache.druid.server.emitter.EmitterModule)

at org.apache.druid.server.emitter.EmitterModule.getServiceEmitter(EmitterModule.java:103) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> org.apache.druid.server.emitter.EmitterModule)

while locating org.apache.druid.java.util.emitter.service.ServiceEmitter

for the 4th parameter of org.apache.druid.server.metrics.MetricsModule.getMonitorScheduler(MetricsModule.java:92)

at org.apache.druid.server.metrics.MetricsModule.getMonitorScheduler(MetricsModule.java:92) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> org.apache.druid.server.metrics.MetricsModule)

while locating org.apache.druid.java.util.metrics.MonitorScheduler

at org.apache.druid.server.metrics.MetricsModule.configure(MetricsModule.java:77) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> org.apache.druid.server.metrics.MetricsModule)

while locating org.apache.druid.java.util.metrics.MonitorScheduler annotated with @com.google.inject.name.Named(value=ForTheEagerness)

1 error

at org.apache.druid.cli.GuiceRunnable.makeInjector(GuiceRunnable.java:72)

at org.apache.druid.cli.ServerRunnable.run(ServerRunnable.java:56)

at org.apache.druid.cli.Main.main(Main.java:113)

Caused by: com.google.inject.CreationException: Unable to create injector, see the followi:[2020-04-24 16:50:46,967] INFO [GroupMetadataManager brokerId=2] Removed 0 expired offsets in 1 milliseconds. (kafka.coordinator.group.GroupMetadataManager)

ng errors:

  1. No implementation for org.apache.druid.java.util.emitter.core.Emitter annotated with @com.google.inject.name.Named(value=kafka) was bound.

while locating org.apache.druid.java.util.emitter.core.Emitter annotated with @com.google.inject.name.Named(value=kafka)

at org.apache.druid.server.emitter.EmitterModule.getServiceEmitter(EmitterModule.java:103) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> org.apache.druid.server.emitter.EmitterModule)

at org.apache.druid.server.emitter.EmitterModule.getServiceEmitter(EmitterModule.java:103) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> org.apache.druid.server.emitter.EmitterModule)

while locating org.apache.druid.java.util.emitter.service.ServiceEmitter

for the 4th parameter of org.apache.druid.server.metrics.MetricsModule.getMonitorScheduler(MetricsModule.java:92)

at org.apache.druid.server.metrics.MetricsModule.getMonitorScheduler(MetricsModule.java:92) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> org.apache.druid.server.metrics.MetricsModule)

while locating org.apache.druid.java.util.metrics.MonitorScheduler

at org.apache.druid.server.metrics.MetricsModule.configure(MetricsModule.java:77) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> org.apache.druid.server.metrics.MetricsModule)

while locating org.apache.druid.java.util.metrics.MonitorScheduler annotated with @com.google.inject.name.Named(value=ForTheEagerness)

1 error

    at com.google.inject.internal.Errors.throwCreationExceptionIfErrorsExist(Errors.java:470)

at com.google.inject.internal.InternalInjectorCreator.injectDynamically(InternalInjectorCreator.java:184)

at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:110)

at com.google.inject.Guice.createInjector(Guice.java:99)

at com.google.inject.Guice.createInjector(Guice.java:73)

at com.google.inject.Guice.createInjector(Guice.java:62)

at org.apache.druid.initialization.Initialization.makeInjectorWithModules(Initialization.java:419)

at org.apache.druid.cli.GuiceRunnable.makeInjector(GuiceRunnable.java:69)

… 2 more

``

It seems druid cant find kafka-emitter plugin.

Does anyone give me some suggestions on configuring kafka-emitter? Thank you.

Hi all, i download kafka-emitter jars by “java -classpath “lib/*” org.apache.druid.cli.Main tools pull-deps --clean -c io.druid.extensions.contrib:kafka-emitter:0.12.1” and configure kafka-emitter as https://druid.apache.org/docs/latest/development/extensions-contrib/kafka-emitter.html#introduction supposed.

The problem is solved. I downloaded wrong version of kafka-emitter jars. After downloading new version of kafka-emitter by “java -classpath “lib/*” org.apache.druid.cli.Main tools pull-deps -c org.apache.druid.extensions.contrib:kafka-emitter:0.16.1-incubating” , everything works fine.