Use RabbitMQ as firehose type - Could not resolve type id 'rabbitmq' into a subtype of

Hi,

Currently, we try to input data into Druid via RabbitMQ. Here’s our spec file.

"ioConfig" : {
  "type" : "realtime",
  "firehose": {
    "type": "rabbitmq",
    "connection" : {
        "host": "localhost",
        "port": "5672",
        "username": "guest",
        "password": "guest"
    },
    "config" : {
        "exchange": "",
        "queue": "druid",
        "routingKey": "druid",
        "durable": "true",
        "exclusive": "false",
        "autoDelete": "false",
        "maxRetries": "10",
        "retryIntervalSeconds": "1",
        "maxDurationSeconds": "300"
    }
  },
  "plumber": {
    "type": "realtime"
  }
},

``

We had ensure there are amqp-client-3.2.1.jar in Druid lib directory.

ubuntu@ip-172-30-1-252:~/druid/druid-0.8.3$ ls -al lib/amqp-client-3.2.1.jar
-rw-rw-r-- 1 ubuntu ubuntu 359243 Nov 7 2013 lib/amqp-client-3.2.1.jar
ubuntu@ip-172-30-1-252:~/druid/druid-0.8.3$ pwd
/home/ubuntu/druid/druid-0.8.3

``

We execute our “realtime” via

java -Xmx512m -Duser.timezone=UTC -Dfile.encoding=UTF-8
-Ddruid.realtime.specFile=examples/indexing/wikipedia.spec
-classpath “config/_common:config/realtime:lib/*”
io.druid.cli.Main server realtime

``

2016-03-10T09:08:05,094 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.guice.RealtimeManagerConfig] from props[druid.realtime.] as [io.druid.guice.RealtimeManagerConfig@3baf6936]
Exception in thread “main” com.google.inject.CreationException: Guice creation errors:

  1. Error injecting constructor, java.lang.RuntimeException: com.fasterxml.jackson.databind.JsonMappingException: Could not resolve type id ‘rabbitmq’ into a subtype of [simple type, class io.druid.data.input.FirehoseFactory]
    at [Source: examples/indexing/wikipedia.spec; line: 45, column: 9] (through reference chain: java.util.ArrayList[0])
    at io.druid.guice.FireDepartmentsProvider.(FireDepartmentsProvider.java:41)
    while locating io.druid.guice.FireDepartmentsProvider
    at io.druid.guice.RealtimeModule.configure(RealtimeModule.java:84)
    while locating java.util.List<io.druid.segment.realtime.FireDepartment>
    for parameter 0 at io.druid.segment.realtime.RealtimeMetricsMonitor.(RealtimeMetricsMonitor.java:42)
    while locating io.druid.segment.realtime.RealtimeMetricsMonitor
    at io.druid.server.metrics.MetricsModule.getMonitorScheduler(MetricsModule.java:87)
    at io.druid.server.metrics.MetricsModule.getMonitorScheduler(MetricsModule.java:87)
    while locating com.metamx.metrics.MonitorScheduler
    at io.druid.server.metrics.MetricsModule.configure(MetricsModule.java:72)
    while locating com.metamx.metrics.MonitorScheduler annotated with @com.google.inject.name.Named(value=ForTheEagerness)

1 error
at com.google.inject.internal.Errors.throwCreationExceptionIfErrorsExist(Errors.java:448)
at com.google.inject.internal.InternalInjectorCreator.injectDynamically(InternalInjectorCreator.java:184)
at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:110)
at com.google.inject.Guice.createInjector(Guice.java:96)
at com.google.inject.Guice.createInjector(Guice.java:73)
at com.google.inject.Guice.createInjector(Guice.java:62)
at io.druid.initialization.Initialization.makeInjectorWithModules(Initialization.java:415)
at io.druid.cli.GuiceRunnable.makeInjector(GuiceRunnable.java:55)
at io.druid.cli.ServerRunnable.run(ServerRunnable.java:37)
at io.druid.cli.Main.main(Main.java:99)
Caused by: java.lang.RuntimeException: com.fasterxml.jackson.databind.JsonMappingException: Could not resolve type id ‘rabbitmq’ into a subtype of [simple type, class io.druid.data.input.FirehoseFactory]
at [Source: examples/indexing/wikipedia.spec; line: 45, column: 9] (through reference chain: java.util.ArrayList[0])
at com.google.common.base.Throwables.propagate(Throwables.java:160)
at io.druid.guice.FireDepartmentsProvider.(FireDepartmentsProvider.java:52)


However, we still get the following error

Any idea what step we had missed out?

Thanks.
Cheok

Hi,

Here’s are the steps which we had done to make it works.

  1. Change

druid.extensions.coordinates=[“io.druid.extensions:druid-examples”,“io.druid.extensions:druid-kafka-eight”]

``

to

druid.extensions.coordinates=[“io.druid.extensions:druid-examples”,“io.druid.extensions:druid-kafka-eight”,“io.druid.extensions:druid-rabbitmq”]

``

in config/_common/common.runtime.properties

  1. Download lyra jar file to

ubuntu@ip-172-30-1-252:~/druid/druid-0.8.3/lib$ pwd
/home/ubuntu/druid/druid-0.8.3/lib
wget http://central.maven.org/maven2/net/jodah/lyra/0.3.1/lyra-0.3.1.jar

``