Docker image for druid kafka indexing service

Hello
I’m working on dockerfile for building image with ready to use druid kafka indexing service. I’m using https://github.com/druid-io/docker-druid as basis.

Here is my current changes https://github.com/nikitsenka/docker-druid-kafka/pull/1/files

Image build works successfully, but having an "“Could not resolve type id ‘kafka’ into a subtype” error when running it in docker

curl -X POST -H ‘Content-Type: application/json’ -d @supervisor-spec.json http://172.17.0.3:8081/druid/indexer/v1/supervisor

{“error”:“Could not resolve type id ‘kafka’ into a subtype of [simple type, class io.druid.indexing.overlord.supervisor.SupervisorSpec]\n at [Source: HttpInputOverHTTP@274de357[c=1175,q=1,[0]=EOF,s=STREAM]; line: 1, column: 4]”}

``

The error log

druid_1 | 2018-05-03T12:20:56,783 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/local/druid/extensions/druid-kafka-indexing-service/lz4-1.3.0.jar]

druid_1 | 2018-05-03T12:20:56,784 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/local/druid/extensions/druid-kafka-indexing-service/slf4j-api-1.6.4.jar]

druid_1 | 2018-05-03T12:20:56,784 INFO [main] io.druid.initialization.Initialization - added URL[file:/usr/local/druid/extensions/druid-kafka-indexing-service/snappy-java-1.1.2.6.jar]

druid_1 | 2018-05-03T12:20:57,092 INFO [main] io.druid.initialization.Initialization - Loading extension [druid-kafka-indexing-service] for class [interface io.druid.initialization.DruidModule]

druid_1 | Error: Could not find or load main class “druid-kafka-indexing-service”]

druid_1 | Error: Could not find or load main class “druid-kafka-indexing-service”]

``

ENV DRUID_VERSION 0.12.0

ENV ZOOKEEPER_VERSION 3.4.12

ENV KAFKA_VERSION 1.1.0

full changes can be found here https://github.com/nikitsenka/docker-druid-kafka/pull/1/files#diff-3254677a7917c6c01f55212f86c57fbfR5

Any advice or ideas how to fix will be appreciated.

If anyone has ready to use kafka-indexting-service configuration supervisord.conf or docker image please share.

Thank you

Ivan

Hi Ivan,

That probably means the druid-kafka-indexing-service extension isn’t being loaded properly. The “Error: Could not find or load main class druid-kafka-indexing-service” bit looks like there is some problem generating the command line – maybe something off by one arg. Try double checking all the whitespace, maybe remove whitespace in druid.extensions.loadList.

Gian,
Yes, your truth

The problem was in whitespace -Ddruid.extensions.loadList=[“mysql-metadata-storage”, “druid-kafka-indexing-service”]

Thank you