I tried to ingest some protobuf data with Kafka Indexing Service following the example given in Druid 0.12.3 documentation (http://druid.io/docs/0.12.3/development/extensions-core/protobuf.html) but failed to ingest any data while no outstanding exceptions/errors appeared in the indexing log.
I was using Druid 0.12.3 version, java version 1.8.0_181 and config files for quickstart (/examples/conf/druid/_common/) with protobuf extensions included and logging changed to DEBUG level; the supervisor spec is the same as the example provided except that dataSource name changed to “metrics_pb”, in addition, I was using a modified publisher python script to generate binary file containing protobuf messages and send them to Kafka producer. I was able to see the messages with Kafka console consumer but I found there was no dataSource created.
I had the supervisor spec, python script for Kafka producer and indexing log in attachment, and any help would be appreciated, thank you!
kafka-metrics-pb.json (1.44 KB)
pb_publisher.py (444 Bytes)