Protobuf from Kafka results in IllegalArgumentException

Hello Everyone,

I’m trying to set up protobuf indexing from kafka but I keep getting IllegalArgumentException.

The flow.proto is from github.com/cloudflare/goflow but gone through some simplification, I derived the flow.json from that as the supervisor configuration and generated flow.desc using protoc -o flow.desc flow.proto.

What am I missing here? I’m clueless here right now as this seems like a syntax problem.

Error message from overlord logs attached along with .proto and supervisor config’s jinja file.

Thanks,

Sandor Kazi

overlord.ERROR.log (7.38 KB)

flow.proto (1.46 KB)

flow.json.jinja2 (1.77 KB)

This doesn’t look like a syntax issue to me. Have you added the druid-protobuf-extensions to the extension load list in common.runtime.properties on all of the nodes?

If you have added it and restarted all of the nodes could you let me know what version of Druid you are running?

Thank you,

Ben

Hi

Thanks for the reply.

Yes, I have done that.

druid 0.13-incubating is the version I’m using.

Potential culprit in my current understanding:

I use github.com/cloudflare/goflow to send packets to Kafka, so maybe I have to specify the protobuf version there somehow and downgrade to 3.1.0

Sandor