[Query] smile/binary json ingestion with kafka-ingestion-service

Hi,
A druid noob here.

Is it possible to provide a deserilizer while ingesting data through Kafka ingestion service?

This is the deserializer that is used during Kafka topic consumption by the ingestion service.
This ingests textual json records from Kafka topic.

What I want: ability to consume Smile encoded json from Kafka.

Is it possible to provide deserializer through some spec [maybe, parserSpec]?
Or, there is some config that I am missing?



Thanks in advance.

Hey Kogu,

Druid’s Kafka ingestion is designed to read raw byte array data from Kafka and then parse it using one of Druid’s parsers. This is because it has to be parsed into a particular kind of Druid Java object (class org.apache.druid.data.input.InputRow). You could build a Druid extension that does this with smile input – check this repo for an example: https://github.com/implydata/druid-example-extension.

Gian