Druid parquet format ingestion for flatten json


I am trying to ingest data which is in parquet format

Sample data is like {name:“pawan”, age:27}.

I want to flatten this data during ingestion something like http://druid.io/docs/latest/ingestion/flatten-json.html.

Please help.

attaching the index json for reference

load_i_parquet.json (1.47 KB)

Have you figured it out? Also trying to flatten parquet

Couldnt find anything about it, any help from someone? It doesn’t seem to work but i wonder if that’s on the avro conversion side? where would one look for adding this kind of support? (or if you guys know about an open ticket)

Hi, apologies for the delay!

Unfortunately, the current version of ‘druid-parquet-extensions’ does not support ‘flattenSpec’ (http://druid.io/docs/latest/ingestion/flatten-json) like json and avro. That said, I currently have a PR open (https://github.com/apache/incubator-druid/pull/6360) that will add this behavior for parquet files, so hopefully this will be able to work for your use case.