Can I split one record to many rows?

Hi, I have a json of the form like below and ingest it by native batch ingestion.
{“id”:“abc”,“data”:[{“value”:“1”,“timestamp”:“2020-01-01 00:00:00”},{“value”:“2”,“timestamp”:“2020-01-01 00:00:01”}]}
(Length of list in “data” may be arbitrary long.)
So what I want to do is to get a table from this with 3 columns [“id”, “value”, “timestamp”] and rows [“abc”,“1”,“2020-01-01 00:00:00”] and [“abc”,“2”,“2020-01-01 00:00:01”].
I tried this by giving some specifications to flattenSpec for example $.data[0].value or $.data[*].value.
But it looks like that this is not for making new rows.
Is there any way to do this?


Welcome to the Druid Forum. As of today, I do not believe Native Ingestion can do this. You may be able to reproduce this result during query time but those would be complex queries and cause maintenance issues. Let me cross-verify this and get back to you.

I would suggest doing this in the data pipeline upstream from Druid.

Thank you for the answer. I’ll follow your suggestion.