Creating a new Druid Data Source from union query result

Hi,

I have a scenario when I need to union multiple data sources and create a new one with the result of the query. The size of the result is ~10 gigabytes, so I would like not to transfer this data to client, but write it instantly to a new table.
Is there any way to do that?

Hi, you can push your query result into kafka or save as file, druid can ingest data from kafka, hdfs. Ingestion method: https://druid.apache.org/docs/latest/ingestion/index.html#ingestion-methods

在2020年10月27日星期二 UTC+8 上午4:30:59king...@gmail.com 写道:

This is new in Druid 0.20 - is this something like what you need?
https://druid.apache.org/docs/0.20.0/ingestion/native-batch.html#combining-input-source

(I think there was also “multi” for hadoop-based ingestion? Check the “TTL” bit of this blog by Nielson: https://medium.com/nmc-techblog/data-retention-and-deletion-in-apache-druid-74ffd12398a8)

@Peter, thank you, but unfortunately, it’s not what I was looking for.