Creating a new Druid Data Source from union query result


I have a scenario when I need to union multiple data sources and create a new one with the result of the query. The size of the result is ~10 gigabytes, so I would like not to transfer this data to client, but write it instantly to a new table.
Is there any way to do that?

Hi, you can push your query result into kafka or save as file, druid can ingest data from kafka, hdfs. Ingestion method:

在2020年10月27日星期二 UTC+8 上午 写道:

This is new in Druid 0.20 - is this something like what you need?

(I think there was also “multi” for hadoop-based ingestion? Check the “TTL” bit of this blog by Nielson:

@Peter, thank you, but unfortunately, it’s not what I was looking for.