Reindex from druid - task not returning data

We have a supervisor reading stream data from kafka with one minute granularity.
Every 15 minutes, we are running a “Reindex from druid” task that aggregates data by fifteenminute granularity for the previous 15 minutes and write it into a new datasource.
The task is running with success, but with no data written in the concerned datasource.
while data is well present in the 1 minute granularity datasource.

if we wait for 1 hour and rerun the same task, the fifteenminute datasource is well populated with data.

Is there any configuration we can add to reindex from druid task to reindex the newly inserted data?

The kafka ingestion task has a task duration of one hour, “Reindex from druid” will not return data from the corresponding interval as long as the kafka task did not stop reading.

Issue solved by putting kafka taskduration to 15 minutes instead of one hour.

1 Like