Ingestion complete, but queries still empty

Hey guys

So I was trying to ingest a 100 MB large JSON-File with the Druid Indexing Service. After about 2 minutes the Overlord Node reports success (and in the webconsole the task is moved from Running to Complete). However, all queries still return empty results. It takes an additional 30-40 seconds after the ingestion completes until the correct results are obtained.

Is there a way to fix this problem or is this unavoidable?



I assume you are doing batch ingestion. After the segments are created, coordinator has to assign them to historical nodes which subsequently load and start serving those segment. So, there is some latency involved there. 30-40 secs look normal to me.

– Himanshu

The indexer polls the segments table on a regular interval (default 60s I think) and THEN starts the scheduling process. So it can take a few minutes for a new historical segment to register.