[druid-user] Monitor indexing time and segment size

Hi,

You can see the segments availlable in coordinator and able to query them because realtime node serves the segment (which is in the process of being built) it is building. However, you don’t see anything in db or in historicals because segment “handoff” from realtime has not happened.

If your usecase is really that of periodically ingesting data into druid then you should rather check batch ingestion (example http://druid.io/docs/latest/tutorials/tutorial-loading-batch-data.html , reference http://druid.io/docs/latest/ingestion/batch-ingestion.html ) .

– Himanshu

Ok, thank you, I thought that with the messageTime rejection policy, the segment would be handoff when an event with a timestamp superior (at least a day later with segmentGranularity = day) arrives. Could you tell me when are the segments handoff with this policy ?
This usecase is just for testing.

Regarding the indexing time, do you have an idea on how to have it ?

Hi,

First handoff will happen after one segment granularity + windowPeriod (even if an event more than a day and window period in future has been seen).

I would guess, you haven’t waited for a whole day and consequently did not see the handoff. (there might be other issues with handoff but this is my first guess).

– Himanshu