I’m new to Druid and I’d like to know how I could see how much time Druid take to index new data.
I use Kafka to ingest data, and I followed the tutorial. For now, I’m just sending files on one topic, locally.
Also, I use the messageTime rejection policy, and I read this : “When using the “messageTime” rejectionPolicy, handoff of a particular segment only occurs after the realtime node sees an event timestamped outside the windowPeriod of that segment.”.
However it does not appear to be the case for me. My segment granularity is set to day, with a 10 min window period, and I send data with Kafka in several files, each file having data from different days (and I send the files in the right order). In the Druid Coordinator Consol, I see the segments expected (under Cluster Segments), but their size is 0 and the historical server current size is 0 too. I can make queries and get right results.
I don’t see any warning except this one : “WARN [DatabaseSegmentManager-Exec–0] io.druid.metadata.SQLMetadataSegmentManager - No segments found in the database!”
I’m using Druid 0.8, MySQL for the metadata, and local file system for deepstorage.