I use Kafka firehose with Realtime node to ingest the data.
Suppose I kill the Realtime node and restart it later ( probably with a different window period).
How does Druid recognise what data has been read from Kafka and what has not been?
Does it re-ingest the same data?
Can duplicate data be recognised by some key or id and removed later from the system (via indexing service probably) ?