I’m sorry. My bad. I think I read it off of the Hadoop-based indexer page.
But maxRowsPerSegment=5000000
I just want to know, why is the Kafka indexer creating empty shards? Or why is it creating a shard at all?
Stream-based ingestion (like Kafka indexing) creates segments that initially show up as 0B while they are still mutable, but then have a concrete size once they become immutable. You should find that this segment gets a real size after a while.
It did show up now as a concrete number. But, it looks like that is an exact replica of the segment already present in that interval. Clicking on an interval displays two segments that call themselves partitionNum 0 and 1. My question is, I didn’t ask for a replica to be created.