Replace existing segments from Kafka indexing service with hadoop job


I have a datasource generated by a Kafka indexing service. I’d like to replace some old segments with a Hadoop indexing job, but instead of replacing the old segments, it just created new segments, and the rows from the two segments are merged in the results.

Should I manually remove/disable the segments generated by the Kafka indexing service before executing the Hadoop indexing task, or is there a way to specify in metadataUpdateSpec to automatically replace them?

Best regards,


Sorry, never mind, it works as it should, I messed up with the
counts, that’s why I thought that we ended up with twice as much
data as expected