[druid-user] Kafka offsets preserved after supervisor termination?

If I terminate a supervisor, wait for a while, and then resubmit the exact same supervisor spec, will the new supervisor pick up the offsets of terminated supervisor?

The documentation suggests that the state is preserved in the metadata database but I wanted to check.

And an additional question: does the behavior of the new supervisor depend of the value of useEarliestOffset?

Thank you.

Hi Chris,

That’s correct, terminating the supervisor does not remove the stored offsets from the metadata DB. You’d need to perform a hard reset to clear that out.

When you create a new supervisor with the same name as one you deleted, it’s essentially the same as submitting a new spec for an existing supervisor.

useEarliestOffset only comes into play when you don’t have a stored offset; if you do, it will use the saved offsets in the metadata DB.

Thanks,
Max

Hi Max,

Just to clarify, is there a retention period for the offsets? Or are they just stored in the metadata database indefinitely until a hard reset is performed?

Hi Chris,

There’s no retention period; only a hard reset wipes out the stored offset as far as I know. If you create another supervisor with the same name, it’ll find the existing entry in druid_dataSource in the metadata DB.

Thanks,
Max

Sounds good. Thanks so much Max.