How to change index spec with tranquility and overlord?

I’m testing with Tranquility and Overlord configuration without any special node for middle manager.

I can index a set of events, everything is working fine. If I need to change the index spec like adding more metrics, it is not taking effect.

If I restart overlord then tranquility is failing to ingest any new data to the old tenant - Any reason? I suspect it is trying to reach the old process but not finding it.

After restarting overlord I can index data to new data source with new index spec, it is working fine.

Is it possible to change the index spec for an existing data source? If I use middle manager as a separate node and peons can it solve this problem?

It’s not currently possible within Druid to change the indexing spec of an existing indexing task, so tranquility does the next best thing: it spawns a new task at the start of the next segmentGranularity period with the new config. So, if your segmentGranularity is “HOUR” then tranquility has an opportunity to update the indexing config at the start of each new hour. This happens automatically and there is no need to mess with the overlord or existing tasks (although you do need to wait for the appropriate amount of time).