Tranquility : Replicants and Partitioning

All,

We are trying to configure replication for our real-time ingestion via Tranquility, and everything is working extremely well.

We want to add in replication to survive the loss of a Peon/MiddleManager. In playing with the settings, when I configure replicants = 2, partitions = 1, I only see a single task. I expected to see two tasks: one on each middle manager, providing redundancy, for a single segment/partition.

However, when I run with replicants = 2, partitions = 2, I see four tasks with the following suffixes:

_0_0

_0_1

_1_0

_1_1

That looks better, but am I to presume that I need partitioning to achieve replication?

(or can you point me to some docs that describe the relationship between partitioning and replication?)

thanks in advance,

-brian

Hey Brian,

You should be able to see what you expect with replicants = 2, partitions = 1. It’s possible that what you’re seeing is due to either:

  • One of the replicants dying/failing before you had a chance to look for it,

  • Or, the fact that changing the replication factor only takes effect when new tasks are created for new time windows. So whenever your segmentGranularity rolls over. If you started at partitions = 1, replicants = 1, and then changed to replicants = 2, this is probably what’s going on. Things should increase to 2 replicants over time.