Paritions with Tranquility Server


There is a property in the Tranquility Server configs called tasks.partitions. I’m wondering how exactly this is working. I have 3 tranquility servers consuming from a single kafka topic with 64 partition and sending these events to one datasource/task with two partitions. Is druid assuming that both partitions have different sets of data?

There is a partitionSpec for hadoop indexing tasks i think where you can specify partition strategy. Is there something like this for Tranquility Server?

All help is appreciated.


tasks.partitions refers to how many Druid segments you’ll get per segmentGranularity period. So if you have segmentGranularity “day” and tasks.partitions = 2 then you get two per day. Usually people set this to a number that gets them good performance and reasonable sized segments.

The default partitioning strategy in Tranquility 0.8.0+ is to partition based on a hash of all the dimensions of your events, which is usually a good strategy. If you want another one you can override it in code by providing a custom Partitioner class (there’s no way to override the default strategy with purely configuration).