I’ve been trying out Tranquility to ingest events from Kafka into Druid and had a few questions regarding the same:
(i) For a given topic, how are Kafka partitions mapped to Tranquility partitions (set using “task.partitions”)?
(ii) What happens if Tranquility is configured to have more partitions than the Kafka topic?
For example, if the Kafka topic has X partitions and Tranquility is configured to ingest this topic with X + Y partitions, would the data from the Kafka topic be
(a) distributed across the X + Y tasks that would be created by Tranquility OR
(b) distributed across X tasks, leaving the other Y tasks idle?
(iii) Do all the tasks created by Tranquility share the same Kafka Consumer Group Id?
Please let me know if I’ve missed some documentation that explains these details.
Thanks in advance,