I have 4 nodes installing imply1.2.0, two for master nodes and two for data nodes. use tranquility-kafka stream push, deep storage is hdfs, mysql for metadata storage. it seems that overlords cannot work at the same time, when one works, the other is sleeping, causing segments in the overlord waiting for handoff. I use local type of druid.indexer.runner.type. thanks for help!
Overlord is not designed for load banlance, but master/slave architecture. so at cluster atmosphere, recommend to use remote index type. I still have a question, all data distribut to middle managers through overlord or not? if so, overlord load will be too heavy. if not, overlord just distributs tasks to zk, and tranquilities distribut task data to middle managers, it's seems mutch better. thanks for your answer.
Yeah, the overlord just manages the tasks and assigns them to the middlemanagers.
In case of tranquility, the data is sent directly to the tasks via HTTP endpoints.
thanks, Nishant Bangarwa