Now we are considering to migrate from real-time node ingestion to tranquility, but we still have some questions remain unclear, could you please help?
Our use case: we have a pool of app hosts to do tranquility ingest, each host runs exact the same code and handles part of the whole traffic.
#1, if I config the partitions=1 on each host, then the result data table will have 1*N partitions, correct?
#2, do I need to set -Duser.timezone=UTC on these app?
#3, what will happen if the overlord dies? If I have 2 overlord nodes, will the other live overlord take over the role?
#4, what will happen if the middle manager/peon dies? there will be data loss in this case, correct?
#5, our middle manager nodes are also used to do batch hadoop ingestion, so the “druid.indexer.runner.javaOpts” is different in that case(heap size, -DHADOOP_USER_NAME=xxx, log, etc.), should we isolate the two kinds of task to avoid running on the same middle manager? How to do that?
#6, what’s the optimized size of events that sent each time? druidService.apply(listOfEvents)
thanks a lot!