Do anyone have experience to run some of druid nodes on YARN? I think running on YARN can expand druid’s scalability.
I think ‘overlord’ or ‘peon’ are suitable to run on YARN, because they do not use a lot of disk, and seems have a reasonable footprint.
Historical nodes seems not suitable to be run on YARN, because from ‘top’ command, I can see it use too much “virtual memory space” than “physical memory space”. YARN have a default or setted ratio of virtual memory a task can have multiplied to physical memory a task can have.
My questions are:
do you suggest running Druid Nodes on YARN?
do you have suggestion if I want run Historical on YARN?