Submit indexing job to another yarn cluster

Hi!

I’m currently working on separating indexing job from fixed yarn cluster. (Setting up with druid)

Because i have multiple tables, and i don’t want several indexing job run in same yarn cluster which is occupy huge disk, network bandwidth.

So i want some of them will go to another idle yarn cluster(separated), and remaining jobs are goes to original yarn cluster.

I add yarn.resourcemanager.address property into indexing job spec.

Hi,

Sorry I don’t have the exact answer, but I’d suggest checking out https://hadoop.apache.org/docs/r2.7.2/hadoop-yarn/hadoop-yarn-common/yarn-default.xml and setting as many of the properties involving yarn.resourcemanager.address and yarn.resourcemanager.hostname as possible.

Hi Gian

Yeah… i already added as many options in there as i can…

Including yarn.resourcemanager.hostname as well.

I met same error again.

2018년 2월 8일 목요일 오전 10시 54분 51초 UTC+9, Gian Merlino 님의 말: