osdir.com

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: HA setting for per-job YARN session


Hi,

The per-job YARN mode also supports HA. Even if the YARN ResourceManager only
brings up one JobManager at a time, you require ZooKeeper for storing
metadata, such as which jobs are supposed to be running. In fact, the
distributed components will still take part in a leader election with only one
contender. The keys that are required to be configured for HA are:
   
    high-availability: zookeeper
    high-availability.zookeeper.quorum: <your zk quorum>
    high-availability.zookeeper.storageDir: <path on DFS>

Best,
Gary

On Mon, Aug 6, 2018 at 6:25 AM, 杨光 <laolang113@xxxxxxxxx> wrote:
Hi,
i am using  the " per-job YARN session " mode deploy flink job on yarn.
The document https://ci.apache.org/projects/flink/flink-docs-release-1.5/ops/jobmanager_high_availability.html#yarn-cluster-high-availability
 says that "we don’t run multiple JobManager (ApplicationMaster)
instances" but still give an example about how to config an Highly
Available YARN Session cluster.

My question is:  does the " per-job YARN session "  mode should  also
use HA setting ? Because in my test case without using zookeeper HA ,
if the YarnApplicationMasterRunner/Jobmanager is failed or killed
,yarn can only restart the YarnApplicationMasterRunner/Jobmanager
program but no job graph is submitted to the Jobmanager again.

THANKS