osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Flink CLI properties with HA


Hi Sampath,

Flink CLI need to retrieve the JobManager leader address, so it need  to
access the HA specific configuration. Because if based on Zookeeper to
implement the HA, the leader address information will fetch from Zookeeper.

The main use of config item *high-availability.storageDir* is storage (Job
graph, checkpoint and so on). Actually, the real data is stored under this
path which used to recover purpose, zookeeper just store a state handle.

---
Thanks.
vino.


2018-07-16 15:28 GMT+08:00 Sampath Bhat <sam414255path@xxxxxxxxx>:

>
> ---------- Forwarded message ----------
> From: Sampath Bhat <sam414255path@xxxxxxxxx>
> Date: Fri, Jul 13, 2018 at 3:18 PM
> Subject: Flink CLI properties with HA
> To: user <user@xxxxxxxxxxxxxxxx>
>
>
> Hello
>
> When HA is enabled in the flink cluster and if I've to submit job via
> flink CLI then in the flink-conf.yaml of flink CLI should contain this
> properties -
> high-availability: zookeeper
> high-availability.cluster-id: flink
> high-availability.zookeeper.path.root: flink
> high-availability.storageDir: <some path>
> high-availability.zookeeper.quorum: <zookeeper IP:port>
>
> What is the need of high-availability.storageDir for flink CLI. Does this
> mean that even flink client should be able to access the mentioned path or
> is it some check being done on the property name?
>
> Without these properties flink cli will not be able to submit job to flink
> cluster when HA is enabled.
>
>