osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

how to override s3 key config in flink job


Hi,

Is there any way to provide s3.access-key and s3.secret-key in flink application, instead of setting
them in flink-conf.yaml?

In our use case, my team provide a flink standalone cluster to users. However, we don't want to let
each user use the same s3 bucket as filesystem to store checkpoints. So, we want to know if is it
feasible to let users provide their checkpoint path and corresponding aws key to access their own 
s3 bucket?

If not, could you show me why it doesn't work currently? And, is it possible to become a new
feature?

Thanks in advance for your help.

Best,
Tony Wei