osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: S3 for state backend in Flink 1.4.0


Hi Aljoscha,

Thinking a little bit more about this, although IBM Object storage is
compatible with Amazon's S3, it's not an eventually consistent file system,
but rather immediately consistent. 

So we won't need the support for eventually consistent FS for our use case
to work, but we would only need that the BucketingSink uses the Flink
FileSystem abstraction instead of directly using the Hadoop FileSystem
abstraction.

Is this something that could be released earlier ?  Or do you have any idea
how we could achieve it ?

Regards,
Edward



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/