OSDir


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Gluster as file system for state backend


Hi,

I am evaluating some File Systems as state backend. I can see that Flink currently supports S3, MAPRFS and HDFS as file systems. 

However, I was hoping I can use Gluster as my state backend, since its already a part of existing eco system. Since I have stateful operators in my job and I am expecting each operator to accumulate large amount of state(in RocksDB) in 10seconds(checkpointing interval), I am looking for a write optimized  file system. 
 

Does anyone has any prior experience with Gluster as file state backend? Are there any caveats with using a replicated/striped replicated volume for Flink state data? 

Thanks,

Chirag