OSDir


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Do I still need hadoop-aws libs when using Flink 1.5 and Presto?


I am trying to figure out how to use S3 as state storage.
The recommended way is https://ci.apache.org/projects/flink/flink-docs-release-1.5/ops/deployment/aws.html#shaded-hadooppresto-s3-file-systems-recommended

Seems like I only have to do two things:
1. Put flink-s3-fs-presto to the lib
2. Configure
s3.access-key: your-access-key
s3.secret-key: your-secret-key

But I see this exception: ClassNotFoundException: NativeS3FileSystem/S3AFileSystem Not Found

https://ci.apache.org/projects/flink/flink-docs-release-1.5/ops/deployment/aws.html#provide-s3-filesystem-dependency

Add it is suggested to add more libs.
So I am confused here, is there a step 3 needed? Isn't the presto jar is all self contained?

Thanks