osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Cherry picking FLINK-9753 and FLINK-9753 on release-1.6


Hi Kostas,

Nice to see these features merged to the release-1.6 branch. Supporting
Parque format in StreamingFileSink
really makes me eager to apply it to some use cases in our company, but I'm
still wondering if the current
StreamingFileSink works well on S3 or I should wait util [FLINK-9752] done.

Anyway, I would still like to see this happen, because it let me have more
alternative to upload data with parquet
format. Thanks for your great efforts on this.

Best Regards,
Tony Wei

https://issues.apache.org/jira/browse/FLINK-9752

2018-07-21 4:24 GMT+08:00 Kostas Kloudas <k.kloudas@xxxxxxxxxxxxxxxxx>:

> Hi all,
>
> We just merged on the master [FLINK-9753] (commit
> 66b1f854a0250bdd048808d40f93aa2990476841)
> and [FLINK-9903] (commit b56c75ca375049b1d2c80d2d0945ae1ae04eb39e).
>
> These two commits introduce:
> 1) a big refactoring to the new StreamingFileSink which allows it to
> support bulk formats in general (FLINK-9753) and more expressive rolling
> and bucketing strategies, and
> 2) support for Parque, based on the previous refactoring.
>
> Both these features were pretty popular in the mailing list.
>
> Given that these 2 features are self contained, as they mainly affect a
> connector,  we would like to cherry pick these commits to the release-1.6
> branch. But for this, we would like to have the OK from the community.
>
> Please let me know what you think,
> Kostas
>
> https://issues.apache.org/jira/browse/FLINK-9903 <
> https://issues.apache.org/jira/browse/FLINK-9903>
> https://issues.apache.org/jira/browse/FLINK-9753 <
> https://issues.apache.org/jira/browse/FLINK-9753>