Hi Flink Community!
The release of Apache Flink 1.5 has happened (yay!) - so it
is a good time to start talking about what to do for release
== Suggested release timeline ==
I would propose to release around end of July (that
is 8-9 weeks from now).
The rational behind that: There was a lot of effort in
release testing automation (end-to-end tests, scripted stress
tests) as part of release 1.5. You may have noticed the big
set of new modules under "flink-end-to-end-tests" in the Flink
repository. It delayed the 1.5 release a bit, and needs to
continue as part of the coming release cycle, but should help
make releasing more lightweight from now on.
(Side note: There are also some nightly stress tests that
we created and run at data Artisans, and where we are looking
whether and in which way it would make sense to contribute
them to Flink.)
== Features and focus areas ==
We had a lot of big and heavy features in Flink 1.5, with
FLIP-6, the new network stack, recovery, SQL joins and client,
... Following something like a "tick-tock-model", I would
suggest to focus the next release more on integrations,
tooling, and reducing user friction.
Of course, this does not mean that no other pull request
gets reviewed, an no other topic will be examined - it is
simply meant as a help to understand where to expect more
activity during the next release cycle. Note that these are
really the coarse focus areas - don't read this as a
This list is my first suggestion, based on discussions with
committers, users, and mailing list questions.
- Support Java 9 and Scala 2.12
- Smoothen the integration in Container environment, like
"Flink as a Library", and easier integration with Kubernetes
services and other proxies.
- Polish the remaing parts of the FLIP-6 rewrite
- Improve state backends with asynchronous timer
snapshots, efficient timer deletes, state TTL, and broadcast
state support in RocksDB.
- Extends Streaming Sinks:
- Bucketing Sink should support S3 properly
(compensate for eventual consistency), work with Flink's
shaded S3 file systems, and efficiently support formats that
compress/index arcoss individual rows (Parquet, ORC, ...)
- Support ElasticSearch's new REST API
- Smoothen State Evolution to support type conversion on
- Enhance Stream SQL and CEP
- Add support for "update by key" Table Sources
- Add more table sources and sinks (Kafka, Kinesis,
Files, K/V stores)
- Expand SQL client
- Integrate CEP and SQL, through MATCH_RECOGNIZE
- Improve CEP Performance of SharedBuffer on RocksDB