OSDir


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [DISCUSS] Flink 1.6 features


Before removing the legacy code, I would still wait a bit and see what the
user feedback is. The legacy mode is a good safety net against severe
deployment regressions. Thus, it should be a very conscious decision to
remove the code.

As far as I know, there is currently nobody actively working on FLINK-7883.
Fixing it properly requires a bit of work (e.g. redesigning the source
interface) and will need a committer to act as a shepherd. But it is
definitely a feature which is quite important to have.

Cheers,
Till

On Mon, Jun 4, 2018 at 11:50 AM Antoine Philippot <
antoine.philippot@xxxxxxxx> wrote:

> Hi Stephen,
>
> Is it planned to consider this ticket
> https://issues.apache.org/jira/browse/FLINK-7883 about an atomic
> cancel-with-savepoint ?
>
> It is my main concern about Flink and I have to maintain a fork myself as
> we can't afford dupplicate events due to reprocess of messages between a
> savepoint and real job stop each time we deploy a new version of our job.
>
> Antoine
>
> Le lun. 4 juin 2018 à 11:21, Stephan Ewen <sewen@xxxxxxxxxx> a écrit :
>
>> Hi Flink Community!
>>
>> The release of Apache Flink 1.5 has happened (yay!) - so it is a good
>> time to start talking about what to do for release 1.6.
>>
>> *== Suggested release timeline ==*
>>
>> I would propose to release around *end of July* (that is 8-9 weeks from
>> now).
>>
>> The rational behind that: There was a lot of effort in release testing
>> automation (end-to-end tests, scripted stress tests) as part of release
>> 1.5. You may have noticed the big set of new modules under
>> "flink-end-to-end-tests" in the Flink repository. It delayed the 1.5
>> release a bit, and needs to continue as part of the coming release cycle,
>> but should help make releasing more lightweight from now on.
>>
>> (Side note: There are also some nightly stress tests that we created and
>> run at data Artisans, and where we are looking whether and in which way it
>> would make sense to contribute them to Flink.)
>>
>> *== Features and focus areas ==*
>>
>> We had a lot of big and heavy features in Flink 1.5, with FLIP-6, the new
>> network stack, recovery, SQL joins and client, ... Following something like
>> a "tick-tock-model", I would suggest to focus the next release more on
>> integrations, tooling, and reducing user friction.
>>
>> Of course, this does not mean that no other pull request gets reviewed,
>> an no other topic will be examined - it is simply meant as a help to
>> understand where to expect more activity during the next release cycle.
>> Note that these are really the coarse focus areas - don't read this as a
>> comprehensive list.
>>
>> This list is my first suggestion, based on discussions with committers,
>> users, and mailing list questions.
>>
>>   - Support Java 9 and Scala 2.12
>>
>>   - Smoothen the integration in Container environment, like "Flink as a
>> Library", and easier integration with Kubernetes services and other proxies.
>>
>>   - Polish the remaing parts of the FLIP-6 rewrite
>>
>>   - Improve state backends with asynchronous timer snapshots, efficient
>> timer deletes, state TTL, and broadcast state support in RocksDB.
>>
>>   - Extends Streaming Sinks:
>>      - Bucketing Sink should support S3 properly (compensate for eventual
>> consistency), work with Flink's shaded S3 file systems, and efficiently
>> support formats that compress/index arcoss individual rows (Parquet, ORC,
>> ...)
>>      - Support ElasticSearch's new REST API
>>
>>   - Smoothen State Evolution to support type conversion on snapshot
>> restore
>>
>>   - Enhance Stream SQL and CEP
>>      - Add support for "update by key" Table Sources
>>      - Add more table sources and sinks (Kafka, Kinesis, Files, K/V
>> stores)
>>      - Expand SQL client
>>      - Integrate CEP and SQL, through MATCH_RECOGNIZE clause
>>      - Improve CEP Performance of SharedBuffer on RocksDB
>>
>>