osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [DISCUSS] Support Interactive Programming in Flink Table API


Hi Jiangjie&Shaoxuan,

Thanks for initiating this great proposal!

Interactive Programming is very useful and user friendly in case of your
examples.
Moreover, especially when a business has to be executed in several stages
with dependencies,such as the pipeline of Flink ML, in order to utilize the
intermediate calculation results we have to submit a job by env.execute().

About the `cache()`  , I think is better to named `persist()`, And The
Flink framework determines whether we internally cache in memory or persist
to the storage system,Maybe save the data into state backend
(MemoryStateBackend or RocksDBStateBackend etc.)

BTW, from the points of my view in the future, support for streaming and
batch mode switching in the same job will also benefit in "Interactive
Programming",  I am looking forward to your JIRAs and FLIP!

Best,
Jincheng


Becket Qin <becket.qin@xxxxxxxxx> 于2018年11月20日周二 下午9:56写道:

> Hi all,
>
> As a few recent email threads have pointed out, it is a promising
> opportunity to enhance Flink Table API in various aspects, including
> functionality and ease of use among others. One of the scenarios where we
> feel Flink could improve is interactive programming. To explain the issues
> and facilitate the discussion on the solution, we put together the
> following document with our proposal.
>
>
> https://docs.google.com/document/d/1d4T2zTyfe7hdncEUAxrlNOYr4e5IMNEZLyqSuuswkA0/edit?usp=sharing
>
> Feedback and comments are very welcome!
>
> Thanks,
>
> Jiangjie (Becket) Qin
>