osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [DISCUSS] Enhancing the functionality and productivity of Table API


Yes, that is my understanding as well.

Manual time management would be another difference.
Something still to be discussed would be whether (or to what extent) it
would be possible to define the physical execution plan with hints or
methods like partitionByHash and sortPartition.

Best, Fabian

Am Di., 13. Nov. 2018, 13:57 hat Piotr Nowojski <piotr@xxxxxxxxxxxxxxxxx>
geschrieben:

> Hi,
>
> > This thread is meant to enhancing the functionalities of TableAPI. I
> don't
> > think that anyone is suggesting either reducing the effort in SQL or
> > DataStream. So let's focus on how we can enhance TableAPI.
>
> I wasn’t thinking about that. As I said before, I was rising a question,
> what Table API should look like in the future if we want to diverge it more
> and more from SQL. It looks to me, that the more or less consensus is that
> it should be expanded and evolve parallel to the DataStream API, but in
> order to better suite different needs with following differences:
> - declarative
> - predefined schema/types
> - no custom state operations (?)
> - optimisations allowed by the above points
>
> Piotrek
>
> > On 7 Nov 2018, at 16:01, Xiaowei Jiang <xiaoweij@xxxxxxxxx> wrote:
> >
> > Hi Piotr:
> >
> > I want to clarify one thing first: I think that we will keep the
> > interoperability between TableAPI and DataStream in any case. So user can
> > switch between the two whenever needed. Given that, it would still be
> very
> > helpful that users can use one API to achieve most of what they do.
> > Currently, TableAPI/SQL is good enough for most data analytics kind of
> > scenarios, but there are some limitations that when removed will help we
> go
> > even further in this direction. An initial list of these is provided in
> > another thread. These are naturally extensions to TableAPI which we need
> to
> > do just for the sake of making TableAPI more usable.
> >
> > TableAPI and SQL share the same underlying implementation, so enhancement
> > in one will end up helping the other. I don't see them as competitive.
> > TableAPI is easier to extend because that we have a bit more freedom in
> > adding new functionalities. In reality, TableAPI can be mixed with SQL as
> > well.
> >
> > On the implementation side, I agree that Table API/SQL and DataStream
> > should try to share as much as possible. But that question is orthogonal
> to
> > the API discussion.
> >
> > This thread is meant to enhancing the functionalities of TableAPI. I
> don't
> > think that anyone is suggesting either reducing the effort in SQL or
> > DataStream. So let's focus on how we can enhance TableAPI.
> >
> > Regards,
> > Xiaowei
>
>