osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Flink Query Optimizer


Hi Albert,

First I guess the query optimizer you mentioned is about Flink table & sql
(for batch API there is another optimizer which is implemented by Flink).

Yes, now for table & sql, Flink use Apache Calcite's query optimizer to
translate into a Calcite plan
which is then optimized according to Calcite's optimization rules.

The following rules are applied so far:
https://github.com/apache/flink/blob/master/flink-libraries/flink-table/src/main/scala/org/apache/flink/table/plan/rules/FlinkRuleSets.scala

In view of Flink depends on the Calcite to do the optimization, I think
enhance Flink and Calcite would be the right direction.

Hope for you provide more idea and details. Flink community welcome your
idea and contribution.

Thanks.
Vino.


2018-07-13 23:39 GMT+08:00 Albert Jonathan <albert@xxxxxxxxxx>:

> Hello,
>
> I am just wondering, does Flink use Apache Calcite's query optimizer to
> generate an optimal logical plan, or does it have its own query optimizer?
> From what I observed so far, the Flink's query optimizer only groups
> operator together without changing the order of aggregation operators
> (e.g., join). Did I miss anything?
>
> I am thinking of extending Flink to apply query optimization as in the
> RDBMS by either integrating it with Calcite or implementing it as a new
> module.
> Any feedback or guidelines will be highly appreciated.
>
> Thank you,
> Albert
>