osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: SQL Query Set Analyzer


Hi Zheng, I think the Babel parser in the latest Calcite 1.17 might be
helpful here. The Babel parser (JIRA here
<https://issues.apache.org/jira/browse/CALCITE-2280>) can allow Calcite to
accept all SQL dialects (e.g. Hive, oracle, and etc.). Before 1.17, Calcite
can only parse its own SQL dialect, but I know people extend the core
Calcite parser to parse other SQL dialects for their own use. Once the
query parsed, you can either use the parser tree or the relational algebra
to analyze the queries and identify common patterns among queries. Hope it
helps.

Shuyi

On Wed, Jul 25, 2018 at 9:07 PM Zheng Shao <zshao9@xxxxxxxxx> wrote:

> Hi,
>
> We are thinking about starting a project to analyze huge number of SQL
> queries (think millions) to identify common patterns:
> * Common sub queries
> * Common filtering conditions (columns) for a table
> * Common join keys for table pairs
>
> Are there any existing projects on that direction using Calcite?  Would
> love to leverage instead of building from scratch.
>
> Zheng
>


-- 
"So you have to trust that the dots will somehow connect in your future."