osdir.com

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[jira] [Created] (CALCITE-2703) Reduce code generation and class loading overhead when executing queries in the EnumerableConvention


Stamatis Zampetakis created CALCITE-2703:
--------------------------------------------

             Summary: Reduce code generation and class loading overhead when executing queries in the EnumerableConvention
                 Key: CALCITE-2703
                 URL: https://issues.apache.org/jira/browse/CALCITE-2703
             Project: Calcite
          Issue Type: Improvement
          Components: core
    Affects Versions: 1.17.0
            Reporter: Stamatis Zampetakis
            Assignee: Julian Hyde
             Fix For: 1.18.0


The queries using Calcite's EnumerableConvention always end-up generating new java classes at runtime (using Janino) that are then instantiated using reflection. This combination of class generation and class loading introduces a big overhead in query response time.

A quick profiling on our Company's internal test suite consisting in 4000 tests with roughly 430000 SQL queries passing through Calcite we observed that a big amount of time is spend on code generation and class loading making the EnumerableInterpretable#toBindable method a performance bottleneck. 

Among the 430000 SQL queries there are many duplicates which are going to lead to the generation of exactly the same code Java. Introducing, a small cache at the level of EnumerableInterpretable class could avoid generating and loading the same code over and over again.

A simple implementation based on Guava improved the overall execution time of the afforementioned test suite by more than 50%.

 

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)