OSDir


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [v1.15] A NPE that looks inevitable !


Back to the original problem (NPE at AggregateNode.getAccumulator, line
150), I've hit this bug again and I'm still struggling to reproduce it
through a test case or evade it altogether.

The bug is reproducible through my custom adaptor using this query:

SELECT cn."COUNTRY_NAME", count(prd."PROD_NAME") Product_Count, count(
*DISTINCT* (prd."PROD_NAME")) Product_Distinct FROM SALES.SALES1 s LEFT
OUTER JOIN SALES.PRODUCTS prd ON s."PROD_ID" = prd."PROD_ID" LEFT OUTER
JOIN SALES.CUSTOMERS c ON s."CUSTOMER_ID" = c."CUST_ID" LEFT OUTER JOIN
SALES.COUNTRIES cn ON c."COUNTRY_ID" = cn."COUNTRY_ID" GROUP BY
cn."COUNTRY_NAME"

I noticed that if I remove the DISTINCT keyword (highlighted in bold), the
bug isn't activated. I developed my adaptor by following the Druid adapter
way, I think I missed something but I can't decide what it is.

I've been thinking that since this piece of code should not be activated,
as it will always throw an exception. Could someone please tell me why
would an executing query hit in the first place ? Or how should I configure
my adaptor, planner, query..etc to NOT hit that code ?

Thanks,
Gelbana

On Wed, Apr 25, 2018 at 9:56 PM, Muhammad Gelbana <m.gelbana@xxxxxxxxx>
wrote:

> Thank you! That helped a lot.
>
> Thanks,
> Gelbana
>
> On Wed, Apr 25, 2018 at 12:00 PM, Stamatis Zampetakis <zabetak@xxxxxxxxx>
> wrote:
>
>> Hi Gelbana,
>>
>> I haven't looked at your snippet but if you want to debug into generated
>> code have a look at the following link:
>> https://calcite.apache.org/docs/howto.html#debugging-generat
>> ed-classes-in-intellij
>>
>> Best,
>> Stamatis
>>
>> 2018-04-21 14:07 GMT+02:00 Muhammad Gelbana <m.gelbana@xxxxxxxxx>:
>>
>> > I'm trying to create the test case for this issue but I can't run a
>> simple
>> > query as a start.
>> >
>> > I'm facing the following exception
>> >
>> > java.sql.SQLException: exception while executing query: null
>> >
>> >     at org.apache.calcite.avatica.Helper.createException(Helper.jav
>> a:56)
>> >     at org.apache.calcite.avatica.Helper.createException(Helper.jav
>> a:41)
>> >     at
>> > org.apache.calcite.avatica.AvaticaConnection.executeQueryInternal(
>> > AvaticaConnection.java:540)
>> >     at
>> > org.apache.calcite.avatica.AvaticaPreparedStatement.executeQuery(
>> > AvaticaPreparedStatement.java:133)
>> >     at
>> > org.apache.calcite.tools.FrameworksTest.aggNPE(FrameworksTest.java:184)
>> >     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> >     at
>> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
>> ssorImpl.java:
>> > 62)
>> >     at
>> > sun.reflect.DelegatingMethodAccessorImpl.invoke(
>> > DelegatingMethodAccessorImpl.java:43)
>> >     at java.lang.reflect.Method.invoke(Method.java:498)
>> >     at
>> > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(
>> > FrameworkMethod.java:50)
>> >     at
>> > org.junit.internal.runners.model.ReflectiveCallable.run(
>> > ReflectiveCallable.java:12)
>> >     at
>> > org.junit.runners.model.FrameworkMethod.invokeExplosively(
>> > FrameworkMethod.java:47)
>> >     at
>> > org.junit.internal.runners.statements.InvokeMethod.
>> > evaluate(InvokeMethod.java:17)
>> >     at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
>> >     at
>> > org.junit.runners.BlockJUnit4ClassRunner.runChild(
>> > BlockJUnit4ClassRunner.java:78)
>> >     at
>> > org.junit.runners.BlockJUnit4ClassRunner.runChild(
>> > BlockJUnit4ClassRunner.java:57)
>> >     at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
>> >     at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
>> >     at org.junit.runners.ParentRunner.runChildren(ParentRunner.
>> java:288)
>> >     at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
>> >     at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
>> >     at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
>> >     at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
>> >     at
>> > com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(
>> > JUnit4IdeaTestRunner.java:68)
>> >     at
>> > com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.
>> > startRunnerWithArgs(IdeaTestRunner.java:47)
>> >     at
>> > com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(
>> > JUnitStarter.java:242)
>> >     at
>> > com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
>> >
>> > *Caused by: java.lang.NullPointerException    at Baz.bind(Unknown
>> Source)*
>> >     at
>> > org.apache.calcite.adapter.enumerable.EnumerableBindable.
>> > bind(EnumerableBindable.java:70)
>> >     at
>> > org.apache.calcite.jdbc.CalcitePrepare$CalciteSignature.enumerable(
>> > CalcitePrepare.java:365)
>> >     at
>> > org.apache.calcite.jdbc.CalciteConnectionImpl.enumerable(
>> > CalciteConnectionImpl.java:301)
>> >     at
>> > org.apache.calcite.jdbc.CalciteMetaImpl._createIterable(
>> > CalciteMetaImpl.java:559)
>> >     at
>> > org.apache.calcite.jdbc.CalciteMetaImpl.createIterable(
>> > CalciteMetaImpl.java:550)
>> >     at
>> > org.apache.calcite.avatica.AvaticaResultSet.execute(
>> > AvaticaResultSet.java:172)
>> >     at
>> > org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResu
>> ltSet.java:67)
>> >     at
>> > org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResu
>> ltSet.java:44)
>> >     at
>> > org.apache.calcite.avatica.AvaticaConnection.executeQueryInternal(
>> > AvaticaConnection.java:536)
>> >     ... 24 more
>> >
>> > This is the test case I'm working on
>> >
>> > @Test
>> > public void aggNPE() throws SqlParseException, ValidationException,
>> > RelConversionException, SQLException {
>> >   Function<Holder, Object> hook = (o1) -> {
>> >     o1.set(true);
>> >     return null;
>> >   };
>> >   Hook.ENABLE_BINDABLE.add(hook);
>> >
>> >   SchemaPlus rootSchema = Frameworks.createRootSchema(true);
>> >   final FrameworkConfig config = Frameworks.newConfigBuilder()
>> >           .defaultSchema(CalciteAssert.addSchema(rootSchema,
>> > CalciteAssert.SchemaSpec.JDBC_FOODMART))
>> >           .ruleSets(RuleSets.ofList(Bindables.RULES))
>> >           .build();
>> >
>> >   Planner planner = Frameworks.getPlanner(config);
>> >   SqlNode parsed = planner.parse("SELECT * FROM \"sales_fact_1997\"");
>> >   SqlNode validated = planner.validate(parsed);
>> >   RelRoot rel = planner.rel(validated);
>> >
>> >   PreparedStatement statement = RelRunners.run(rel.rel);
>> >   ResultSet rs = statement.executeQuery();
>> >
>> >   rs.close();
>> >   statement.close();
>> > }
>> >
>> > I'm trying to be as close as possible to what I have in my schema so I
>> > enabled the bindable convention and enabled the bindables rules. What
>> am I
>> > doing wrong here to cause this NPE ?
>> > I can't debug into the *Baz* class to figure out what's going on as the
>> > class seems to be generated at runtime or something.
>> >
>> >
>> > Thanks,
>> > Gelbana
>> >
>> > On Thu, Mar 29, 2018 at 1:34 AM, Muhammad Gelbana <m.gelbana@xxxxxxxxx>
>> > wrote:
>> >
>> > > With pleasure. I'll do that ASAP.
>> > >
>> > > Thanks,
>> > > Gelbana
>> > >
>> > > On Thu, Mar 29, 2018 at 12:38 AM, Shuyi Chen <suez1224@xxxxxxxxx>
>> wrote:
>> > >
>> > >> It will be great if you can provide a test to reproduce the problem,
>> and
>> > >> attach it in the JIRA.
>> > >>
>> > >> On Wed, Mar 28, 2018 at 2:50 PM, Muhammad Gelbana <
>> m.gelbana@xxxxxxxxx>
>> > >> wrote:
>> > >>
>> > >> > Absolutely: https://issues.apache.org/jira/browse/CALCITE-2228
>> > >> >
>> > >> > Please tell me if I should modify any of the information I
>> supplied.
>> > >> Like
>> > >> > the affected version\component.
>> > >> >
>> > >> > Thanks,
>> > >> > Gelbana
>> > >> >
>> > >> > On Wed, Mar 28, 2018 at 11:26 PM, Julian Hyde <jhyde@xxxxxxxxxx>
>> > wrote:
>> > >> >
>> > >> > > It does indeed look as if that piece of code (AggregateNode.java
>> > line
>> > >> > 149)
>> > >> > > has never been run. Can you please log a JIRA case?
>> > >> > >
>> > >> > > > On Mar 28, 2018, at 1:42 PM, Muhammad Gelbana <
>> > m.gelbana@xxxxxxxxx>
>> > >> > > wrote:
>> > >> > > >
>> > >> > > > Looking at this line
>> > >> > > > <https://github.com/apache/calcite/blob/master/core/src/
>> > >> > > main/java/org/apache/calcite/interpreter/AggregateNode.java#
>> L150>,
>> > >> > > > and the referenced instance variable
>> > >> > > > <https://github.com/apache/calcite/blob/master/core/src/
>> > >> > > main/java/org/apache/calcite/adapter/enumerable/AggImpState.
>> > >> java#L33>,
>> > >> > > > it looks to me that *int stateSize = agg.state.size* will
>> always
>> > >> throw
>> > >> > a
>> > >> > > > NPE because the *agg.state* object reference is never
>> initialized
>> > >> > before
>> > >> > > > it's being used !
>> > >> > > >
>> > >> > > > I'm facing the following exception
>> > >> > > >
>> > >> > > > java.sql.SQLException: Error while executing SQL "SELECT
>> > >> > MAX(COUNTRY_ID),
>> > >> > > > MAX(COUNTRY_TOTAL_ID) FROM SALES.COUNTRIES": null
>> > >> > > >    at org.apache.calcite.avatica.Hel
>> per.createException(Helper.
>> > >> > java:56)
>> > >> > > >    at org.apache.calcite.avatica.Hel
>> per.createException(Helper.
>> > >> > java:41)
>> > >> > > >    at
>> > >> > > > org.apache.calcite.avatica.AvaticaStatement.executeInternal(
>> > >> > > AvaticaStatement.java:156)
>> > >> > > >    at
>> > >> > > > org.apache.calcite.avatica.AvaticaStatement.executeQuery(
>> > >> > > AvaticaStatement.java:218)
>> > >> > > >    at
>> > >> > > > com.mg.sqlinterface.data.provider.calcite.internal.
>> > >> > > Test_SQLi.testMG(Test_SQLi.java:35)
>> > >> > > >    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>> Method)
>> > >> > > >    at
>> > >> > > > sun.reflect.NativeMethodAccessorImpl.invoke(
>> > >> > > NativeMethodAccessorImpl.java:62)
>> > >> > > >    at
>> > >> > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(
>> > >> > > DelegatingMethodAccessorImpl.java:43)
>> > >> > > >    at java.lang.reflect.Method.invoke(Method.java:498)
>> > >> > > >    at
>> > >> > > > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(
>> > >> > > FrameworkMethod.java:50)
>> > >> > > >    at
>> > >> > > > org.junit.internal.runners.model.ReflectiveCallable.run(
>> > >> > > ReflectiveCallable.java:12)
>> > >> > > >    at
>> > >> > > > org.junit.runners.model.FrameworkMethod.invokeExplosively(
>> > >> > > FrameworkMethod.java:47)
>> > >> > > >    at
>> > >> > > > org.junit.internal.runners.statements.InvokeMethod.
>> > >> > > evaluate(InvokeMethod.java:17)
>> > >> > > >    at org.junit.runners.ParentRunner.runLeaf(
>> > ParentRunner.java:325)
>> > >> > > >    at
>> > >> > > > org.junit.runners.BlockJUnit4ClassRunner.runChild(
>> > >> > > BlockJUnit4ClassRunner.java:78)
>> > >> > > >    at
>> > >> > > > org.junit.runners.BlockJUnit4ClassRunner.runChild(
>> > >> > > BlockJUnit4ClassRunner.java:57)
>> > >> > > >    at org.junit.runners.ParentRunner
>> $3.run(ParentRunner.java:290)
>> > >> > > >    at org.junit.runners.ParentRunner$1.schedule(
>> > ParentRunner.java:
>> > >> 71)
>> > >> > > >    at org.junit.runners.ParentRunner.runChildren(
>> > >> > ParentRunner.java:288)
>> > >> > > >    at org.junit.runners.ParentRunner.access$000(
>> > ParentRunner.java:
>> > >> 58)
>> > >> > > >    at org.junit.runners.ParentRunner$2.evaluate(
>> > ParentRunner.java:
>> > >> 268)
>> > >> > > >    at
>> > >> > > > org.junit.internal.runners.statements.RunBefores.
>> > >> > > evaluate(RunBefores.java:26)
>> > >> > > >    at
>> > >> > > > org.junit.internal.runners.statements.RunAfters.evaluate(
>> > >> > > RunAfters.java:27)
>> > >> > > >    at org.junit.runners.ParentRunner
>> .run(ParentRunner.java:363)
>> > >> > > >    at
>> > >> > > > org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.
>> run(
>> > >> > > JUnit4TestReference.java:86)
>> > >> > > >    at
>> > >> > > > org.eclipse.jdt.internal.junit.runner.TestExecution.
>> > >> > > run(TestExecution.java:38)
>> > >> > > >    at
>> > >> > > > org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.
>> > >> > > runTests(RemoteTestRunner.java:459)
>> > >> > > >    at
>> > >> > > > org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.
>> > >> > > runTests(RemoteTestRunner.java:678)
>> > >> > > >    at
>> > >> > > > org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.
>> > >> > > run(RemoteTestRunner.java:382)
>> > >> > > >    at
>> > >> > > > org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.
>> > >> > > main(RemoteTestRunner.java:192)
>> > >> > > > Caused by: *java.lang.NullPointerException*
>> > >> > > >    at org.apache.calcite.interpreter.AggregateNode.
>> > getAccumulator(
>> > >> > > > *AggregateNode.java:150*)
>> > >> > > >    at
>> > >> > > > org.apache.calcite.interpreter.AggregateNode.<
>> > >> > > init>(AggregateNode.java:86)
>> > >> > > >    at
>> > >> > > > org.apache.calcite.interpreter.Nodes$CoreCompiler.visit(
>> > >> Nodes.java:47)
>> > >> > > >    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>> Method)
>> > >> > > >    at
>> > >> > > > sun.reflect.NativeMethodAccessorImpl.invoke(
>> > >> > > NativeMethodAccessorImpl.java:62)
>> > >> > > >    at
>> > >> > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(
>> > >> > > DelegatingMethodAccessorImpl.java:43)
>> > >> > > >    at java.lang.reflect.Method.invoke(Method.java:498)
>> > >> > > >    at
>> > >> > > > org.apache.calcite.util.ReflectUtil.invokeVisitorInternal(
>> > >> > > ReflectUtil.java:257)
>> > >> > > >    at
>> > >> > > > org.apache.calcite.util.ReflectUtil.invokeVisitor(
>> > >> > ReflectUtil.java:214)
>> > >> > > >    at
>> > >> > > > org.apache.calcite.util.ReflectUtil$1.invokeVisitor(
>> > >> > > ReflectUtil.java:464)
>> > >> > > >    at
>> > >> > > > org.apache.calcite.interpreter.Interpreter$Compiler.visit(
>> > >> Interpreter.
>> > >> > > java:480)
>> > >> > > >    at
>> > >> > > > org.apache.calcite.interpreter.Interpreter$Compiler.visitRoot(
>> > >> > > Interpreter.java:437)
>> > >> > > >    at
>> > >> > > > org.apache.calcite.interpreter.Interpreter.<init>(
>> > >> Interpreter.java:78)
>> > >> > > >    at Baz.bind(Unknown Source)
>> > >> > > >    at
>> > >> > > > org.apache.calcite.jdbc.CalcitePrepare$
>> > CalciteSignature.enumerable(
>> > >> > > CalcitePrepare.java:365)
>> > >> > > >    at
>> > >> > > > org.apache.calcite.jdbc.CalciteConnectionImpl.enumerable(
>> > >> > > CalciteConnectionImpl.java:301)
>> > >> > > >    at
>> > >> > > > org.apache.calcite.jdbc.CalciteMetaImpl._createIterable(
>> > >> > > CalciteMetaImpl.java:559)
>> > >> > > >    at
>> > >> > > > org.apache.calcite.jdbc.CalciteMetaImpl.createIterable(
>> > >> > > CalciteMetaImpl.java:550)
>> > >> > > >    at
>> > >> > > > org.apache.calcite.avatica.AvaticaResultSet.execute(
>> > >> > > AvaticaResultSet.java:204)
>> > >> > > >    at
>> > >> > > > org.apache.calcite.jdbc.CalciteResultSet.execute(
>> > >> > > CalciteResultSet.java:67)
>> > >> > > >    at
>> > >> > > > org.apache.calcite.jdbc.CalciteResultSet.execute(
>> > >> > > CalciteResultSet.java:44)
>> > >> > > >    at
>> > >> > > > org.apache.calcite.avatica.AvaticaConnection$1.execute(
>> > >> > > AvaticaConnection.java:630)
>> > >> > > >    at
>> > >> > > > org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(
>> > >> > > CalciteMetaImpl.java:619)
>> > >> > > >    at
>> > >> > > > org.apache.calcite.avatica.AvaticaConnection.
>> > >> > prepareAndExecuteInternal(
>> > >> > > AvaticaConnection.java:638)
>> > >> > > >    at
>> > >> > > > org.apache.calcite.avatica.AvaticaStatement.executeInternal(
>> > >> > > AvaticaStatement.java:149)
>> > >> > > >    ... 27 more
>> > >> > > >
>> > >> > > >
>> > >> > > > Thanks,
>> > >> > > > Gelbana
>> > >> > > > ​​
>> > >> > >
>> > >> > >
>> > >> >
>> > >>
>> > >>
>> > >>
>> > >> --
>> > >> "So you have to trust that the dots will somehow connect in your
>> > future."
>> > >>
>> > >
>> > >
>> >
>>
>
>