osdir.com

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: RxNode (Filter) / Translator for various adapters (ES, Mongo, Geode etc.). Possible Bug ?


Unfortunately, I'm not sure why this is the case. Perhaps Julian can give
some insight since he was the original author.
--
Michael Mior
mmior@xxxxxxxxxx



Le mar. 29 mai 2018 à 21:53, Andrei Sereda <andrei@xxxxxxxxx> a écrit :

> Can somebody please give me a hint on why Mongo Translator
> <
> https://github.com/apache/calcite/blob/master/mongodb/src/main/java/org/apache/calcite/adapter/mongodb/MongoFilter.java#L77
> >
> was
> implemented the way it is now ? Ie why RexVisitor
> <
> https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/rex/RexVisitor.java
> >
> is not being used to (recursively) translate RelNode into mongo query ?
>
> Would like to take a stub at CALCITE-2331 but, before,  want to ensure I'm
> not missing anything.
>
> On Tue, May 29, 2018 at 7:20 AM, Andrei Sereda <andrei@xxxxxxxxx> wrote:
>
> > Any reason why RexNode visitor API is not used for implementing Mongo
> > query translator ?
> >
> > Looking at current source code (MongoFilter
> > <
> https://github.com/apache/calcite/blob/master/mongodb/src/main/java/org/apache/calcite/adapter/mongodb/MongoFilter.java#L91
> >)
> > it seems one expects certain order of conditions (eg. first AND then OR)
> > which is not generic enough. Is it for operator precedence ?
> >
> >
> >
> > On Mon, May 28, 2018 at 11:20 PM, Andrei Sereda <andrei@xxxxxxxxx>
> wrote:
> >
> >> Yes seems like subtree is not visited recursively for OR opeator(s).
> >>
> >> I have added a test to track this issue: https://github.com/apac
> >> he/calcite/pull/703
> >>
> >>
> >>
> >> On Mon, May 28, 2018 at 11:06 PM, liu xiaorui <sharyedge@xxxxxxxxx>
> >> wrote:
> >>
> >>> I also encountered this problem .   After I debug the code I found the
> >>> cause of the problem .  for example  condition is A AND (B OR C )   .
> >>>
> >>>      orNode is AND (A,OR (B,C))  and  then execute method translateAdd
> >>> , condition A can  be translated successfully , and exception will be
> throw
> >>> out when translating  condition  OR(B,C) .  so I think we can think
> >>> about recursively.
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> On 29 May 2018 at 09:56, Julian Hyde <jhyde@xxxxxxxxxx> wrote:
> >>>
> >>>> Looks like a bug. I don't know that code well, so I can't speculate
> >>>> what might be the cause.
> >>>>
> >>>> Can you log a JIRA case please? If you can provide a unit test (say, a
> >>>> method to add to MongoAdapterIT) even better.
> >>>>
> >>>> On Mon, May 28, 2018 at 2:47 PM, Andrei Sereda <andrei@xxxxxxxxx>
> >>>> wrote:
> >>>> > Hello,
> >>>> >
> >>>> > Please correct me if my expectations are inaccurate about query
> >>>> predicates
> >>>> > in Calcite.
> >>>> >
> >>>> > I think there is an issue in how expression translators work when
> AST
> >>>> is
> >>>> > more complex (has more depth). Same code / logic is used across
> >>>> different
> >>>> > adapters (Mongo / Geode / ES) so multiple data-sources might be
> >>>> affected.
> >>>> >
> >>>> > Examples
> >>>> >
> >>>> > 1. Works
> >>>> >
> >>>> > 0: jdbc:calcite:model=target/test-classes/mod> select count(*) from
> >>>> > "elastic" where _MAP['foo'] in ('1') and true;
> >>>> >
> >>>> > 2. Works
> >>>> > 0: jdbc:calcite:model=target/test-classes/mod> select count(*) from
> >>>> > "elastic" where _MAP['foo'] ='1' and true;
> >>>> >
> >>>> > 3. Works
> >>>> > 0: jdbc:calcite:model=target/test-classes/mod> select count(*) from
> >>>> > "elastic" where _MAP['foo'] in ('1', '2');
> >>>> >
> >>>> >
> >>>> > 4. Fails
> >>>> >
> >>>> > 0: jdbc:calcite:model=target/test-classes/mod> select count(*) from
> >>>> > "elastic" where _MAP['foo'] in ('1', '2') and true;
> >>>> >
> >>>> >
> >>>> > 5. Fails
> >>>> >
> >>>> > 0: jdbc:calcite:model=target/test-classes/mod> select count(*) from
> >>>> > "elastic" where true and _MAP['foo'] in ('1', '2');
> >>>> >
> >>>> >
> >>>> > 6. Fails
> >>>> >
> >>>> > 0: jdbc:calcite:model=target/test-classes/mod> select count(*) from
> >>>> > "elastic" where (_MAP['foo'] ='1' or _MAP['foo'] = '2') and true;
> >>>> >
> >>>> >
> >>>> >
> >>>> > Exception Stack (Elastic Adapter)
> >>>> >
> >>>> >
> >>>> > java.lang.AssertionError: cannot translate OR(=(ITEM($0, 'foo'),
> '1'),
> >>>> > =(ITEM($0, 'foo'), '2'))
> >>>> >
> >>>> >         at org.apache.calcite.adapter.ela
> >>>> sticsearch.ElasticsearchFilter$
> >>>> > Translator.translateMatch2(ElasticsearchFilter.java:234)
> >>>> >
> >>>> >         at org.apache.calcite.adapter.ela
> >>>> sticsearch.ElasticsearchFilter$
> >>>> > Translator.translateAnd(ElasticsearchFilter.java:158)
> >>>> >
> >>>> >         at org.apache.calcite.adapter.ela
> >>>> sticsearch.ElasticsearchFilter$
> >>>> > Translator.translateOr(ElasticsearchFilter.java:115)
> >>>> >
> >>>> >         at org.apache.calcite.adapter.ela
> >>>> sticsearch.ElasticsearchFilter$
> >>>> > Translator.translateMatch(ElasticsearchFilter.java:101)
> >>>> >
> >>>> >         at org.apache.calcite.adapter.ela
> >>>> sticsearch.ElasticsearchFilter$
> >>>> > Translator.access$000(ElasticsearchFilter.java:87)
> >>>> >
> >>>> >         at org.apache.calcite.adapter.elasticsearch.
> >>>> > ElasticsearchFilter.implement(ElasticsearchFilter.java:80)
> >>>> >
> >>>> >         at org.apache.calcite.adapter.elasticsearch.
> >>>> > ElasticsearchRel$Implementor.visitChild(ElasticsearchRel.java:53)
> >>>> >
> >>>> >         at org.apache.calcite.adapter.elasticsearch.
> >>>> > ElasticsearchToEnumerableConverter.implement(ElasticsearchTo
> >>>> EnumerableConve
> >>>> > rter.java:71)
> >>>> >
> >>>> >         at org.apache.calcite.adapter.enu
> >>>> merable.EnumerableRelImplementor.
> >>>> > visitChild(EnumerableRelImplementor.java:98)
> >>>> >
> >>>> >         at org.apache.calcite.adapter.enumerable.
> >>>> > EnumerableAggregate.implement(EnumerableAggregate.java:106)
> >>>> >
> >>>> > Exception Stack (Mongo Adapter)
> >>>> >
> >>>> > 0: jdbc:calcite:model=target/test-classes/mod> select * from
> >>>> "mongo"."col"
> >>>> > where _MAP['foo'] in ('1', '2') and true;
> >>>> >
> >>>> > java.lang.AssertionError: cannot translate OR(=(ITEM($0, 'foo'),
> '1'),
> >>>> > =(ITEM($0, 'foo'), '2'))
> >>>> >
> >>>> >         at org.apache.calcite.adapter.mongodb.MongoFilter$
> >>>> > Translator.translateMatch2(MongoFilter.java:181)
> >>>> >
> >>>> >         at org.apache.calcite.adapter.mongodb.MongoFilter$
> >>>> > Translator.translateAnd(MongoFilter.java:116)
> >>>> >
> >>>> >         at org.apache.calcite.adapter.mongodb.MongoFilter$
> >>>> > Translator.translateOr(MongoFilter.java:98)
> >>>> >
> >>>> >         at org.apache.calcite.adapter.mongodb.MongoFilter$
> >>>> > Translator.translateMatch(MongoFilter.java:91)
> >>>> >
> >>>> >         at org.apache.calcite.adapter.mongodb.MongoFilter$
> >>>> > Translator.access$000(MongoFilter.java:77)
> >>>> >
> >>>> >         at org.apache.calcite.adapter.mongodb.MongoFilter.implement(
> >>>> > MongoFilter.java:72)
> >>>> >
> >>>> >         at org.apache.calcite.adapter.mongodb.MongoRel$Implementor.
> >>>> > visitChild(MongoRel.java:51)
> >>>> >
> >>>> >         at org.apache.calcite.adapter.mon
> >>>> godb.MongoToEnumerableConverter.
> >>>> > implement(MongoToEnumerableConverter.java:84)
> >>>> >
> >>>> >         at org.apache.calcite.adapter.enu
> >>>> merable.EnumerableRelImplementor.
> >>>> > implementRoot(EnumerableRelImplementor.java:103)
> >>>> >
> >>>> >         at org.apache.calcite.adapter.enu
> >>>> merable.EnumerableInterpretable.
> >>>> > toBindable(EnumerableInterpretable.java:92)
> >>>> >
> >>>> >
> >>>> >
> >>>> > Anything I'm doing wrong ?
> >>>> >
> >>>> > Thanks,
> >>>> > Andrei.
> >>>>
> >>>
> >>>
> >>
> >
>