osdir.com

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: RxNode (Filter) / Translator for various adapters (ES, Mongo, Geode etc.). Possible Bug ?


Can somebody please give me a hint on why Mongo Translator
<https://github.com/apache/calcite/blob/master/mongodb/src/main/java/org/apache/calcite/adapter/mongodb/MongoFilter.java#L77>
was
implemented the way it is now ? Ie why RexVisitor
<https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/rex/RexVisitor.java>
is not being used to (recursively) translate RelNode into mongo query ?

Would like to take a stub at CALCITE-2331 but, before,  want to ensure I'm
not missing anything.

On Tue, May 29, 2018 at 7:20 AM, Andrei Sereda <andrei@xxxxxxxxx> wrote:

> Any reason why RexNode visitor API is not used for implementing Mongo
> query translator ?
>
> Looking at current source code (MongoFilter
> <https://github.com/apache/calcite/blob/master/mongodb/src/main/java/org/apache/calcite/adapter/mongodb/MongoFilter.java#L91>)
> it seems one expects certain order of conditions (eg. first AND then OR)
> which is not generic enough. Is it for operator precedence ?
>
>
>
> On Mon, May 28, 2018 at 11:20 PM, Andrei Sereda <andrei@xxxxxxxxx> wrote:
>
>> Yes seems like subtree is not visited recursively for OR opeator(s).
>>
>> I have added a test to track this issue: https://github.com/apac
>> he/calcite/pull/703
>>
>>
>>
>> On Mon, May 28, 2018 at 11:06 PM, liu xiaorui <sharyedge@xxxxxxxxx>
>> wrote:
>>
>>> I also encountered this problem .   After I debug the code I found the
>>> cause of the problem .  for example  condition is A AND (B OR C )   .
>>>
>>>      orNode is AND (A,OR (B,C))  and  then execute method translateAdd
>>> , condition A can  be translated successfully , and exception will be throw
>>> out when translating  condition  OR(B,C) .  so I think we can think
>>> about recursively.
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> On 29 May 2018 at 09:56, Julian Hyde <jhyde@xxxxxxxxxx> wrote:
>>>
>>>> Looks like a bug. I don't know that code well, so I can't speculate
>>>> what might be the cause.
>>>>
>>>> Can you log a JIRA case please? If you can provide a unit test (say, a
>>>> method to add to MongoAdapterIT) even better.
>>>>
>>>> On Mon, May 28, 2018 at 2:47 PM, Andrei Sereda <andrei@xxxxxxxxx>
>>>> wrote:
>>>> > Hello,
>>>> >
>>>> > Please correct me if my expectations are inaccurate about query
>>>> predicates
>>>> > in Calcite.
>>>> >
>>>> > I think there is an issue in how expression translators work when AST
>>>> is
>>>> > more complex (has more depth). Same code / logic is used across
>>>> different
>>>> > adapters (Mongo / Geode / ES) so multiple data-sources might be
>>>> affected.
>>>> >
>>>> > Examples
>>>> >
>>>> > 1. Works
>>>> >
>>>> > 0: jdbc:calcite:model=target/test-classes/mod> select count(*) from
>>>> > "elastic" where _MAP['foo'] in ('1') and true;
>>>> >
>>>> > 2. Works
>>>> > 0: jdbc:calcite:model=target/test-classes/mod> select count(*) from
>>>> > "elastic" where _MAP['foo'] ='1' and true;
>>>> >
>>>> > 3. Works
>>>> > 0: jdbc:calcite:model=target/test-classes/mod> select count(*) from
>>>> > "elastic" where _MAP['foo'] in ('1', '2');
>>>> >
>>>> >
>>>> > 4. Fails
>>>> >
>>>> > 0: jdbc:calcite:model=target/test-classes/mod> select count(*) from
>>>> > "elastic" where _MAP['foo'] in ('1', '2') and true;
>>>> >
>>>> >
>>>> > 5. Fails
>>>> >
>>>> > 0: jdbc:calcite:model=target/test-classes/mod> select count(*) from
>>>> > "elastic" where true and _MAP['foo'] in ('1', '2');
>>>> >
>>>> >
>>>> > 6. Fails
>>>> >
>>>> > 0: jdbc:calcite:model=target/test-classes/mod> select count(*) from
>>>> > "elastic" where (_MAP['foo'] ='1' or _MAP['foo'] = '2') and true;
>>>> >
>>>> >
>>>> >
>>>> > Exception Stack (Elastic Adapter)
>>>> >
>>>> >
>>>> > java.lang.AssertionError: cannot translate OR(=(ITEM($0, 'foo'), '1'),
>>>> > =(ITEM($0, 'foo'), '2'))
>>>> >
>>>> >         at org.apache.calcite.adapter.ela
>>>> sticsearch.ElasticsearchFilter$
>>>> > Translator.translateMatch2(ElasticsearchFilter.java:234)
>>>> >
>>>> >         at org.apache.calcite.adapter.ela
>>>> sticsearch.ElasticsearchFilter$
>>>> > Translator.translateAnd(ElasticsearchFilter.java:158)
>>>> >
>>>> >         at org.apache.calcite.adapter.ela
>>>> sticsearch.ElasticsearchFilter$
>>>> > Translator.translateOr(ElasticsearchFilter.java:115)
>>>> >
>>>> >         at org.apache.calcite.adapter.ela
>>>> sticsearch.ElasticsearchFilter$
>>>> > Translator.translateMatch(ElasticsearchFilter.java:101)
>>>> >
>>>> >         at org.apache.calcite.adapter.ela
>>>> sticsearch.ElasticsearchFilter$
>>>> > Translator.access$000(ElasticsearchFilter.java:87)
>>>> >
>>>> >         at org.apache.calcite.adapter.elasticsearch.
>>>> > ElasticsearchFilter.implement(ElasticsearchFilter.java:80)
>>>> >
>>>> >         at org.apache.calcite.adapter.elasticsearch.
>>>> > ElasticsearchRel$Implementor.visitChild(ElasticsearchRel.java:53)
>>>> >
>>>> >         at org.apache.calcite.adapter.elasticsearch.
>>>> > ElasticsearchToEnumerableConverter.implement(ElasticsearchTo
>>>> EnumerableConve
>>>> > rter.java:71)
>>>> >
>>>> >         at org.apache.calcite.adapter.enu
>>>> merable.EnumerableRelImplementor.
>>>> > visitChild(EnumerableRelImplementor.java:98)
>>>> >
>>>> >         at org.apache.calcite.adapter.enumerable.
>>>> > EnumerableAggregate.implement(EnumerableAggregate.java:106)
>>>> >
>>>> > Exception Stack (Mongo Adapter)
>>>> >
>>>> > 0: jdbc:calcite:model=target/test-classes/mod> select * from
>>>> "mongo"."col"
>>>> > where _MAP['foo'] in ('1', '2') and true;
>>>> >
>>>> > java.lang.AssertionError: cannot translate OR(=(ITEM($0, 'foo'), '1'),
>>>> > =(ITEM($0, 'foo'), '2'))
>>>> >
>>>> >         at org.apache.calcite.adapter.mongodb.MongoFilter$
>>>> > Translator.translateMatch2(MongoFilter.java:181)
>>>> >
>>>> >         at org.apache.calcite.adapter.mongodb.MongoFilter$
>>>> > Translator.translateAnd(MongoFilter.java:116)
>>>> >
>>>> >         at org.apache.calcite.adapter.mongodb.MongoFilter$
>>>> > Translator.translateOr(MongoFilter.java:98)
>>>> >
>>>> >         at org.apache.calcite.adapter.mongodb.MongoFilter$
>>>> > Translator.translateMatch(MongoFilter.java:91)
>>>> >
>>>> >         at org.apache.calcite.adapter.mongodb.MongoFilter$
>>>> > Translator.access$000(MongoFilter.java:77)
>>>> >
>>>> >         at org.apache.calcite.adapter.mongodb.MongoFilter.implement(
>>>> > MongoFilter.java:72)
>>>> >
>>>> >         at org.apache.calcite.adapter.mongodb.MongoRel$Implementor.
>>>> > visitChild(MongoRel.java:51)
>>>> >
>>>> >         at org.apache.calcite.adapter.mon
>>>> godb.MongoToEnumerableConverter.
>>>> > implement(MongoToEnumerableConverter.java:84)
>>>> >
>>>> >         at org.apache.calcite.adapter.enu
>>>> merable.EnumerableRelImplementor.
>>>> > implementRoot(EnumerableRelImplementor.java:103)
>>>> >
>>>> >         at org.apache.calcite.adapter.enu
>>>> merable.EnumerableInterpretable.
>>>> > toBindable(EnumerableInterpretable.java:92)
>>>> >
>>>> >
>>>> >
>>>> > Anything I'm doing wrong ?
>>>> >
>>>> > Thanks,
>>>> > Andrei.
>>>>
>>>
>>>
>>
>