osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Generic Map type (_MAP) and select star (*)


Hi Stamatis,

Sorry for not replying earlier.

So when querying on views (and using order by) I am able to map expression
(`EXPR$1`) and original column name (`a`) using rowType (from converter).
```sql
create view as select _MAP['a'] as a, _MAP['b'] as b from elastic;
select * from view order by a
```

When querying (rather using `order by _MAP['a']`) the mapping between
`EXPR$1` and `a` is not available in rowType (of converter):

```sql
-- mapping between EXPR$1 and 'a' is lost in RelNode.rowType()
select * from elastic order by _MAP['a']
```

I can still manually get the mapping from Project, but not sure if this is
the best way to handle such cases.

Hope it makes sense.

Regards,
Andrei.

On Mon, Oct 8, 2018 at 2:57 AM Stamatis Zampetakis <zabetak@xxxxxxxxx>
wrote:

> Hi Andrei,
>
> I have not played around with map types so I am not familiar with how it
> works.
>
> On the other hand, I am not sure I understand what names are missing in the
> example you posted.
> I suppose that instead of EXPR$1, EXPR$2, EXPR$3 you are expecting
> something different.
> Can you provide an example of what is missing?
>
> Best,
> Stamatis
>
>
>
>
> Στις Παρ, 5 Οκτ 2018 στις 4:56 μ.μ., ο/η Andrei Sereda <andrei@xxxxxxxxx>
> έγραψε:
>
> > Hi Stamatis,
> >
> > Thanks for your reply. It makes more sense now.
> >
> > I have one more question though. Do you know what is recommended way to
> > retrieve original projection names (in converter) when using map types :
> > ```sql
> > select _MAP['a'], _MAP['b'] from table order by _MAP['c']
> > ```
> >
> > `getRowType.fieldList` Is translated to RelDataType(s) where original
> field
> > names are lost
> >
> > ```
> > 0 = {RelDataTypeFieldImpl@12260} "_MAP" -> "(VARCHAR, ANY) MAP"
> > 1 = {RelDataTypeFieldImpl@12261} "EXPR$1" -> "ANY"
> > 2 = {RelDataTypeFieldImpl@12262} "EXPR$2" -> "ANY"
> > 3 = {RelDataTypeFieldImpl@12263} "EXPR$3" -> "ANY"
> > ```
> > I presume some implementation of RexVisitor ?
> >
> > Regards,
> > Andrei.
> >
> > On Thu, Oct 4, 2018 at 6:09 PM Stamatis Zampetakis <zabetak@xxxxxxxxx>
> > wrote:
> >
> > > Hi Andrei,
> > >
> > > As you correctly noticed the extra projection column comes from the
> ORDER
> > > BY clause.
> > >
> > > In order to understand why have a look at the Javadoc of RelRoot class
> (
> > >
> > >
> >
> https://github.com/apache/calcite/blob/c39bfaa02a06ac91575076a6e74f29863923f5eb/core/src/main/java/org/apache/calcite/rel/RelRoot.java#L37
> > > ).
> > >
> > > If you want to understand how RelRoot is used the best way would be to
> > > start from the creation of the object in CalcitePrepareImpl class.(
> > >
> > >
> >
> https://github.com/apache/calcite/blob/c39bfaa02a06ac91575076a6e74f29863923f5eb/core/src/main/java/org/apache/calcite/prepare/CalcitePrepareImpl.java#L1113
> > > )
> > >
> > > Best,
> > > Stamatis
> > >
> > >
> > > Στις Πέμ, 4 Οκτ 2018 στις 9:19 μ.μ., ο/η Andrei Sereda
> <andrei@xxxxxxxxx
> > >
> > > έγραψε:
> > >
> > > > but just a minor point that when you post links to specific lines of
> > > code,
> > > > it’s helpful if that link is to a specific revision
> > > >
> > > > Sure will do.
> > > >
> > > > Doing some more debugging it seems that projections are “imported”
> from
> > > > order
> > > > by clause :
> > > >
> > > > select * from elastic.zips order by _MAP['state'], _MAP['city']
> > > >
> > > >  Projects (3) : [$0, ITEM($0, 'state'), ITEM($0, 'city')]
> > > >
> > > > On Thu, Oct 4, 2018 at 2:53 PM Michael Mior mmior@xxxxxxxxxx
> > > > <http://mailto:mmior@xxxxxxxxxx> wrote:
> > > > >
> > > >
> > > > Unfortunately I don’t have an answer, but just a minor point that
> when
> > > you
> > > > post links to specific lines of code, it’s helpful if that link is
> to a
> > > > specific revision. You can press the “y” shortcut to get this link
> > > (pasted
> > > > below).
> > > >
> > > >
> > > >
> > >
> >
> https://github.com/apache/calcite/blob/c39bfaa02a06ac91575076a6e74f29863923f5eb/elasticsearch/src/main/java/org/apache/calcite/adapter/elasticsearch/ElasticsearchProject.java#L67
> > > >
> > > > —
> > > > Michael Mior
> > > > mmior@xxxxxxxxxx
> > > >
> > > > Le jeu. 4 oct. 2018 à 14:47, Andrei Sereda andrei@xxxxxxxxx
> > > > <http://mailto:andrei@xxxxxxxxx> a écrit :
> > > >
> > > > Hello,
> > > >
> > > > I would like to understand how exactly generic map type (_MAP) works
> > > > with select
> > > >
> > > >    - expression. Say I have the following query (on document
> database)
> > :
> > > >
> > > > — first query: 2 projectionsselect * from elastic.zips order by
> > > > _MAP[‘state’] — second query: single projectionselect _MAP[‘state’]
> > > > from elastic.zips order by _MAP[‘state’]
> > > >
> > > > For first query (select *) I’m getting 2 projections (inside
> > > > ElasticsearchProject
> > > > <
> > > >
> > > >
> > >
> >
> https://github.com/apache/calcite/blob/master/elasticsearch/src/main/java/org/apache/calcite/adapter/elasticsearch/ElasticsearchProject.java#L67
> > > >
> > > > )
> > > > for some reason
> > > >
> > > > 0 = {RexInputRef@12207} “[image: 0]0”
> > > > type = {MapSqlType@12211} “(VARCHAR, ANY) MAP”
> > > > digest = “[image: 0]0, ‘state’)”
> > > > op = {SqlItemOperator@12213} “ITEM”
> > > > operands = {RegularImmutableList@12214} size = 2
> > > > type = {BasicSqlType@12215} “ANY”
> > > > digest = “ITEM($0, ‘state’)”
> > > >
> > > > For second query and when using views there is no such issue.
> > > >
> > > > My question is how to correctly identify select * when rowType is
> map (
> > > > MapSqlType
> > > > <
> > > >
> > > >
> > >
> >
> https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql/type/MapSqlType.java
> > > >
> > > > )?
> > > >
> > > > Thank you,
> > > > Andrei.
> > > >
> > >
> >
>