OSDir


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Elasticsearch integration test failures


The issue seems to have started since CALCITE-1967 (ES5 support).
ES2 expects keyword _source in scripted fields while ES5 param._source.
See  ElasticsearchProject
<https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/adapter/elasticsearch/ElasticsearchProject.java#L79>
.

Also ES2 doesn't like when both script_fields and _source are present in
the query. So if there are some explicit projections all have to be present
in script_fields.

Any elegant  way to differentiate at runtime ES2 vs ES5 inside
ElasticsearchProject ?



On Mon, Jun 11, 2018 at 5:46 PM Christian Beikov <christian.beikov@xxxxxxxxx>
wrote:

> I'm fine with a single PR as long as you split commits up.
>
> Andrei Sereda <andrei@xxxxxxxxx> schrieb am Mo., 11. Juni 2018, 23:24:
>
> > Hi Christian,
> >
> > I'm working in parallel on CALCITE-2331 evaluation of predicate "(A or B)
> > and C" for ES. Can I fix existing bugs as part of that (or separate)
> commit
> > ?
> >
> > I was planning to do it just not in the same PR.
> >
> > On Mon, Jun 11, 2018 at 4:31 PM, Christian Beikov <
> > christian.beikov@xxxxxxxxx> wrote:
> >
> > > For me that's fine, though it would be great if you could fix those
> given
> > > your time permits it.
> > >
> > > Andrei Sereda <andrei@xxxxxxxxx> schrieb am Mo., 11. Juni 2018, 22:12:
> > >
> > > > There are some tests which have been failing for while. Is it OK if I
> > > > temporary mark them @Ignore(d) ?
> > > >
> > > > On Mon, Jun 11, 2018 at 3:48 PM, Christian Beikov <
> > > > christian.beikov@xxxxxxxxx> wrote:
> > > >
> > > > > If you migrate the existing tests to use the local node in that PR,
> > I'd
> > > > > consider merging that PR given that Travis CI reports no test
> > failures.
> > > > >
> > > > >
> > > > > Mit freundlichen Grüßen,
> > > > > ------------------------------------------------------------
> > > ------------
> > > > > *Christian Beikov*
> > > > > Am 11.06.2018 um 14:33 schrieb Andrei Sereda:
> > > > >
> > > > >> Do the tests succeed with the local node?
> > > > >>>
> > > > >> IT tests haven't been migrated to local node yet. I wanted to do
> it
> > as
> > > > >> separate commit.
> > > > >> There are new tests which run against embedded instance
> > successfully.
> > > > Pls
> > > > >> see PR 716 for details.
> > > > >>
> > > > >> I don't see a reason for running ES tests again on a "real" ES
> > > instance.
> > > > >>>
> > > > >> What does that even mean?
> > > > >> The only reason, I see, in connecting to external instance is
> > testing
> > > > >> compatibility with different versions of the engine.
> > > > >>
> > > > >> On Mon, Jun 11, 2018 at 7:16 AM, Michael Mior <mmior@xxxxxxxxxx>
> > > wrote:
> > > > >>
> > > > >> Good point. Since ES is written in Java and we're running the same
> > > code
> > > > >>> testing on a local node, it's probably not necessary to test
> > against
> > > a
> > > > >>> separate ES cluster.
> > > > >>> --
> > > > >>> Michael Mior
> > > > >>> mmior@xxxxxxxxxx
> > > > >>>
> > > > >>>
> > > > >>> Le lun. 11 juin 2018 à 01:44, Christian Beikov <
> > > > >>> christian.beikov@xxxxxxxxx
> > > > >>> a écrit :
> > > > >>>
> > > > >>> I don't see a reason for running ES tests again on a "real" ES
> > > > instance.
> > > > >>>> What does that even mean?
> > > > >>>>
> > > > >>>> With a local node, we would be testing functionality against
> > almost
> > > > the
> > > > >>>> real thing. The only difference is that a real setup has a
> > cluster,
> > > > but
> > > > >>>> that doesn't change the functionality. So I'm all in on using
> > just a
> > > > >>>> local node which is also something I wanted to do for a long
> time,
> > > but
> > > > >>>> you got to it before I did.
> > > > >>>>
> > > > >>>> Do the tests succeed with the local node?
> > > > >>>>
> > > > >>>>
> > > > >>>> Mit freundlichen Grüßen,
> > > > >>>> ------------------------------------------------------------
> > > > >>>> ------------
> > > > >>>> *Christian Beikov*
> > > > >>>> Am 07.06.2018 um 20:47 schrieb Andrei Sereda:
> > > > >>>>
> > > > >>>>> Hi Volodymyr,
> > > > >>>>>
> > > > >>>>> We're trying to bridge the gap between unit and integration
> tests
> > > > >>>>>
> > > > >>>> (which
> > > > >>>
> > > > >>>> are run infrequently) using fakes
> > > > >>>>> <https://martinfowler.com/bliki/InMemoryTestDatabase.html> in
> > > > Calcite.
> > > > >>>>>
> > > > >>>>> Currently the plan is to use fongo (see CALCITE-2345 for mongo
> > > > adapter)
> > > > >>>>>
> > > > >>>> and
> > > > >>>>
> > > > >>>>> local elastic node (see CALCITE-2347 for elastic).
> > > > >>>>>
> > > > >>>>> During unit tests code will run against fake instances. For IT
> > > > profile
> > > > >>>>>
> > > > >>>> they
> > > > >>>>
> > > > >>>>> will run against real instances (as before).
> > > > >>>>>
> > > > >>>>> Regards,
> > > > >>>>> Andrei.
> > > > >>>>>
> > > > >>>>>
> > > > >>>>> On Thu, Jun 7, 2018 at 2:32 PM, Julian Hyde <jhyde@xxxxxxxxxx>
> > > > wrote:
> > > > >>>>>
> > > > >>>>> Unfortunately, I don’t know. We don’t run the integration tests
> > as
> > > > >>>>>>
> > > > >>>>> often
> > > > >>>
> > > > >>>> as we should.
> > > > >>>>>>
> > > > >>>>>> On Jun 7, 2018, at 10:20 AM, Vova Vysotskyi <vvovyk@xxxxxxxxx
> >
> > > > >>>>>>>
> > > > >>>>>> wrote:
> > > > >>>
> > > > >>>> Hi all,
> > > > >>>>>>>
> > > > >>>>>>> I tried to run Elasticsearch integration tests for the
> current
> > > > >>>>>>>
> > > > >>>>>> master,
> > > > >>>
> > > > >>>> and
> > > > >>>>>>
> > > > >>>>>>> many tests failed.
> > > > >>>>>>>
> > > > >>>>>>> Is it a known problem, or just an issue of my env?
> > > > >>>>>>>
> > > > >>>>>>> I noticed that one failure appears after the change made in
> > > > >>>>>>>
> > > > >>>>>> CALCITE-2316.
> > > > >>>>
> > > > >>>>> Most of the other failures are observed for 1.16 version also.
> > > > >>>>>>>
> > > > >>>>>>> Kind regards,
> > > > >>>>>>> Volodymyr Vysotskyi
> > > > >>>>>>>
> > > > >>>>>>
> > > > >>>>
> > > > >
> > > >
> > >
> >
>