OSDir


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Elasticsearch integration test failures


Interesting, I tested both environments back then and it seemed to work. Good thing that we are about to replace the IT tests with using local ES nodes. This will finally allow us to run these tests regularly.

Regarding ES2 and ES5, I'd suggest you have separate ElasticsearchProjectRule for each version. In Elasticsearch2Table and Elasticsearch5Table you add the rules to the ElasticsearchTableScan constructor.


Mit freundlichen Grüßen,
------------------------------------------------------------------------
*Christian Beikov*
Am 13.06.2018 um 06:15 schrieb Andrei Sereda:
The issue seems to have started since CALCITE-1967 (ES5 support).
ES2 expects keyword _source in scripted fields while ES5 param._source.
See  ElasticsearchProject
<https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/adapter/elasticsearch/ElasticsearchProject.java#L79>
.

Also ES2 doesn't like when both script_fields and _source are present in
the query. So if there are some explicit projections all have to be present
in script_fields.

Any elegant  way to differentiate at runtime ES2 vs ES5 inside
ElasticsearchProject ?



On Mon, Jun 11, 2018 at 5:46 PM Christian Beikov <christian.beikov@xxxxxxxxx>
wrote:

I'm fine with a single PR as long as you split commits up.

Andrei Sereda <andrei@xxxxxxxxx> schrieb am Mo., 11. Juni 2018, 23:24:

Hi Christian,

I'm working in parallel on CALCITE-2331 evaluation of predicate "(A or B)
and C" for ES. Can I fix existing bugs as part of that (or separate)
commit
?

I was planning to do it just not in the same PR.

On Mon, Jun 11, 2018 at 4:31 PM, Christian Beikov <
christian.beikov@xxxxxxxxx> wrote:

For me that's fine, though it would be great if you could fix those
given
your time permits it.

Andrei Sereda <andrei@xxxxxxxxx> schrieb am Mo., 11. Juni 2018, 22:12:

There are some tests which have been failing for while. Is it OK if I
temporary mark them @Ignore(d) ?

On Mon, Jun 11, 2018 at 3:48 PM, Christian Beikov <
christian.beikov@xxxxxxxxx> wrote:

If you migrate the existing tests to use the local node in that PR,
I'd
consider merging that PR given that Travis CI reports no test
failures.

Mit freundlichen Grüßen,
------------------------------------------------------------
------------
*Christian Beikov*
Am 11.06.2018 um 14:33 schrieb Andrei Sereda:

Do the tests succeed with the local node?
IT tests haven't been migrated to local node yet. I wanted to do
it
as
separate commit.
There are new tests which run against embedded instance
successfully.
Pls
see PR 716 for details.

I don't see a reason for running ES tests again on a "real" ES
instance.
What does that even mean?
The only reason, I see, in connecting to external instance is
testing
compatibility with different versions of the engine.

On Mon, Jun 11, 2018 at 7:16 AM, Michael Mior <mmior@xxxxxxxxxx>
wrote:
Good point. Since ES is written in Java and we're running the same
code
testing on a local node, it's probably not necessary to test
against
a
separate ES cluster.
--
Michael Mior
mmior@xxxxxxxxxx


Le lun. 11 juin 2018 à 01:44, Christian Beikov <
christian.beikov@xxxxxxxxx
a écrit :

I don't see a reason for running ES tests again on a "real" ES
instance.
What does that even mean?

With a local node, we would be testing functionality against
almost
the
real thing. The only difference is that a real setup has a
cluster,
but
that doesn't change the functionality. So I'm all in on using
just a
local node which is also something I wanted to do for a long
time,
but
you got to it before I did.

Do the tests succeed with the local node?


Mit freundlichen Grüßen,
------------------------------------------------------------
------------
*Christian Beikov*
Am 07.06.2018 um 20:47 schrieb Andrei Sereda:

Hi Volodymyr,

We're trying to bridge the gap between unit and integration
tests
(which
are run infrequently) using fakes
<https://martinfowler.com/bliki/InMemoryTestDatabase.html> in
Calcite.
Currently the plan is to use fongo (see CALCITE-2345 for mongo
adapter)
and

local elastic node (see CALCITE-2347 for elastic).

During unit tests code will run against fake instances. For IT
profile
they

will run against real instances (as before).

Regards,
Andrei.


On Thu, Jun 7, 2018 at 2:32 PM, Julian Hyde <jhyde@xxxxxxxxxx>
wrote:
Unfortunately, I don’t know. We don’t run the integration tests
as
often
as we should.
On Jun 7, 2018, at 10:20 AM, Vova Vysotskyi <vvovyk@xxxxxxxxx
wrote:
Hi all,
I tried to run Elasticsearch integration tests for the
current
master,
and
many tests failed.

Is it a known problem, or just an issue of my env?

I noticed that one failure appears after the change made in

CALCITE-2316.
Most of the other failures are observed for 1.16 version also.
Kind regards,
Volodymyr Vysotskyi