osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: CassandraAdapterTest failure


Is it possible that you run several maven processes in parallel ?

Unfortunately currently there can be only one CassandraAdapterTest (setup()
method really) running on a box in parallel.

On Wed, Aug 1, 2018 at 4:55 PM Andrei Sereda <andrei@xxxxxxxxx> wrote:

> Hi Julian,
>
> Can you please tell if you're getting same (or different) error for all
> java versions (8,9,10 etc.)  ?
> Do you get this error consistently or intermittently (flaky behaviour) ?
> What is the stack trace ?
>
> It will help me debug the issue.
>
> Regards,
> Andrei.
>
> On Wed, Aug 1, 2018 at 4:40 PM Igor Kryvenko <kryvenko7igor@xxxxxxxxx>
> wrote:
>
>> Also works fine for JDK 10.
>>
>> ➜  cassandra git:(master) ✗ java -version
>> java version "10.0.2" 2018-07-17
>> Java(TM) SE Runtime Environment 18.3 (build 10.0.2+13)
>> Java HotSpot(TM) 64-Bit Server VM 18.3 (build 10.0.2+13, mixed mode)
>>
>> Kind regards
>> Igor Kryvenko
>>
>>
>> On Wed, 1 Aug 2018 at 23:03, Julian Hyde <jhyde@xxxxxxxxxx> wrote:
>>
>> > What about on JDK 10? Here’s my java version:
>> >
>> > $ java -version
>> > java version "10" 2018-03-20
>> > Java(TM) SE Runtime Environment 18.3 (build 10+46)
>> > Java HotSpot(TM) 64-Bit Server VM 18.3 (build 10+46, mixed mode)
>> >
>> >
>> > > On Aug 1, 2018, at 12:58 PM, Igor Kryvenko <kryvenko7igor@xxxxxxxxx>
>> > wrote:
>> > >
>> > > "mvn -Pit clean test" works fine for me.
>> > > Ubuntu 18.04
>> > > java version "1.8.0_171"
>> > >
>> > > Kind regards
>> > > Igor Kryvenko
>> > >
>> > >
>> > > On Wed, 1 Aug 2018 at 22:34, Julian Hyde <jhyde@xxxxxxxxxx> wrote:
>> > >
>> > >> The test is failing every time for me on JDK 10.  The command “mvn
>> -Pit
>> > >> clean test” will probably reproduce it for most people.
>> > >>
>> > >> Can other folks please try to reproduce this? I’m getting close to
>> > saying
>> > >> that we should back this change out even though apparently only I can
>> > >> reproduce the failure.
>> > >>
>> > >> Julian
>> > >>
>> > >>
>> > >>
>> > >>
>> > >>
>> > >>
>> > >>> On Jul 30, 2018, at 1:35 PM, Andrei Sereda <andrei@xxxxxxxxx>
>> wrote:
>> > >>>
>> > >>> Julian, I have amended the commit message in f0b00f0c
>> > >>>
>> > >>>
>> > >>> On Mon, Jul 30, 2018 at 4:22 PM Julian Hyde <jhyde@xxxxxxxxxx>
>> wrote:
>> > >>>
>> > >>>> I’m running tests on the PR now.
>> > >>>>
>> > >>>> Since commit comments are a soap-box of mine, I will remark that
>> > rather
>> > >>>> than
>> > >>>>
>> > >>>> [CALCITE-2428] Fix cassandra unit test initialization. (Andrei
>> Sereda)
>> > >>>>
>> > >>>> the commit comment should be
>> > >>>>
>> > >>>> [CALCITE-2428] Cassandra unit test fails to parse version string
>> > >> (Andrei
>> > >>>> Sereda)
>> > >>>>
>> > >>>> Julian
>> > >>>>
>> > >>>>
>> > >>>>> On Jul 30, 2018, at 1:13 PM, Michael Mior <mmior@xxxxxxxxxx>
>> wrote:
>> > >>>>>
>> > >>>>> Thanks Andrei for digging into this! Since I haven't reproduced
>> this
>> > >>>>> failure myself, if someone else could check out the PR, that
>> would be
>> > >>>>> great.
>> > >>>>> --
>> > >>>>> Michael Mior
>> > >>>>> mmior@xxxxxxxxxx
>> > >>>>>
>> > >>>>>
>> > >>>>>
>> > >>>>> Le lun. 30 juil. 2018 à 13:29, Andrei Sereda <andrei@xxxxxxxxx> a
>> > >> écrit
>> > >>>> :
>> > >>>>>
>> > >>>>>> Agree, flaky tests are pretty annoying. I'll try to watch more
>> > >> carefully
>> > >>>>>> new "embedded data-source" issues (fongo, ES, cassandra). They
>> > >>>> introduced
>> > >>>>>> more "non-determinism" because they now run as part of regular
>> build
>> > >>>> which
>> > >>>>>> means executed much more often than IT.
>> > >>>>>>
>> > >>>>>> Last commit was for a more deterministic issue (incorrect
>> parsing of
>> > >> new
>> > >>>>>> java version format <http://openjdk.java.net/jeps/223>).
>> > >>>>>>
>> > >>>>>>
>> > >>>>>> On Mon, Jul 30, 2018 at 12:58 PM Julian Hyde <jhyde@xxxxxxxxxx>
>> > >> wrote:
>> > >>>>>>
>> > >>>>>>> On balance, I don’t think we should back out
>> CassandraAdapterTest.
>> > >> But
>> > >>>> we
>> > >>>>>>> do need to continue working to make it more resilient. Any test
>> > that
>> > >>>>>>> generates too many false negatives over the long run should be
>> > >>>> disabled,
>> > >>>>>>> and this is no exception.
>> > >>>>>>>
>> > >>>>>>> And as the other test results show, it’s not the only flaky
>> part of
>> > >> the
>> > >>>>>>> test suite, when the machine is stressed. One thing we can do
>> is to
>> > >>>> avoid
>> > >>>>>>> timeouts whenever possible.
>> > >>>>>>>
>> > >>>>>>> Julian
>> > >>>>>>>
>> > >>>>>>>
>> > >>>>>>>> On Jul 30, 2018, at 9:30 AM, Andrei Sereda <andrei@xxxxxxxxx>
>> > >> wrote:
>> > >>>>>>>>
>> > >>>>>>>> Pls check and confirm that the following PR fixes the issue:
>> > >>>>>>>> https://github.com/apache/calcite/pull/770
>> > >>>>>>>> It addresses build failures due to version parsing.
>> > >>>>>>>>
>> > >>>>>>>> Stale folders / files will be addressed in a different PR.
>> > >>>>>>>>
>> > >>>>>>>>
>> > >>>>>>>> On Mon, Jul 30, 2018 at 10:10 AM Andrei Sereda
>> <andrei@xxxxxxxxx>
>> > >>>>>> wrote:
>> > >>>>>>>>
>> > >>>>>>>>> This must be something specific to Cassandra Unit. Will check
>> > >>>>>>>>>
>> > >>>>>>>>> On Mon, Jul 30, 2018, 08:59 Sergey Nuyanzin <
>> snuyanzin@xxxxxxxxx
>> > >
>> > >>>>>>> wrote:
>> > >>>>>>>>>
>> > >>>>>>>>>> There is one more strange thing (at least on Windows): while
>> > >>>>>> building a
>> > >>>>>>>>>> file with name ".toDelete" is generated under
>> calcite\cassandra
>> > >> and
>> > >>>>>>> it's
>> > >>>>>>>>>> not removed by the end of tests
>> > >>>>>>>>>> Is there a way to make cassandra generates these files in
>> target
>> > >>>>>>> directory
>> > >>>>>>>>>> e.g.?
>> > >>>>>>>>>>
>> > >>>>>>>>>>
>> > >>>>>>>>>>
>> > >>>>>>>>>> On Mon, Jul 30, 2018 at 3:52 PM, Andrei Sereda
>> <andrei@xxxxxxxxx
>> > >
>> > >>>>>>> wrote:
>> > >>>>>>>>>>
>> > >>>>>>>>>>> Most of the problems are during test init Phase. Most likely
>> > with
>> > >>>>>>>>>> version
>> > >>>>>>>>>>> string (eg. 11-ea for JDK11).
>> > >>>>>>>>>>> I'll fix that.
>> > >>>>>>>>>>>
>> > >>>>>>>>>>>
>> > >>>>>>>>>>> On Mon, Jul 30, 2018 at 8:16 AM Michael Mior <
>> mmior@xxxxxxxxxx
>> > >
>> > >>>>>>> wrote:
>> > >>>>>>>>>>>
>> > >>>>>>>>>>>> I tested myself a fair bit under Ubuntu before pushing this
>> > and
>> > >>>>>>> didn't
>> > >>>>>>>>>>> see
>> > >>>>>>>>>>>> any of these issues myself. That said, I agree that it's
>> > >> important
>> > >>>>>>> the
>> > >>>>>>>>>>> test
>> > >>>>>>>>>>>> suite be stable. I'm fine with reverting for now or (more
>> > >>>>>> preferably
>> > >>>>>>>>>> IMO)
>> > >>>>>>>>>>>> just disabling these tests by default.
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>> --
>> > >>>>>>>>>>>> Michael Mior
>> > >>>>>>>>>>>> mmior@xxxxxxxxxx
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>> Le lun. 30 juil. 2018 à 00:59, Julian Hyde <
>> jhyde@xxxxxxxxxx>
>> > a
>> > >>>>>>>>>> écrit :
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>>> I ran on several JDK versions, all on Ubuntu Linux. The
>> > machine
>> > >>>>>> was
>> > >>>>>>>>>>>>> fairly heavily loaded (I was copying one filesystem to
>> > another
>> > >> at
>> > >>>>>>>>>> the
>> > >>>>>>>>>>>>> time).
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>> Here's the error from OpenJDK10:
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>> [INFO] Running
>> org.apache.calcite.test.CassandraAdapterTest
>> > >>>>>>>>>>>>> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0,
>> > Time
>> > >>>>>>>>>>>>> elapsed: 0.001 s <<< FAILURE! - in
>> > >>>>>>>>>>>>> org.apache.calcite.test.CassandraAdapterTest
>> > >>>>>>>>>>>>> [ERROR] org.apache.calcite.test.CassandraAdapterTest  Time
>> > >>>>>> elapsed:
>> > >>>>>>>>>>>>> 0.001 s  <<< ERROR!
>> > >>>>>>>>>>>>> java.lang.ExceptionInInitializerError
>> > >>>>>>>>>>>>>     at
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>
>> > >>>>>>
>> org.apache.calcite.test.CassandraAdapterTest.initCassandraIfEnabled(
>> > >>>>>>>>>>> CassandraAdapterTest.java:106)
>> > >>>>>>>>>>>>>     at
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>> org.apache.calcite.test.CassandraAdapterTest.<clinit>(
>> > >>>>>>>>>>> CassandraAdapterTest.java:56)
>> > >>>>>>>>>>>>> Caused by: java.lang.StringIndexOutOfBoundsException:
>> begin
>> > 0,
>> > >>>> end
>> > >>>>>>>>>> -1,
>> > >>>>>>>>>>>>> length 5
>> > >>>>>>>>>>>>>     at
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>
>> > >>>>>>
>> org.apache.calcite.test.CassandraAdapterTest.initCassandraIfEnabled(
>> > >>>>>>>>>>> CassandraAdapterTest.java:106)
>> > >>>>>>>>>>>>>     at
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>> org.apache.calcite.test.CassandraAdapterTest.<clinit>(
>> > >>>>>>>>>>> CassandraAdapterTest.java:56)
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>> Here's the error in JDK 11:
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>> [INFO] Running
>> org.apache.calcite.test.CassandraAdapterTest
>> > >>>>>>>>>>>>> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0,
>> > Time
>> > >>>>>>>>>>>>> elapsed: 0 s <<< FAILURE! - in
>> > >>>>>>>>>>>>> org.apache.calcite.test.CassandraAdapterTest
>> > >>>>>>>>>>>>> [ERROR] org.apache.calcite.test.CassandraAdapterTest  Time
>> > >>>>>> elapsed:
>> > >>>>>>>>>> 0
>> > >>>>>>>>>>>>> s  <<< ERROR!
>> > >>>>>>>>>>>>> java.lang.ExceptionInInitializerError
>> > >>>>>>>>>>>>>     at
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>> org.apache.calcite.test.CassandraAdapterTest.enabled(
>> > >>>>>>>>>>> CassandraAdapterTest.java:81)
>> > >>>>>>>>>>>>>     at
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>
>> > >>>>>>
>> org.apache.calcite.test.CassandraAdapterTest.initCassandraIfEnabled(
>> > >>>>>>>>>>> CassandraAdapterTest.java:88)
>> > >>>>>>>>>>>>>     at
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>> org.apache.calcite.test.CassandraAdapterTest.<clinit>(
>> > >>>>>>>>>>> CassandraAdapterTest.java:56)
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>> Here's the failure from JDK 9:
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>> [INFO] Running
>> org.apache.calcite.test.CassandraAdapterTest
>> > >>>>>>>>>>>>> [ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0,
>> > Time
>> > >>>>>>>>>>>>> elapsed: 0.001 s <<< FAILURE! - in
>> > >>>>>>>>>>>>> org.apache.calcite.test.CassandraAdapterTest
>> > >>>>>>>>>>>>> [ERROR] org.apache.calcite.test.CassandraAdapterTest  Time
>> > >>>>>> elapsed:
>> > >>>>>>>>>>>>> 0.001 s  <<< FAILURE!
>> > >>>>>>>>>>>>> java.lang.AssertionError: Cassandra daemon did not start
>> > within
>> > >>>>>>>>>> timeout
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>> Here's the failure from JDK 10:
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>> [INFO] Running
>> > org.apache.calcite.jdbc.CalciteRemoteDriverTest
>> > >>>>>>>>>>>>> [ERROR] Tests run: 28, Failures: 1, Errors: 0, Skipped: 0,
>> > Time
>> > >>>>>>>>>>>>> elapsed: 0.642 s <<< FAILURE! - in
>> > >>>>>>>>>>>>> org.apache.calcite.jdbc.CalciteRemoteDriverTest
>> > >>>>>>>>>>>>> [ERROR]
>> > >>>>>>>>>>>>> testRemoteExecuteQuery(org.apache.calcite.jdbc.
>> > >>>>>>>>>>> CalciteRemoteDriverTest)
>> > >>>>>>>>>>>>> Time elapsed: 0.047 s  <<< FAILURE!
>> > >>>>>>>>>>>>> java.lang.AssertionError:
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>> Expected: "EXPR$0=1; EXPR$1=a\nEXPR$0=null; EXPR$1=b"
>> > >>>>>>>>>>>>>  but: was ""
>> > >>>>>>>>>>>>>     at
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>
>> > >>>>>>
>> > >>
>> org.apache.calcite.jdbc.CalciteRemoteDriverTest.testRemoteExecuteQuery(
>> > >>>>>>>>>>> CalciteRemoteDriverTest.java:280)
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>> In conclusion: It's worrying that the suite shows 4
>> different
>> > >>>>>> cracks
>> > >>>>>>>>>>>>> under 4 different JDKs. Clearly the load on my machine was
>> > >> making
>> > >>>>>>>>>>>>> problems worse, and granted, the problems are just testing
>> > >>>>>> problems,
>> > >>>>>>>>>>>>> not real bugs. But flaky test suites waste time and
>> effort.
>> > >> There
>> > >>>>>>>>>> are
>> > >>>>>>>>>>>>> indications that the new embedded Cassandra test is more
>> > flaky
>> > >>>>>> than
>> > >>>>>>>>>>>>> most.
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>> Julian
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>> On Sun, Jul 29, 2018 at 11:33 AM, Andrei Sereda
>> > >> <andrei@xxxxxxxxx
>> > >>>>>>>
>> > >>>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>>> What version of java / OS / maven do you have ? What is
>> your
>> > >>>>>> maven
>> > >>>>>>>>>>>>> command
>> > >>>>>>>>>>>>>> ?
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> Things pass for me on MacOS X java 8,9 and 10.
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> On Sun, Jul 29, 2018 at 2:05 PM Julian Hyde <
>> > jhyde@xxxxxxxxxx
>> > >>>
>> > >>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>> I'm seeing the following error when I run the tests on
>> > >>>> bd0e14002
>> > >>>>>>>>>>>>>>> origin/master. Anyone else see it?
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>> [INFO] Running
>> org.apache.calcite.test.CassandraAdapterTest
>> > >>>>>>>>>>>>>>> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped:
>> 0,
>> > >> Time
>> > >>>>>>>>>>>>>>> elapsed: 0.001 s <<< FAILURE! - in
>> > >>>>>>>>>>>>>>> org.apache.calcite.test.CassandraAdapterTest
>> > >>>>>>>>>>>>>>> [ERROR] org.apache.calcite.test.CassandraAdapterTest
>> Time
>> > >>>>>>>>>> elapsed:
>> > >>>>>>>>>>>>>>> 0.001 s  <<< ERROR!
>> > >>>>>>>>>>>>>>> java.lang.ExceptionInInitializerError
>> > >>>>>>>>>>>>>>>     at
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>
>> > >>>>>>
>> org.apache.calcite.test.CassandraAdapterTest.initCassandraIfEnabled(
>> > >>>>>>>>>>> CassandraAdapterTest.java:106)
>> > >>>>>>>>>>>>>>>     at
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>> org.apache.calcite.test.CassandraAdapterTest.<clinit>(
>> > >>>>>>>>>>> CassandraAdapterTest.java:56)
>> > >>>>>>>>>>>>>>> Caused by: java.lang.StringIndexOutOfBoundsException:
>> begin
>> > >> 0,
>> > >>>>>>>>>> end
>> > >>>>>>>>>>> -1,
>> > >>>>>>>>>>>>>>> length 5
>> > >>>>>>>>>>>>>>>     at
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>
>> > >>>>>>
>> org.apache.calcite.test.CassandraAdapterTest.initCassandraIfEnabled(
>> > >>>>>>>>>>> CassandraAdapterTest.java:106)
>> > >>>>>>>>>>>>>>>     at
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>> org.apache.calcite.test.CassandraAdapterTest.<clinit>(
>> > >>>>>>>>>>> CassandraAdapterTest.java:56)
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>
>> > >>>>>>>>>>
>> > >>>>>>>>>>
>> > >>>>>>>>>>
>> > >>>>>>>>>> --
>> > >>>>>>>>>> Best regards,
>> > >>>>>>>>>> Sergey
>> > >>>>>>>>>>
>> > >>>>>>>>>
>> > >>>>>>>
>> > >>>>>>>
>> > >>>>>>
>> > >>>>
>> > >>>>
>> > >>
>> > >>
>> >
>> >
>>
>