osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: CassandraAdapterTest failure


Julian, I have amended the commit message in f0b00f0c


On Mon, Jul 30, 2018 at 4:22 PM Julian Hyde <jhyde@xxxxxxxxxx> wrote:

> I’m running tests on the PR now.
>
> Since commit comments are a soap-box of mine, I will remark that rather
> than
>
>   [CALCITE-2428] Fix cassandra unit test initialization. (Andrei Sereda)
>
> the commit comment should be
>
>   [CALCITE-2428] Cassandra unit test fails to parse version string (Andrei
> Sereda)
>
> Julian
>
>
> > On Jul 30, 2018, at 1:13 PM, Michael Mior <mmior@xxxxxxxxxx> wrote:
> >
> > Thanks Andrei for digging into this! Since I haven't reproduced this
> > failure myself, if someone else could check out the PR, that would be
> > great.
> > --
> > Michael Mior
> > mmior@xxxxxxxxxx
> >
> >
> >
> > Le lun. 30 juil. 2018 à 13:29, Andrei Sereda <andrei@xxxxxxxxx> a écrit
> :
> >
> >> Agree, flaky tests are pretty annoying. I'll try to watch more carefully
> >> new "embedded data-source" issues (fongo, ES, cassandra). They
> introduced
> >> more "non-determinism" because they now run as part of regular build
> which
> >> means executed much more often than IT.
> >>
> >> Last commit was for a more deterministic issue (incorrect parsing of new
> >> java version format <http://openjdk.java.net/jeps/223>).
> >>
> >>
> >> On Mon, Jul 30, 2018 at 12:58 PM Julian Hyde <jhyde@xxxxxxxxxx> wrote:
> >>
> >>> On balance, I don’t think we should back out CassandraAdapterTest. But
> we
> >>> do need to continue working to make it more resilient. Any test that
> >>> generates too many false negatives over the long run should be
> disabled,
> >>> and this is no exception.
> >>>
> >>> And as the other test results show, it’s not the only flaky part of the
> >>> test suite, when the machine is stressed. One thing we can do is to
> avoid
> >>> timeouts whenever possible.
> >>>
> >>> Julian
> >>>
> >>>
> >>>> On Jul 30, 2018, at 9:30 AM, Andrei Sereda <andrei@xxxxxxxxx> wrote:
> >>>>
> >>>> Pls check and confirm that the following PR fixes the issue:
> >>>> https://github.com/apache/calcite/pull/770
> >>>> It addresses build failures due to version parsing.
> >>>>
> >>>> Stale folders / files will be addressed in a different PR.
> >>>>
> >>>>
> >>>> On Mon, Jul 30, 2018 at 10:10 AM Andrei Sereda <andrei@xxxxxxxxx>
> >> wrote:
> >>>>
> >>>>> This must be something specific to Cassandra Unit. Will check
> >>>>>
> >>>>> On Mon, Jul 30, 2018, 08:59 Sergey Nuyanzin <snuyanzin@xxxxxxxxx>
> >>> wrote:
> >>>>>
> >>>>>> There is one more strange thing (at least on Windows): while
> >> building a
> >>>>>> file with name ".toDelete" is generated under calcite\cassandra and
> >>> it's
> >>>>>> not removed by the end of tests
> >>>>>> Is there a way to make cassandra generates these files in target
> >>> directory
> >>>>>> e.g.?
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> On Mon, Jul 30, 2018 at 3:52 PM, Andrei Sereda <andrei@xxxxxxxxx>
> >>> wrote:
> >>>>>>
> >>>>>>> Most of the problems are during test init Phase. Most likely with
> >>>>>> version
> >>>>>>> string (eg. 11-ea for JDK11).
> >>>>>>> I'll fix that.
> >>>>>>>
> >>>>>>>
> >>>>>>> On Mon, Jul 30, 2018 at 8:16 AM Michael Mior <mmior@xxxxxxxxxx>
> >>> wrote:
> >>>>>>>
> >>>>>>>> I tested myself a fair bit under Ubuntu before pushing this and
> >>> didn't
> >>>>>>> see
> >>>>>>>> any of these issues myself. That said, I agree that it's important
> >>> the
> >>>>>>> test
> >>>>>>>> suite be stable. I'm fine with reverting for now or (more
> >> preferably
> >>>>>> IMO)
> >>>>>>>> just disabling these tests by default.
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> Michael Mior
> >>>>>>>> mmior@xxxxxxxxxx
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> Le lun. 30 juil. 2018 à 00:59, Julian Hyde <jhyde@xxxxxxxxxx> a
> >>>>>> écrit :
> >>>>>>>>
> >>>>>>>>> I ran on several JDK versions, all on Ubuntu Linux. The machine
> >> was
> >>>>>>>>> fairly heavily loaded (I was copying one filesystem to another at
> >>>>>> the
> >>>>>>>>> time).
> >>>>>>>>>
> >>>>>>>>> Here's the error from OpenJDK10:
> >>>>>>>>>
> >>>>>>>>> [INFO] Running org.apache.calcite.test.CassandraAdapterTest
> >>>>>>>>> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time
> >>>>>>>>> elapsed: 0.001 s <<< FAILURE! - in
> >>>>>>>>> org.apache.calcite.test.CassandraAdapterTest
> >>>>>>>>> [ERROR] org.apache.calcite.test.CassandraAdapterTest  Time
> >> elapsed:
> >>>>>>>>> 0.001 s  <<< ERROR!
> >>>>>>>>> java.lang.ExceptionInInitializerError
> >>>>>>>>>       at
> >>>>>>>>>
> >>>>>>>>
> >> org.apache.calcite.test.CassandraAdapterTest.initCassandraIfEnabled(
> >>>>>>> CassandraAdapterTest.java:106)
> >>>>>>>>>       at
> >>>>>>>>>
> >>>>>>>> org.apache.calcite.test.CassandraAdapterTest.<clinit>(
> >>>>>>> CassandraAdapterTest.java:56)
> >>>>>>>>> Caused by: java.lang.StringIndexOutOfBoundsException: begin 0,
> end
> >>>>>> -1,
> >>>>>>>>> length 5
> >>>>>>>>>       at
> >>>>>>>>>
> >>>>>>>>
> >> org.apache.calcite.test.CassandraAdapterTest.initCassandraIfEnabled(
> >>>>>>> CassandraAdapterTest.java:106)
> >>>>>>>>>       at
> >>>>>>>>>
> >>>>>>>> org.apache.calcite.test.CassandraAdapterTest.<clinit>(
> >>>>>>> CassandraAdapterTest.java:56)
> >>>>>>>>>
> >>>>>>>>> Here's the error in JDK 11:
> >>>>>>>>>
> >>>>>>>>> [INFO] Running org.apache.calcite.test.CassandraAdapterTest
> >>>>>>>>> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time
> >>>>>>>>> elapsed: 0 s <<< FAILURE! - in
> >>>>>>>>> org.apache.calcite.test.CassandraAdapterTest
> >>>>>>>>> [ERROR] org.apache.calcite.test.CassandraAdapterTest  Time
> >> elapsed:
> >>>>>> 0
> >>>>>>>>> s  <<< ERROR!
> >>>>>>>>> java.lang.ExceptionInInitializerError
> >>>>>>>>>       at
> >>>>>>>>>
> >>>>>>>> org.apache.calcite.test.CassandraAdapterTest.enabled(
> >>>>>>> CassandraAdapterTest.java:81)
> >>>>>>>>>       at
> >>>>>>>>>
> >>>>>>>>
> >> org.apache.calcite.test.CassandraAdapterTest.initCassandraIfEnabled(
> >>>>>>> CassandraAdapterTest.java:88)
> >>>>>>>>>       at
> >>>>>>>>>
> >>>>>>>> org.apache.calcite.test.CassandraAdapterTest.<clinit>(
> >>>>>>> CassandraAdapterTest.java:56)
> >>>>>>>>>
> >>>>>>>>> Here's the failure from JDK 9:
> >>>>>>>>>
> >>>>>>>>> [INFO] Running org.apache.calcite.test.CassandraAdapterTest
> >>>>>>>>> [ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time
> >>>>>>>>> elapsed: 0.001 s <<< FAILURE! - in
> >>>>>>>>> org.apache.calcite.test.CassandraAdapterTest
> >>>>>>>>> [ERROR] org.apache.calcite.test.CassandraAdapterTest  Time
> >> elapsed:
> >>>>>>>>> 0.001 s  <<< FAILURE!
> >>>>>>>>> java.lang.AssertionError: Cassandra daemon did not start within
> >>>>>> timeout
> >>>>>>>>>
> >>>>>>>>> Here's the failure from JDK 10:
> >>>>>>>>>
> >>>>>>>>> [INFO] Running org.apache.calcite.jdbc.CalciteRemoteDriverTest
> >>>>>>>>> [ERROR] Tests run: 28, Failures: 1, Errors: 0, Skipped: 0, Time
> >>>>>>>>> elapsed: 0.642 s <<< FAILURE! - in
> >>>>>>>>> org.apache.calcite.jdbc.CalciteRemoteDriverTest
> >>>>>>>>> [ERROR]
> >>>>>>>>> testRemoteExecuteQuery(org.apache.calcite.jdbc.
> >>>>>>> CalciteRemoteDriverTest)
> >>>>>>>>> Time elapsed: 0.047 s  <<< FAILURE!
> >>>>>>>>> java.lang.AssertionError:
> >>>>>>>>>
> >>>>>>>>> Expected: "EXPR$0=1; EXPR$1=a\nEXPR$0=null; EXPR$1=b"
> >>>>>>>>>    but: was ""
> >>>>>>>>>       at
> >>>>>>>>>
> >>>>>>>>
> >>>>>>
> >> org.apache.calcite.jdbc.CalciteRemoteDriverTest.testRemoteExecuteQuery(
> >>>>>>> CalciteRemoteDriverTest.java:280)
> >>>>>>>>>
> >>>>>>>>> In conclusion: It's worrying that the suite shows 4 different
> >> cracks
> >>>>>>>>> under 4 different JDKs. Clearly the load on my machine was making
> >>>>>>>>> problems worse, and granted, the problems are just testing
> >> problems,
> >>>>>>>>> not real bugs. But flaky test suites waste time and effort. There
> >>>>>> are
> >>>>>>>>> indications that the new embedded Cassandra test is more flaky
> >> than
> >>>>>>>>> most.
> >>>>>>>>>
> >>>>>>>>> Julian
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Sun, Jul 29, 2018 at 11:33 AM, Andrei Sereda <andrei@xxxxxxxxx
> >>>
> >>>>>>>> wrote:
> >>>>>>>>>> What version of java / OS / maven do you have ? What is your
> >> maven
> >>>>>>>>> command
> >>>>>>>>>> ?
> >>>>>>>>>>
> >>>>>>>>>> Things pass for me on MacOS X java 8,9 and 10.
> >>>>>>>>>>
> >>>>>>>>>> On Sun, Jul 29, 2018 at 2:05 PM Julian Hyde <jhyde@xxxxxxxxxx>
> >>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> I'm seeing the following error when I run the tests on
> bd0e14002
> >>>>>>>>>>> origin/master. Anyone else see it?
> >>>>>>>>>>>
> >>>>>>>>>>> [INFO] Running org.apache.calcite.test.CassandraAdapterTest
> >>>>>>>>>>> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time
> >>>>>>>>>>> elapsed: 0.001 s <<< FAILURE! - in
> >>>>>>>>>>> org.apache.calcite.test.CassandraAdapterTest
> >>>>>>>>>>> [ERROR] org.apache.calcite.test.CassandraAdapterTest  Time
> >>>>>> elapsed:
> >>>>>>>>>>> 0.001 s  <<< ERROR!
> >>>>>>>>>>> java.lang.ExceptionInInitializerError
> >>>>>>>>>>>       at
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >> org.apache.calcite.test.CassandraAdapterTest.initCassandraIfEnabled(
> >>>>>>> CassandraAdapterTest.java:106)
> >>>>>>>>>>>       at
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>> org.apache.calcite.test.CassandraAdapterTest.<clinit>(
> >>>>>>> CassandraAdapterTest.java:56)
> >>>>>>>>>>> Caused by: java.lang.StringIndexOutOfBoundsException: begin 0,
> >>>>>> end
> >>>>>>> -1,
> >>>>>>>>>>> length 5
> >>>>>>>>>>>       at
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >> org.apache.calcite.test.CassandraAdapterTest.initCassandraIfEnabled(
> >>>>>>> CassandraAdapterTest.java:106)
> >>>>>>>>>>>       at
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>> org.apache.calcite.test.CassandraAdapterTest.<clinit>(
> >>>>>>> CassandraAdapterTest.java:56)
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> Best regards,
> >>>>>> Sergey
> >>>>>>
> >>>>>
> >>>
> >>>
> >>
>
>