osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: CassandraAdapterTest failure


Thanks Andrei for digging into this! Since I haven't reproduced this
failure myself, if someone else could check out the PR, that would be
great.
--
Michael Mior
mmior@xxxxxxxxxx



Le lun. 30 juil. 2018 à 13:29, Andrei Sereda <andrei@xxxxxxxxx> a écrit :

> Agree, flaky tests are pretty annoying. I'll try to watch more carefully
> new "embedded data-source" issues (fongo, ES, cassandra). They introduced
> more "non-determinism" because they now run as part of regular build which
> means executed much more often than IT.
>
> Last commit was for a more deterministic issue (incorrect parsing of new
> java version format <http://openjdk.java.net/jeps/223>).
>
>
> On Mon, Jul 30, 2018 at 12:58 PM Julian Hyde <jhyde@xxxxxxxxxx> wrote:
>
> > On balance, I don’t think we should back out CassandraAdapterTest. But we
> > do need to continue working to make it more resilient. Any test that
> > generates too many false negatives over the long run should be disabled,
> > and this is no exception.
> >
> > And as the other test results show, it’s not the only flaky part of the
> > test suite, when the machine is stressed. One thing we can do is to avoid
> > timeouts whenever possible.
> >
> > Julian
> >
> >
> > > On Jul 30, 2018, at 9:30 AM, Andrei Sereda <andrei@xxxxxxxxx> wrote:
> > >
> > > Pls check and confirm that the following PR fixes the issue:
> > > https://github.com/apache/calcite/pull/770
> > > It addresses build failures due to version parsing.
> > >
> > > Stale folders / files will be addressed in a different PR.
> > >
> > >
> > > On Mon, Jul 30, 2018 at 10:10 AM Andrei Sereda <andrei@xxxxxxxxx>
> wrote:
> > >
> > >> This must be something specific to Cassandra Unit. Will check
> > >>
> > >> On Mon, Jul 30, 2018, 08:59 Sergey Nuyanzin <snuyanzin@xxxxxxxxx>
> > wrote:
> > >>
> > >>> There is one more strange thing (at least on Windows): while
> building a
> > >>> file with name ".toDelete" is generated under calcite\cassandra and
> > it's
> > >>> not removed by the end of tests
> > >>> Is there a way to make cassandra generates these files in target
> > directory
> > >>> e.g.?
> > >>>
> > >>>
> > >>>
> > >>> On Mon, Jul 30, 2018 at 3:52 PM, Andrei Sereda <andrei@xxxxxxxxx>
> > wrote:
> > >>>
> > >>>> Most of the problems are during test init Phase. Most likely with
> > >>> version
> > >>>> string (eg. 11-ea for JDK11).
> > >>>> I'll fix that.
> > >>>>
> > >>>>
> > >>>> On Mon, Jul 30, 2018 at 8:16 AM Michael Mior <mmior@xxxxxxxxxx>
> > wrote:
> > >>>>
> > >>>>> I tested myself a fair bit under Ubuntu before pushing this and
> > didn't
> > >>>> see
> > >>>>> any of these issues myself. That said, I agree that it's important
> > the
> > >>>> test
> > >>>>> suite be stable. I'm fine with reverting for now or (more
> preferably
> > >>> IMO)
> > >>>>> just disabling these tests by default.
> > >>>>>
> > >>>>> --
> > >>>>> Michael Mior
> > >>>>> mmior@xxxxxxxxxx
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> Le lun. 30 juil. 2018 à 00:59, Julian Hyde <jhyde@xxxxxxxxxx> a
> > >>> écrit :
> > >>>>>
> > >>>>>> I ran on several JDK versions, all on Ubuntu Linux. The machine
> was
> > >>>>>> fairly heavily loaded (I was copying one filesystem to another at
> > >>> the
> > >>>>>> time).
> > >>>>>>
> > >>>>>> Here's the error from OpenJDK10:
> > >>>>>>
> > >>>>>> [INFO] Running org.apache.calcite.test.CassandraAdapterTest
> > >>>>>> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time
> > >>>>>> elapsed: 0.001 s <<< FAILURE! - in
> > >>>>>> org.apache.calcite.test.CassandraAdapterTest
> > >>>>>> [ERROR] org.apache.calcite.test.CassandraAdapterTest  Time
> elapsed:
> > >>>>>> 0.001 s  <<< ERROR!
> > >>>>>> java.lang.ExceptionInInitializerError
> > >>>>>>        at
> > >>>>>>
> > >>>>>
> org.apache.calcite.test.CassandraAdapterTest.initCassandraIfEnabled(
> > >>>> CassandraAdapterTest.java:106)
> > >>>>>>        at
> > >>>>>>
> > >>>>> org.apache.calcite.test.CassandraAdapterTest.<clinit>(
> > >>>> CassandraAdapterTest.java:56)
> > >>>>>> Caused by: java.lang.StringIndexOutOfBoundsException: begin 0, end
> > >>> -1,
> > >>>>>> length 5
> > >>>>>>        at
> > >>>>>>
> > >>>>>
> org.apache.calcite.test.CassandraAdapterTest.initCassandraIfEnabled(
> > >>>> CassandraAdapterTest.java:106)
> > >>>>>>        at
> > >>>>>>
> > >>>>> org.apache.calcite.test.CassandraAdapterTest.<clinit>(
> > >>>> CassandraAdapterTest.java:56)
> > >>>>>>
> > >>>>>> Here's the error in JDK 11:
> > >>>>>>
> > >>>>>> [INFO] Running org.apache.calcite.test.CassandraAdapterTest
> > >>>>>> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time
> > >>>>>> elapsed: 0 s <<< FAILURE! - in
> > >>>>>> org.apache.calcite.test.CassandraAdapterTest
> > >>>>>> [ERROR] org.apache.calcite.test.CassandraAdapterTest  Time
> elapsed:
> > >>> 0
> > >>>>>> s  <<< ERROR!
> > >>>>>> java.lang.ExceptionInInitializerError
> > >>>>>>        at
> > >>>>>>
> > >>>>> org.apache.calcite.test.CassandraAdapterTest.enabled(
> > >>>> CassandraAdapterTest.java:81)
> > >>>>>>        at
> > >>>>>>
> > >>>>>
> org.apache.calcite.test.CassandraAdapterTest.initCassandraIfEnabled(
> > >>>> CassandraAdapterTest.java:88)
> > >>>>>>        at
> > >>>>>>
> > >>>>> org.apache.calcite.test.CassandraAdapterTest.<clinit>(
> > >>>> CassandraAdapterTest.java:56)
> > >>>>>>
> > >>>>>> Here's the failure from JDK 9:
> > >>>>>>
> > >>>>>> [INFO] Running org.apache.calcite.test.CassandraAdapterTest
> > >>>>>> [ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time
> > >>>>>> elapsed: 0.001 s <<< FAILURE! - in
> > >>>>>> org.apache.calcite.test.CassandraAdapterTest
> > >>>>>> [ERROR] org.apache.calcite.test.CassandraAdapterTest  Time
> elapsed:
> > >>>>>> 0.001 s  <<< FAILURE!
> > >>>>>> java.lang.AssertionError: Cassandra daemon did not start within
> > >>> timeout
> > >>>>>>
> > >>>>>> Here's the failure from JDK 10:
> > >>>>>>
> > >>>>>> [INFO] Running org.apache.calcite.jdbc.CalciteRemoteDriverTest
> > >>>>>> [ERROR] Tests run: 28, Failures: 1, Errors: 0, Skipped: 0, Time
> > >>>>>> elapsed: 0.642 s <<< FAILURE! - in
> > >>>>>> org.apache.calcite.jdbc.CalciteRemoteDriverTest
> > >>>>>> [ERROR]
> > >>>>>> testRemoteExecuteQuery(org.apache.calcite.jdbc.
> > >>>> CalciteRemoteDriverTest)
> > >>>>>> Time elapsed: 0.047 s  <<< FAILURE!
> > >>>>>> java.lang.AssertionError:
> > >>>>>>
> > >>>>>> Expected: "EXPR$0=1; EXPR$1=a\nEXPR$0=null; EXPR$1=b"
> > >>>>>>     but: was ""
> > >>>>>>        at
> > >>>>>>
> > >>>>>
> > >>>
> org.apache.calcite.jdbc.CalciteRemoteDriverTest.testRemoteExecuteQuery(
> > >>>> CalciteRemoteDriverTest.java:280)
> > >>>>>>
> > >>>>>> In conclusion: It's worrying that the suite shows 4 different
> cracks
> > >>>>>> under 4 different JDKs. Clearly the load on my machine was making
> > >>>>>> problems worse, and granted, the problems are just testing
> problems,
> > >>>>>> not real bugs. But flaky test suites waste time and effort. There
> > >>> are
> > >>>>>> indications that the new embedded Cassandra test is more flaky
> than
> > >>>>>> most.
> > >>>>>>
> > >>>>>> Julian
> > >>>>>>
> > >>>>>>
> > >>>>>> On Sun, Jul 29, 2018 at 11:33 AM, Andrei Sereda <andrei@xxxxxxxxx
> >
> > >>>>> wrote:
> > >>>>>>> What version of java / OS / maven do you have ? What is your
> maven
> > >>>>>> command
> > >>>>>>> ?
> > >>>>>>>
> > >>>>>>> Things pass for me on MacOS X java 8,9 and 10.
> > >>>>>>>
> > >>>>>>> On Sun, Jul 29, 2018 at 2:05 PM Julian Hyde <jhyde@xxxxxxxxxx>
> > >>>> wrote:
> > >>>>>>>
> > >>>>>>>> I'm seeing the following error when I run the tests on bd0e14002
> > >>>>>>>> origin/master. Anyone else see it?
> > >>>>>>>>
> > >>>>>>>> [INFO] Running org.apache.calcite.test.CassandraAdapterTest
> > >>>>>>>> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time
> > >>>>>>>> elapsed: 0.001 s <<< FAILURE! - in
> > >>>>>>>> org.apache.calcite.test.CassandraAdapterTest
> > >>>>>>>> [ERROR] org.apache.calcite.test.CassandraAdapterTest  Time
> > >>> elapsed:
> > >>>>>>>> 0.001 s  <<< ERROR!
> > >>>>>>>> java.lang.ExceptionInInitializerError
> > >>>>>>>>        at
> > >>>>>>>>
> > >>>>>>
> > >>>>>
> org.apache.calcite.test.CassandraAdapterTest.initCassandraIfEnabled(
> > >>>> CassandraAdapterTest.java:106)
> > >>>>>>>>        at
> > >>>>>>>>
> > >>>>>>
> > >>>>> org.apache.calcite.test.CassandraAdapterTest.<clinit>(
> > >>>> CassandraAdapterTest.java:56)
> > >>>>>>>> Caused by: java.lang.StringIndexOutOfBoundsException: begin 0,
> > >>> end
> > >>>> -1,
> > >>>>>>>> length 5
> > >>>>>>>>        at
> > >>>>>>>>
> > >>>>>>
> > >>>>>
> org.apache.calcite.test.CassandraAdapterTest.initCassandraIfEnabled(
> > >>>> CassandraAdapterTest.java:106)
> > >>>>>>>>        at
> > >>>>>>>>
> > >>>>>>
> > >>>>> org.apache.calcite.test.CassandraAdapterTest.<clinit>(
> > >>>> CassandraAdapterTest.java:56)
> > >>>>>>>>
> > >>>>>>
> > >>>>>
> > >>>>
> > >>>
> > >>>
> > >>>
> > >>> --
> > >>> Best regards,
> > >>> Sergey
> > >>>
> > >>
> >
> >
>