osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: CassandraAdapterTest failure


The test is failing every time for me on JDK 10.  The command “mvn -Pit clean test” will probably reproduce it for most people.

Can other folks please try to reproduce this? I’m getting close to saying that we should back this change out even though apparently only I can reproduce the failure.

Julian

 




> On Jul 30, 2018, at 1:35 PM, Andrei Sereda <andrei@xxxxxxxxx> wrote:
> 
> Julian, I have amended the commit message in f0b00f0c
> 
> 
> On Mon, Jul 30, 2018 at 4:22 PM Julian Hyde <jhyde@xxxxxxxxxx> wrote:
> 
>> I’m running tests on the PR now.
>> 
>> Since commit comments are a soap-box of mine, I will remark that rather
>> than
>> 
>>  [CALCITE-2428] Fix cassandra unit test initialization. (Andrei Sereda)
>> 
>> the commit comment should be
>> 
>>  [CALCITE-2428] Cassandra unit test fails to parse version string (Andrei
>> Sereda)
>> 
>> Julian
>> 
>> 
>>> On Jul 30, 2018, at 1:13 PM, Michael Mior <mmior@xxxxxxxxxx> wrote:
>>> 
>>> Thanks Andrei for digging into this! Since I haven't reproduced this
>>> failure myself, if someone else could check out the PR, that would be
>>> great.
>>> --
>>> Michael Mior
>>> mmior@xxxxxxxxxx
>>> 
>>> 
>>> 
>>> Le lun. 30 juil. 2018 à 13:29, Andrei Sereda <andrei@xxxxxxxxx> a écrit
>> :
>>> 
>>>> Agree, flaky tests are pretty annoying. I'll try to watch more carefully
>>>> new "embedded data-source" issues (fongo, ES, cassandra). They
>> introduced
>>>> more "non-determinism" because they now run as part of regular build
>> which
>>>> means executed much more often than IT.
>>>> 
>>>> Last commit was for a more deterministic issue (incorrect parsing of new
>>>> java version format <http://openjdk.java.net/jeps/223>).
>>>> 
>>>> 
>>>> On Mon, Jul 30, 2018 at 12:58 PM Julian Hyde <jhyde@xxxxxxxxxx> wrote:
>>>> 
>>>>> On balance, I don’t think we should back out CassandraAdapterTest. But
>> we
>>>>> do need to continue working to make it more resilient. Any test that
>>>>> generates too many false negatives over the long run should be
>> disabled,
>>>>> and this is no exception.
>>>>> 
>>>>> And as the other test results show, it’s not the only flaky part of the
>>>>> test suite, when the machine is stressed. One thing we can do is to
>> avoid
>>>>> timeouts whenever possible.
>>>>> 
>>>>> Julian
>>>>> 
>>>>> 
>>>>>> On Jul 30, 2018, at 9:30 AM, Andrei Sereda <andrei@xxxxxxxxx> wrote:
>>>>>> 
>>>>>> Pls check and confirm that the following PR fixes the issue:
>>>>>> https://github.com/apache/calcite/pull/770
>>>>>> It addresses build failures due to version parsing.
>>>>>> 
>>>>>> Stale folders / files will be addressed in a different PR.
>>>>>> 
>>>>>> 
>>>>>> On Mon, Jul 30, 2018 at 10:10 AM Andrei Sereda <andrei@xxxxxxxxx>
>>>> wrote:
>>>>>> 
>>>>>>> This must be something specific to Cassandra Unit. Will check
>>>>>>> 
>>>>>>> On Mon, Jul 30, 2018, 08:59 Sergey Nuyanzin <snuyanzin@xxxxxxxxx>
>>>>> wrote:
>>>>>>> 
>>>>>>>> There is one more strange thing (at least on Windows): while
>>>> building a
>>>>>>>> file with name ".toDelete" is generated under calcite\cassandra and
>>>>> it's
>>>>>>>> not removed by the end of tests
>>>>>>>> Is there a way to make cassandra generates these files in target
>>>>> directory
>>>>>>>> e.g.?
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On Mon, Jul 30, 2018 at 3:52 PM, Andrei Sereda <andrei@xxxxxxxxx>
>>>>> wrote:
>>>>>>>> 
>>>>>>>>> Most of the problems are during test init Phase. Most likely with
>>>>>>>> version
>>>>>>>>> string (eg. 11-ea for JDK11).
>>>>>>>>> I'll fix that.
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> On Mon, Jul 30, 2018 at 8:16 AM Michael Mior <mmior@xxxxxxxxxx>
>>>>> wrote:
>>>>>>>>> 
>>>>>>>>>> I tested myself a fair bit under Ubuntu before pushing this and
>>>>> didn't
>>>>>>>>> see
>>>>>>>>>> any of these issues myself. That said, I agree that it's important
>>>>> the
>>>>>>>>> test
>>>>>>>>>> suite be stable. I'm fine with reverting for now or (more
>>>> preferably
>>>>>>>> IMO)
>>>>>>>>>> just disabling these tests by default.
>>>>>>>>>> 
>>>>>>>>>> --
>>>>>>>>>> Michael Mior
>>>>>>>>>> mmior@xxxxxxxxxx
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> Le lun. 30 juil. 2018 à 00:59, Julian Hyde <jhyde@xxxxxxxxxx> a
>>>>>>>> écrit :
>>>>>>>>>> 
>>>>>>>>>>> I ran on several JDK versions, all on Ubuntu Linux. The machine
>>>> was
>>>>>>>>>>> fairly heavily loaded (I was copying one filesystem to another at
>>>>>>>> the
>>>>>>>>>>> time).
>>>>>>>>>>> 
>>>>>>>>>>> Here's the error from OpenJDK10:
>>>>>>>>>>> 
>>>>>>>>>>> [INFO] Running org.apache.calcite.test.CassandraAdapterTest
>>>>>>>>>>> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time
>>>>>>>>>>> elapsed: 0.001 s <<< FAILURE! - in
>>>>>>>>>>> org.apache.calcite.test.CassandraAdapterTest
>>>>>>>>>>> [ERROR] org.apache.calcite.test.CassandraAdapterTest  Time
>>>> elapsed:
>>>>>>>>>>> 0.001 s  <<< ERROR!
>>>>>>>>>>> java.lang.ExceptionInInitializerError
>>>>>>>>>>>      at
>>>>>>>>>>> 
>>>>>>>>>> 
>>>> org.apache.calcite.test.CassandraAdapterTest.initCassandraIfEnabled(
>>>>>>>>> CassandraAdapterTest.java:106)
>>>>>>>>>>>      at
>>>>>>>>>>> 
>>>>>>>>>> org.apache.calcite.test.CassandraAdapterTest.<clinit>(
>>>>>>>>> CassandraAdapterTest.java:56)
>>>>>>>>>>> Caused by: java.lang.StringIndexOutOfBoundsException: begin 0,
>> end
>>>>>>>> -1,
>>>>>>>>>>> length 5
>>>>>>>>>>>      at
>>>>>>>>>>> 
>>>>>>>>>> 
>>>> org.apache.calcite.test.CassandraAdapterTest.initCassandraIfEnabled(
>>>>>>>>> CassandraAdapterTest.java:106)
>>>>>>>>>>>      at
>>>>>>>>>>> 
>>>>>>>>>> org.apache.calcite.test.CassandraAdapterTest.<clinit>(
>>>>>>>>> CassandraAdapterTest.java:56)
>>>>>>>>>>> 
>>>>>>>>>>> Here's the error in JDK 11:
>>>>>>>>>>> 
>>>>>>>>>>> [INFO] Running org.apache.calcite.test.CassandraAdapterTest
>>>>>>>>>>> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time
>>>>>>>>>>> elapsed: 0 s <<< FAILURE! - in
>>>>>>>>>>> org.apache.calcite.test.CassandraAdapterTest
>>>>>>>>>>> [ERROR] org.apache.calcite.test.CassandraAdapterTest  Time
>>>> elapsed:
>>>>>>>> 0
>>>>>>>>>>> s  <<< ERROR!
>>>>>>>>>>> java.lang.ExceptionInInitializerError
>>>>>>>>>>>      at
>>>>>>>>>>> 
>>>>>>>>>> org.apache.calcite.test.CassandraAdapterTest.enabled(
>>>>>>>>> CassandraAdapterTest.java:81)
>>>>>>>>>>>      at
>>>>>>>>>>> 
>>>>>>>>>> 
>>>> org.apache.calcite.test.CassandraAdapterTest.initCassandraIfEnabled(
>>>>>>>>> CassandraAdapterTest.java:88)
>>>>>>>>>>>      at
>>>>>>>>>>> 
>>>>>>>>>> org.apache.calcite.test.CassandraAdapterTest.<clinit>(
>>>>>>>>> CassandraAdapterTest.java:56)
>>>>>>>>>>> 
>>>>>>>>>>> Here's the failure from JDK 9:
>>>>>>>>>>> 
>>>>>>>>>>> [INFO] Running org.apache.calcite.test.CassandraAdapterTest
>>>>>>>>>>> [ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time
>>>>>>>>>>> elapsed: 0.001 s <<< FAILURE! - in
>>>>>>>>>>> org.apache.calcite.test.CassandraAdapterTest
>>>>>>>>>>> [ERROR] org.apache.calcite.test.CassandraAdapterTest  Time
>>>> elapsed:
>>>>>>>>>>> 0.001 s  <<< FAILURE!
>>>>>>>>>>> java.lang.AssertionError: Cassandra daemon did not start within
>>>>>>>> timeout
>>>>>>>>>>> 
>>>>>>>>>>> Here's the failure from JDK 10:
>>>>>>>>>>> 
>>>>>>>>>>> [INFO] Running org.apache.calcite.jdbc.CalciteRemoteDriverTest
>>>>>>>>>>> [ERROR] Tests run: 28, Failures: 1, Errors: 0, Skipped: 0, Time
>>>>>>>>>>> elapsed: 0.642 s <<< FAILURE! - in
>>>>>>>>>>> org.apache.calcite.jdbc.CalciteRemoteDriverTest
>>>>>>>>>>> [ERROR]
>>>>>>>>>>> testRemoteExecuteQuery(org.apache.calcite.jdbc.
>>>>>>>>> CalciteRemoteDriverTest)
>>>>>>>>>>> Time elapsed: 0.047 s  <<< FAILURE!
>>>>>>>>>>> java.lang.AssertionError:
>>>>>>>>>>> 
>>>>>>>>>>> Expected: "EXPR$0=1; EXPR$1=a\nEXPR$0=null; EXPR$1=b"
>>>>>>>>>>>   but: was ""
>>>>>>>>>>>      at
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>> 
>>>> org.apache.calcite.jdbc.CalciteRemoteDriverTest.testRemoteExecuteQuery(
>>>>>>>>> CalciteRemoteDriverTest.java:280)
>>>>>>>>>>> 
>>>>>>>>>>> In conclusion: It's worrying that the suite shows 4 different
>>>> cracks
>>>>>>>>>>> under 4 different JDKs. Clearly the load on my machine was making
>>>>>>>>>>> problems worse, and granted, the problems are just testing
>>>> problems,
>>>>>>>>>>> not real bugs. But flaky test suites waste time and effort. There
>>>>>>>> are
>>>>>>>>>>> indications that the new embedded Cassandra test is more flaky
>>>> than
>>>>>>>>>>> most.
>>>>>>>>>>> 
>>>>>>>>>>> Julian
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> On Sun, Jul 29, 2018 at 11:33 AM, Andrei Sereda <andrei@xxxxxxxxx
>>>>> 
>>>>>>>>>> wrote:
>>>>>>>>>>>> What version of java / OS / maven do you have ? What is your
>>>> maven
>>>>>>>>>>> command
>>>>>>>>>>>> ?
>>>>>>>>>>>> 
>>>>>>>>>>>> Things pass for me on MacOS X java 8,9 and 10.
>>>>>>>>>>>> 
>>>>>>>>>>>> On Sun, Jul 29, 2018 at 2:05 PM Julian Hyde <jhyde@xxxxxxxxxx>
>>>>>>>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>>> I'm seeing the following error when I run the tests on
>> bd0e14002
>>>>>>>>>>>>> origin/master. Anyone else see it?
>>>>>>>>>>>>> 
>>>>>>>>>>>>> [INFO] Running org.apache.calcite.test.CassandraAdapterTest
>>>>>>>>>>>>> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time
>>>>>>>>>>>>> elapsed: 0.001 s <<< FAILURE! - in
>>>>>>>>>>>>> org.apache.calcite.test.CassandraAdapterTest
>>>>>>>>>>>>> [ERROR] org.apache.calcite.test.CassandraAdapterTest  Time
>>>>>>>> elapsed:
>>>>>>>>>>>>> 0.001 s  <<< ERROR!
>>>>>>>>>>>>> java.lang.ExceptionInInitializerError
>>>>>>>>>>>>>      at
>>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> 
>>>> org.apache.calcite.test.CassandraAdapterTest.initCassandraIfEnabled(
>>>>>>>>> CassandraAdapterTest.java:106)
>>>>>>>>>>>>>      at
>>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> org.apache.calcite.test.CassandraAdapterTest.<clinit>(
>>>>>>>>> CassandraAdapterTest.java:56)
>>>>>>>>>>>>> Caused by: java.lang.StringIndexOutOfBoundsException: begin 0,
>>>>>>>> end
>>>>>>>>> -1,
>>>>>>>>>>>>> length 5
>>>>>>>>>>>>>      at
>>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> 
>>>> org.apache.calcite.test.CassandraAdapterTest.initCassandraIfEnabled(
>>>>>>>>> CassandraAdapterTest.java:106)
>>>>>>>>>>>>>      at
>>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> org.apache.calcite.test.CassandraAdapterTest.<clinit>(
>>>>>>>>> CassandraAdapterTest.java:56)
>>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> --
>>>>>>>> Best regards,
>>>>>>>> Sergey
>>>>>>>> 
>>>>>>> 
>>>>> 
>>>>> 
>>>> 
>> 
>>