osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: CassandraAdapterTest failure


I’m running tests on the PR now.

Since commit comments are a soap-box of mine, I will remark that rather than

  [CALCITE-2428] Fix cassandra unit test initialization. (Andrei Sereda)

the commit comment should be

  [CALCITE-2428] Cassandra unit test fails to parse version string (Andrei Sereda)

Julian


> On Jul 30, 2018, at 1:13 PM, Michael Mior <mmior@xxxxxxxxxx> wrote:
> 
> Thanks Andrei for digging into this! Since I haven't reproduced this
> failure myself, if someone else could check out the PR, that would be
> great.
> --
> Michael Mior
> mmior@xxxxxxxxxx
> 
> 
> 
> Le lun. 30 juil. 2018 à 13:29, Andrei Sereda <andrei@xxxxxxxxx> a écrit :
> 
>> Agree, flaky tests are pretty annoying. I'll try to watch more carefully
>> new "embedded data-source" issues (fongo, ES, cassandra). They introduced
>> more "non-determinism" because they now run as part of regular build which
>> means executed much more often than IT.
>> 
>> Last commit was for a more deterministic issue (incorrect parsing of new
>> java version format <http://openjdk.java.net/jeps/223>).
>> 
>> 
>> On Mon, Jul 30, 2018 at 12:58 PM Julian Hyde <jhyde@xxxxxxxxxx> wrote:
>> 
>>> On balance, I don’t think we should back out CassandraAdapterTest. But we
>>> do need to continue working to make it more resilient. Any test that
>>> generates too many false negatives over the long run should be disabled,
>>> and this is no exception.
>>> 
>>> And as the other test results show, it’s not the only flaky part of the
>>> test suite, when the machine is stressed. One thing we can do is to avoid
>>> timeouts whenever possible.
>>> 
>>> Julian
>>> 
>>> 
>>>> On Jul 30, 2018, at 9:30 AM, Andrei Sereda <andrei@xxxxxxxxx> wrote:
>>>> 
>>>> Pls check and confirm that the following PR fixes the issue:
>>>> https://github.com/apache/calcite/pull/770
>>>> It addresses build failures due to version parsing.
>>>> 
>>>> Stale folders / files will be addressed in a different PR.
>>>> 
>>>> 
>>>> On Mon, Jul 30, 2018 at 10:10 AM Andrei Sereda <andrei@xxxxxxxxx>
>> wrote:
>>>> 
>>>>> This must be something specific to Cassandra Unit. Will check
>>>>> 
>>>>> On Mon, Jul 30, 2018, 08:59 Sergey Nuyanzin <snuyanzin@xxxxxxxxx>
>>> wrote:
>>>>> 
>>>>>> There is one more strange thing (at least on Windows): while
>> building a
>>>>>> file with name ".toDelete" is generated under calcite\cassandra and
>>> it's
>>>>>> not removed by the end of tests
>>>>>> Is there a way to make cassandra generates these files in target
>>> directory
>>>>>> e.g.?
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> On Mon, Jul 30, 2018 at 3:52 PM, Andrei Sereda <andrei@xxxxxxxxx>
>>> wrote:
>>>>>> 
>>>>>>> Most of the problems are during test init Phase. Most likely with
>>>>>> version
>>>>>>> string (eg. 11-ea for JDK11).
>>>>>>> I'll fix that.
>>>>>>> 
>>>>>>> 
>>>>>>> On Mon, Jul 30, 2018 at 8:16 AM Michael Mior <mmior@xxxxxxxxxx>
>>> wrote:
>>>>>>> 
>>>>>>>> I tested myself a fair bit under Ubuntu before pushing this and
>>> didn't
>>>>>>> see
>>>>>>>> any of these issues myself. That said, I agree that it's important
>>> the
>>>>>>> test
>>>>>>>> suite be stable. I'm fine with reverting for now or (more
>> preferably
>>>>>> IMO)
>>>>>>>> just disabling these tests by default.
>>>>>>>> 
>>>>>>>> --
>>>>>>>> Michael Mior
>>>>>>>> mmior@xxxxxxxxxx
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> Le lun. 30 juil. 2018 à 00:59, Julian Hyde <jhyde@xxxxxxxxxx> a
>>>>>> écrit :
>>>>>>>> 
>>>>>>>>> I ran on several JDK versions, all on Ubuntu Linux. The machine
>> was
>>>>>>>>> fairly heavily loaded (I was copying one filesystem to another at
>>>>>> the
>>>>>>>>> time).
>>>>>>>>> 
>>>>>>>>> Here's the error from OpenJDK10:
>>>>>>>>> 
>>>>>>>>> [INFO] Running org.apache.calcite.test.CassandraAdapterTest
>>>>>>>>> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time
>>>>>>>>> elapsed: 0.001 s <<< FAILURE! - in
>>>>>>>>> org.apache.calcite.test.CassandraAdapterTest
>>>>>>>>> [ERROR] org.apache.calcite.test.CassandraAdapterTest  Time
>> elapsed:
>>>>>>>>> 0.001 s  <<< ERROR!
>>>>>>>>> java.lang.ExceptionInInitializerError
>>>>>>>>>       at
>>>>>>>>> 
>>>>>>>> 
>> org.apache.calcite.test.CassandraAdapterTest.initCassandraIfEnabled(
>>>>>>> CassandraAdapterTest.java:106)
>>>>>>>>>       at
>>>>>>>>> 
>>>>>>>> org.apache.calcite.test.CassandraAdapterTest.<clinit>(
>>>>>>> CassandraAdapterTest.java:56)
>>>>>>>>> Caused by: java.lang.StringIndexOutOfBoundsException: begin 0, end
>>>>>> -1,
>>>>>>>>> length 5
>>>>>>>>>       at
>>>>>>>>> 
>>>>>>>> 
>> org.apache.calcite.test.CassandraAdapterTest.initCassandraIfEnabled(
>>>>>>> CassandraAdapterTest.java:106)
>>>>>>>>>       at
>>>>>>>>> 
>>>>>>>> org.apache.calcite.test.CassandraAdapterTest.<clinit>(
>>>>>>> CassandraAdapterTest.java:56)
>>>>>>>>> 
>>>>>>>>> Here's the error in JDK 11:
>>>>>>>>> 
>>>>>>>>> [INFO] Running org.apache.calcite.test.CassandraAdapterTest
>>>>>>>>> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time
>>>>>>>>> elapsed: 0 s <<< FAILURE! - in
>>>>>>>>> org.apache.calcite.test.CassandraAdapterTest
>>>>>>>>> [ERROR] org.apache.calcite.test.CassandraAdapterTest  Time
>> elapsed:
>>>>>> 0
>>>>>>>>> s  <<< ERROR!
>>>>>>>>> java.lang.ExceptionInInitializerError
>>>>>>>>>       at
>>>>>>>>> 
>>>>>>>> org.apache.calcite.test.CassandraAdapterTest.enabled(
>>>>>>> CassandraAdapterTest.java:81)
>>>>>>>>>       at
>>>>>>>>> 
>>>>>>>> 
>> org.apache.calcite.test.CassandraAdapterTest.initCassandraIfEnabled(
>>>>>>> CassandraAdapterTest.java:88)
>>>>>>>>>       at
>>>>>>>>> 
>>>>>>>> org.apache.calcite.test.CassandraAdapterTest.<clinit>(
>>>>>>> CassandraAdapterTest.java:56)
>>>>>>>>> 
>>>>>>>>> Here's the failure from JDK 9:
>>>>>>>>> 
>>>>>>>>> [INFO] Running org.apache.calcite.test.CassandraAdapterTest
>>>>>>>>> [ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time
>>>>>>>>> elapsed: 0.001 s <<< FAILURE! - in
>>>>>>>>> org.apache.calcite.test.CassandraAdapterTest
>>>>>>>>> [ERROR] org.apache.calcite.test.CassandraAdapterTest  Time
>> elapsed:
>>>>>>>>> 0.001 s  <<< FAILURE!
>>>>>>>>> java.lang.AssertionError: Cassandra daemon did not start within
>>>>>> timeout
>>>>>>>>> 
>>>>>>>>> Here's the failure from JDK 10:
>>>>>>>>> 
>>>>>>>>> [INFO] Running org.apache.calcite.jdbc.CalciteRemoteDriverTest
>>>>>>>>> [ERROR] Tests run: 28, Failures: 1, Errors: 0, Skipped: 0, Time
>>>>>>>>> elapsed: 0.642 s <<< FAILURE! - in
>>>>>>>>> org.apache.calcite.jdbc.CalciteRemoteDriverTest
>>>>>>>>> [ERROR]
>>>>>>>>> testRemoteExecuteQuery(org.apache.calcite.jdbc.
>>>>>>> CalciteRemoteDriverTest)
>>>>>>>>> Time elapsed: 0.047 s  <<< FAILURE!
>>>>>>>>> java.lang.AssertionError:
>>>>>>>>> 
>>>>>>>>> Expected: "EXPR$0=1; EXPR$1=a\nEXPR$0=null; EXPR$1=b"
>>>>>>>>>    but: was ""
>>>>>>>>>       at
>>>>>>>>> 
>>>>>>>> 
>>>>>> 
>> org.apache.calcite.jdbc.CalciteRemoteDriverTest.testRemoteExecuteQuery(
>>>>>>> CalciteRemoteDriverTest.java:280)
>>>>>>>>> 
>>>>>>>>> In conclusion: It's worrying that the suite shows 4 different
>> cracks
>>>>>>>>> under 4 different JDKs. Clearly the load on my machine was making
>>>>>>>>> problems worse, and granted, the problems are just testing
>> problems,
>>>>>>>>> not real bugs. But flaky test suites waste time and effort. There
>>>>>> are
>>>>>>>>> indications that the new embedded Cassandra test is more flaky
>> than
>>>>>>>>> most.
>>>>>>>>> 
>>>>>>>>> Julian
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> On Sun, Jul 29, 2018 at 11:33 AM, Andrei Sereda <andrei@xxxxxxxxx
>>> 
>>>>>>>> wrote:
>>>>>>>>>> What version of java / OS / maven do you have ? What is your
>> maven
>>>>>>>>> command
>>>>>>>>>> ?
>>>>>>>>>> 
>>>>>>>>>> Things pass for me on MacOS X java 8,9 and 10.
>>>>>>>>>> 
>>>>>>>>>> On Sun, Jul 29, 2018 at 2:05 PM Julian Hyde <jhyde@xxxxxxxxxx>
>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>>> I'm seeing the following error when I run the tests on bd0e14002
>>>>>>>>>>> origin/master. Anyone else see it?
>>>>>>>>>>> 
>>>>>>>>>>> [INFO] Running org.apache.calcite.test.CassandraAdapterTest
>>>>>>>>>>> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time
>>>>>>>>>>> elapsed: 0.001 s <<< FAILURE! - in
>>>>>>>>>>> org.apache.calcite.test.CassandraAdapterTest
>>>>>>>>>>> [ERROR] org.apache.calcite.test.CassandraAdapterTest  Time
>>>>>> elapsed:
>>>>>>>>>>> 0.001 s  <<< ERROR!
>>>>>>>>>>> java.lang.ExceptionInInitializerError
>>>>>>>>>>>       at
>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>> org.apache.calcite.test.CassandraAdapterTest.initCassandraIfEnabled(
>>>>>>> CassandraAdapterTest.java:106)
>>>>>>>>>>>       at
>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> org.apache.calcite.test.CassandraAdapterTest.<clinit>(
>>>>>>> CassandraAdapterTest.java:56)
>>>>>>>>>>> Caused by: java.lang.StringIndexOutOfBoundsException: begin 0,
>>>>>> end
>>>>>>> -1,
>>>>>>>>>>> length 5
>>>>>>>>>>>       at
>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>> org.apache.calcite.test.CassandraAdapterTest.initCassandraIfEnabled(
>>>>>>> CassandraAdapterTest.java:106)
>>>>>>>>>>>       at
>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> org.apache.calcite.test.CassandraAdapterTest.<clinit>(
>>>>>>> CassandraAdapterTest.java:56)
>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> --
>>>>>> Best regards,
>>>>>> Sergey
>>>>>> 
>>>>> 
>>> 
>>> 
>>