osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Intermediate failure of materialized views in Cassandra adapter


I have created a PR perhaps you can rebase on the top :
https://github.com/apache/calcite/pull/769

There is one issue with JDK11. I believe cassandra 3.x doesn't work with
JDK11 yet: see CASSANDRA-9608
<https://issues.apache.org/jira/browse/CASSANDRA-9608>

Can this test be ignored on JDK11 ?

On Fri, Jul 27, 2018 at 4:39 PM Andrei Sereda <andrei@xxxxxxxxx> wrote:

> Take a look at my branch c389451 :
>
>
> https://github.com/asereda-gs/calcite/commit/c38945122e1edf4b417db6b82823b4a1b8c2c8b5
>
> It works both with Maven and IntelliJ
>
> On Fri, Jul 27, 2018 at 10:36 AM Michael Mior <mmior@xxxxxxxxxxxx> wrote:
>
>> That would be great if you could check that out as I'm not an IntelliJ
>> user
>> myself. Thanks!
>>
>> --
>> Michael Mior
>> mmior@xxxxxxxxxxxx
>>
>>
>> Le jeu. 26 juil. 2018 à 22:11, Andrei Sereda <andrei@xxxxxxxxx> a écrit :
>>
>> > I think there are several problems:
>> >
>> > 1) "target/embeddedCassandra" is not in the classpath (of cassandra
>> daemon)
>> > so .properties file is not found.
>> > 2) There are multiple loggers in the classpath. First one (which
>> happens to
>> > be logback) is selected by slf4j.
>> >
>> > SLF4J: Class path contains multiple SLF4J bindings.
>> > SLF4J: Found binding in
>> >
>> >
>> [jar:file:/Users/pupkin/.m2/repository/ch/qos/logback/logback-classic/1.1.3/logback-classic-1.1.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>> > SLF4J: Found binding in
>> >
>> >
>> [jar:file:/Users/pupkin/.m2/repository/org/slf4j/slf4j-log4j12/1.7.13/slf4j-log4j12-1.7.13.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>> > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
>> > explanation.
>> >
>> > To reduce DEBUG noise remove slf4j-log4j12 as dependency (from pom.xml)
>> and
>> > add logback-test.xml with sensitive defaults to src/test/resources/
>> >
>> > Running from IDE (IntelliJ)
>> > One problem I have noticed is that test fails when running manually from
>> > IntelliJ. Do you want me to take a look ?
>> >
>> > java.lang.ExceptionInInitializerError
>> > at
>> >
>> >
>> org.cassandraunit.utils.EmbeddedCassandraServerHelper.mkdir(EmbeddedCassandraServerHelper.java:321)
>> > at
>> >
>> >
>> org.cassandraunit.utils.EmbeddedCassandraServerHelper.copy(EmbeddedCassandraServerHelper.java:306)
>> > at
>> >
>> >
>> org.cassandraunit.utils.EmbeddedCassandraServerHelper.startEmbeddedCassandra(EmbeddedCassandraServerHelper.java:106)
>> > at
>> >
>> >
>> org.cassandraunit.utils.EmbeddedCassandraServerHelper.startEmbeddedCassandra(EmbeddedCassandraServerHelper.java:88)
>> > at
>> >
>> >
>> org.cassandraunit.utils.EmbeddedCassandraServerHelper.startEmbeddedCassandra(EmbeddedCassandraServerHelper.java:80)
>> > at org.cassandraunit.BaseCassandraUnit.before(BaseCassandraUnit.java:31)
>> >
>> > *Caused by: java.lang.NullPointerException*
>> > * at
>> >
>> >
>> org.apache.cassandra.config.DatabaseDescriptor.getDiskFailurePolicy(DatabaseDescriptor.java:1881)*
>> > at
>> >
>> >
>> org.apache.cassandra.utils.JVMStabilityInspector.inspectThrowable(JVMStabilityInspector.java:82)
>> > at org.apache.cassandra.io.util.FileUtils.<clinit>(FileUtils.java:79)
>> > ... 14 more
>> >
>> > Regards,
>> > Andrei.
>> >
>> > On Thu, Jul 26, 2018 at 2:55 PM Andrei Sereda <andrei@xxxxxxxxx> wrote:
>> >
>> > > I'll check that tonight
>> > >
>> > > On Thu, Jul 26, 2018, 13:29 Michael Mior <mmior@xxxxxxxxxx> wrote:
>> > >
>> > >> I already tried basically the same myself and the test consistently
>> > passes
>> > >> now. I updated the PR. If you're able to give it a try before I
>> merge,
>> > >> that
>> > >> would be great. Also, I can't seem to figure out the right
>> configuration
>> > >> to
>> > >> turn off the incredibly verbose logging CassandraUnit uses. It's not
>> a
>> > >> blocker, but would be nice to fix. (You can see the config which gets
>> > >> pulled in under
>> > >>
>> cassandra/target/embeddedCassandra/log4j-embedded-cassandra.properties)
>> > >>
>> > >> --
>> > >> Michael Mior
>> > >> mmior@xxxxxxxxxx
>> > >>
>> > >>
>> > >>
>> > >> Le mer. 25 juil. 2018 à 19:41, Andrei Sereda <andrei@xxxxxxxxx> a
>> > écrit :
>> > >>
>> > >> > My issue was with JDK10 and maven javadoc plugin (see MJAVADOC-517
>> > >> > <https://issues.apache.org/jira/browse/MJAVADOC-517>)
>> > >> >
>> > >> > Can you double-check if @NotThreadSafe fixes the issue with the
>> > >> following
>> > >> > commands:
>> > >> >
>> > >> > $ cd cassandra
>> > >> >
>> > >> > $ seq 20 | xargs -i sh -c "mvn -Pit integration-test verify || exit
>> > 255"
>> > >> >
>> > >> >
>> > >> >  You can also use -DforkCount=0 to run tests sequentially.
>> > >> >
>> > >> >
>> > >> > On Wed, Jul 25, 2018 at 7:16 PM Michael Mior <mmior@xxxxxxxxxx>
>> > wrote:
>> > >> >
>> > >> > > Good call. Adding a @NotThreadSafe annotation to the test class
>> > seems
>> > >> to
>> > >> > > solve the problem. The setup and teardown of CassandraUnit is
>> still
>> > >> > shared
>> > >> > > between all the tests and each test (after setup) runs rather
>> > quickly
>> > >> so
>> > >> > I
>> > >> > > don't think running the tests serially is a problem. I've never
>> seen
>> > >> the
>> > >> > > error you're seeing before. A full stack trace might help.
>> > >> > >
>> > >> > > --
>> > >> > > Michael Mior
>> > >> > > mmior@xxxxxxxxxx
>> > >> > >
>> > >> > >
>> > >> > >
>> > >> > > Le mer. 25 juil. 2018 à 18:16, Andrei Sereda <andrei@xxxxxxxxx>
>> a
>> > >> écrit
>> > >> > :
>> > >> > >
>> > >> > > > Did you run the tests from IDE or command line (maven) ? Do you
>> > get
>> > >> > some
>> > >> > > > behaviour in both ?
>> > >> > > >
>> > >> > > > Integration tests (-Pit) run by maven (surefire) are executed
>> in
>> > >> > parallel
>> > >> > > > (see pom.xml <
>> > >> > https://github.com/apache/calcite/blob/master/pom.xml#L876
>> > >> > > > >).
>> > >> > > > This may contribute to non-determinism (eg. sharing some static
>> > >> > variable
>> > >> > > > like Hook).
>> > >> > > >
>> > >> > > > I will try to reproduce it myself to better understand what's
>> > >> happening
>> > >> > > > with cassandra adapter. Still getting NPE in maven for some
>> > reason.
>> > >> > > >
>> > >> > > > On Wed, Jul 25, 2018 at 4:53 PM Michael Mior <mmior@xxxxxxxxxx
>> >
>> > >> wrote:
>> > >> > > >
>> > >> > > > > This is unrelated to that branch (except that I don't want to
>> > >> merge
>> > >> > the
>> > >> > > > > branch with CassandraUnit until tests pass).
>> > >> > > > >
>> > >> > > > > --
>> > >> > > > > Michael Mior
>> > >> > > > > mmior@xxxxxxxxxx
>> > >> > > > >
>> > >> > > > >
>> > >> > > > >
>> > >> > > > > Le mer. 25 juil. 2018 à 16:36, Andrei Sereda
>> <andrei@xxxxxxxxx>
>> > a
>> > >> > > écrit
>> > >> > > > :
>> > >> > > > >
>> > >> > > > > > Mike,
>> > >> > > > > >
>> > >> > > > > > Is your branch michaelmior:2419-cassandra-unit up to date ?
>> > I'm
>> > >> > > getting
>> > >> > > > > > NPE:
>> > >> > > > > >
>> > >> > > > > >
>> > >> > > > > > > Caused by: java.lang.NullPointerException
>> > >> > > > > > > at
>> > >> > > > > > >
>> > >> > > > > >
>> > >> > > > >
>> > >> > > >
>> > >> > >
>> > >> >
>> > >>
>> >
>> org.apache.cassandra.config.DatabaseDescriptor.getDiskFailurePolicy(DatabaseDescriptor.java:1881)
>> > >> > > > > > > at
>> > >> > > > > > >
>> > >> > > > > >
>> > >> > > > >
>> > >> > > >
>> > >> > >
>> > >> >
>> > >>
>> >
>> org.apache.cassandra.utils.JVMStabilityInspector.inspectThrowable(JVMStabilityInspector.java:82)
>> > >> > > > > > > at
>> > >> > >
>> org.apache.cassandra.io.util.FileUtils.<clinit>(FileUtils.java:79)
>> > >> > > > > > > ... 13 more
>> > >> > > > > >
>> > >> > > > > >
>> > >> > > > > > What error are you getting ?
>> > >> > > > > >
>> > >> > > > > > Regards,
>> > >> > > > > > Andrei.
>> > >> > > > > >
>> > >> > > > > > On Wed, Jul 25, 2018 at 1:29 PM Michael Mior <
>> > mmior@xxxxxxxxxx>
>> > >> > > wrote:
>> > >> > > > > >
>> > >> > > > > > > I'd appreciate some help with the JIRA below. I'm using
>> > >> > > Hook.TRIMMED
>> > >> > > > > > > to expose materialized views to the adapter and a test
>> which
>> > >> > checks
>> > >> > > > > > > that the view can be used for query rewriting is failing
>> > about
>> > >> > 50%
>> > >> > > of
>> > >> > > > > > > the time. I don't see why there should be any
>> > non-determinism
>> > >> > here
>> > >> > > so
>> > >> > > > > > > any ideas appreciated :)
>> > >> > > > > > >
>> > >> > > > > > > https://issues.apache.org/jira/browse/CALCITE-2423
>> > >> > > > > > >
>> > >> > > > > > > --
>> > >> > > > > > > Michael Mior
>> > >> > > > > > > mmior@xxxxxxxxxx
>> > >> > > > > > >
>> > >> > > > > >
>> > >> > > > >
>> > >> > > >
>> > >> > >
>> > >> >
>> > >>
>> > >
>> >
>>
>