osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Intermediate failure of materialized views in Cassandra adapter


Take a look at my branch c389451 :

https://github.com/asereda-gs/calcite/commit/c38945122e1edf4b417db6b82823b4a1b8c2c8b5

It works both with Maven and IntelliJ

On Fri, Jul 27, 2018 at 10:36 AM Michael Mior <mmior@xxxxxxxxxxxx> wrote:

> That would be great if you could check that out as I'm not an IntelliJ user
> myself. Thanks!
>
> --
> Michael Mior
> mmior@xxxxxxxxxxxx
>
>
> Le jeu. 26 juil. 2018 à 22:11, Andrei Sereda <andrei@xxxxxxxxx> a écrit :
>
> > I think there are several problems:
> >
> > 1) "target/embeddedCassandra" is not in the classpath (of cassandra
> daemon)
> > so .properties file is not found.
> > 2) There are multiple loggers in the classpath. First one (which happens
> to
> > be logback) is selected by slf4j.
> >
> > SLF4J: Class path contains multiple SLF4J bindings.
> > SLF4J: Found binding in
> >
> >
> [jar:file:/Users/pupkin/.m2/repository/ch/qos/logback/logback-classic/1.1.3/logback-classic-1.1.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> > SLF4J: Found binding in
> >
> >
> [jar:file:/Users/pupkin/.m2/repository/org/slf4j/slf4j-log4j12/1.7.13/slf4j-log4j12-1.7.13.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> > explanation.
> >
> > To reduce DEBUG noise remove slf4j-log4j12 as dependency (from pom.xml)
> and
> > add logback-test.xml with sensitive defaults to src/test/resources/
> >
> > Running from IDE (IntelliJ)
> > One problem I have noticed is that test fails when running manually from
> > IntelliJ. Do you want me to take a look ?
> >
> > java.lang.ExceptionInInitializerError
> > at
> >
> >
> org.cassandraunit.utils.EmbeddedCassandraServerHelper.mkdir(EmbeddedCassandraServerHelper.java:321)
> > at
> >
> >
> org.cassandraunit.utils.EmbeddedCassandraServerHelper.copy(EmbeddedCassandraServerHelper.java:306)
> > at
> >
> >
> org.cassandraunit.utils.EmbeddedCassandraServerHelper.startEmbeddedCassandra(EmbeddedCassandraServerHelper.java:106)
> > at
> >
> >
> org.cassandraunit.utils.EmbeddedCassandraServerHelper.startEmbeddedCassandra(EmbeddedCassandraServerHelper.java:88)
> > at
> >
> >
> org.cassandraunit.utils.EmbeddedCassandraServerHelper.startEmbeddedCassandra(EmbeddedCassandraServerHelper.java:80)
> > at org.cassandraunit.BaseCassandraUnit.before(BaseCassandraUnit.java:31)
> >
> > *Caused by: java.lang.NullPointerException*
> > * at
> >
> >
> org.apache.cassandra.config.DatabaseDescriptor.getDiskFailurePolicy(DatabaseDescriptor.java:1881)*
> > at
> >
> >
> org.apache.cassandra.utils.JVMStabilityInspector.inspectThrowable(JVMStabilityInspector.java:82)
> > at org.apache.cassandra.io.util.FileUtils.<clinit>(FileUtils.java:79)
> > ... 14 more
> >
> > Regards,
> > Andrei.
> >
> > On Thu, Jul 26, 2018 at 2:55 PM Andrei Sereda <andrei@xxxxxxxxx> wrote:
> >
> > > I'll check that tonight
> > >
> > > On Thu, Jul 26, 2018, 13:29 Michael Mior <mmior@xxxxxxxxxx> wrote:
> > >
> > >> I already tried basically the same myself and the test consistently
> > passes
> > >> now. I updated the PR. If you're able to give it a try before I merge,
> > >> that
> > >> would be great. Also, I can't seem to figure out the right
> configuration
> > >> to
> > >> turn off the incredibly verbose logging CassandraUnit uses. It's not a
> > >> blocker, but would be nice to fix. (You can see the config which gets
> > >> pulled in under
> > >>
> cassandra/target/embeddedCassandra/log4j-embedded-cassandra.properties)
> > >>
> > >> --
> > >> Michael Mior
> > >> mmior@xxxxxxxxxx
> > >>
> > >>
> > >>
> > >> Le mer. 25 juil. 2018 à 19:41, Andrei Sereda <andrei@xxxxxxxxx> a
> > écrit :
> > >>
> > >> > My issue was with JDK10 and maven javadoc plugin (see MJAVADOC-517
> > >> > <https://issues.apache.org/jira/browse/MJAVADOC-517>)
> > >> >
> > >> > Can you double-check if @NotThreadSafe fixes the issue with the
> > >> following
> > >> > commands:
> > >> >
> > >> > $ cd cassandra
> > >> >
> > >> > $ seq 20 | xargs -i sh -c "mvn -Pit integration-test verify || exit
> > 255"
> > >> >
> > >> >
> > >> >  You can also use -DforkCount=0 to run tests sequentially.
> > >> >
> > >> >
> > >> > On Wed, Jul 25, 2018 at 7:16 PM Michael Mior <mmior@xxxxxxxxxx>
> > wrote:
> > >> >
> > >> > > Good call. Adding a @NotThreadSafe annotation to the test class
> > seems
> > >> to
> > >> > > solve the problem. The setup and teardown of CassandraUnit is
> still
> > >> > shared
> > >> > > between all the tests and each test (after setup) runs rather
> > quickly
> > >> so
> > >> > I
> > >> > > don't think running the tests serially is a problem. I've never
> seen
> > >> the
> > >> > > error you're seeing before. A full stack trace might help.
> > >> > >
> > >> > > --
> > >> > > Michael Mior
> > >> > > mmior@xxxxxxxxxx
> > >> > >
> > >> > >
> > >> > >
> > >> > > Le mer. 25 juil. 2018 à 18:16, Andrei Sereda <andrei@xxxxxxxxx> a
> > >> écrit
> > >> > :
> > >> > >
> > >> > > > Did you run the tests from IDE or command line (maven) ? Do you
> > get
> > >> > some
> > >> > > > behaviour in both ?
> > >> > > >
> > >> > > > Integration tests (-Pit) run by maven (surefire) are executed in
> > >> > parallel
> > >> > > > (see pom.xml <
> > >> > https://github.com/apache/calcite/blob/master/pom.xml#L876
> > >> > > > >).
> > >> > > > This may contribute to non-determinism (eg. sharing some static
> > >> > variable
> > >> > > > like Hook).
> > >> > > >
> > >> > > > I will try to reproduce it myself to better understand what's
> > >> happening
> > >> > > > with cassandra adapter. Still getting NPE in maven for some
> > reason.
> > >> > > >
> > >> > > > On Wed, Jul 25, 2018 at 4:53 PM Michael Mior <mmior@xxxxxxxxxx>
> > >> wrote:
> > >> > > >
> > >> > > > > This is unrelated to that branch (except that I don't want to
> > >> merge
> > >> > the
> > >> > > > > branch with CassandraUnit until tests pass).
> > >> > > > >
> > >> > > > > --
> > >> > > > > Michael Mior
> > >> > > > > mmior@xxxxxxxxxx
> > >> > > > >
> > >> > > > >
> > >> > > > >
> > >> > > > > Le mer. 25 juil. 2018 à 16:36, Andrei Sereda <andrei@xxxxxxxxx
> >
> > a
> > >> > > écrit
> > >> > > > :
> > >> > > > >
> > >> > > > > > Mike,
> > >> > > > > >
> > >> > > > > > Is your branch michaelmior:2419-cassandra-unit up to date ?
> > I'm
> > >> > > getting
> > >> > > > > > NPE:
> > >> > > > > >
> > >> > > > > >
> > >> > > > > > > Caused by: java.lang.NullPointerException
> > >> > > > > > > at
> > >> > > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> org.apache.cassandra.config.DatabaseDescriptor.getDiskFailurePolicy(DatabaseDescriptor.java:1881)
> > >> > > > > > > at
> > >> > > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> org.apache.cassandra.utils.JVMStabilityInspector.inspectThrowable(JVMStabilityInspector.java:82)
> > >> > > > > > > at
> > >> > > org.apache.cassandra.io.util.FileUtils.<clinit>(FileUtils.java:79)
> > >> > > > > > > ... 13 more
> > >> > > > > >
> > >> > > > > >
> > >> > > > > > What error are you getting ?
> > >> > > > > >
> > >> > > > > > Regards,
> > >> > > > > > Andrei.
> > >> > > > > >
> > >> > > > > > On Wed, Jul 25, 2018 at 1:29 PM Michael Mior <
> > mmior@xxxxxxxxxx>
> > >> > > wrote:
> > >> > > > > >
> > >> > > > > > > I'd appreciate some help with the JIRA below. I'm using
> > >> > > Hook.TRIMMED
> > >> > > > > > > to expose materialized views to the adapter and a test
> which
> > >> > checks
> > >> > > > > > > that the view can be used for query rewriting is failing
> > about
> > >> > 50%
> > >> > > of
> > >> > > > > > > the time. I don't see why there should be any
> > non-determinism
> > >> > here
> > >> > > so
> > >> > > > > > > any ideas appreciated :)
> > >> > > > > > >
> > >> > > > > > > https://issues.apache.org/jira/browse/CALCITE-2423
> > >> > > > > > >
> > >> > > > > > > --
> > >> > > > > > > Michael Mior
> > >> > > > > > > mmior@xxxxxxxxxx
> > >> > > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> >
>