osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [VOTE] Release 1.6.0, release candidate #4


Thanks for reporting this problem Yaz. I just pushed a commit which should
update the links accordingly once the Flink documentation gets rebuilt
(over night). Tomorrow it should be fixed.

Cheers,
Till

On Thu, Aug 9, 2018 at 2:53 PM Yaz Sh <yazdanc1@xxxxxxxxx> wrote:

> Great Job on Release 1.6!
>
> I just checked it out and still I can see v.1.6-SNAPSHOT on the title of
> https://ci.apache.org/projects/flink/flink-docs-release-1.6/ <
> https://ci.apache.org/projects/flink/flink-docs-release-1.6/>
>
> and when I click on any options, it redirects me to master docs
> 1.7-SNAPSHOT.
>
> I opened this ticket https://issues.apache.org/jira/browse/FLINK-10112 <
> https://issues.apache.org/jira/browse/FLINK-10112>
>
> Also I don’t see v1.6 on “Pick Docs Version" drop down
>
> Cheers,
> Yaz
>
> > On Aug 8, 2018, at 3:24 PM, Timo Walther <twalthr@xxxxxxxxxx> wrote:
> >
> > +1
> >
> > - successfully run `mvn clean verify` locally
> > - successfully run end-to-end tests locally (except for SQL Client
> end-to-end test)
> >
> > Found a bug in the class loading of SQL JAR files. This is not a blocker
> but a bug that we should fix soon. As an easy workaround user should not
> use different Kafka versions as SQL Client dependencies.
> >
> > Regards,
> > Timo
> >
> > Am 08.08.18 um 18:10 schrieb Dawid Wysakowicz:
> >> +1
> >>
> >> - verified compilation, tests
> >> - verified checksum and gpg files
> >> - verified sbt templates (g8, quickstart) - run assemblies on local
> cluster
> >>
> >> - I could not execute the nightly-tests.sh though. The tests that were
> >> failing most often are:
> >>     - test_streaming_file_sink.sh
> >>     - test_streaming_elasticsearch.sh
> >>
> >> Those are connectors though and it might be only tests flakiness so I
> >> think it should not block the release.
> >>
> >> On 08/08/18 16:36, Chesnay Schepler wrote:
> >>> I did not use the tools/list_deps.py script as I wasn't aware that it
> >>> existed.
> >>>
> >>> Even if I were I wouldn't have used it and in fact would advocate for
> >>> removing it.
> >>> It manually parses and constructs dependency information which is
> >>> utterly unnecessary as maven already provides this functionality, with
> >>> the added bonus of also accounting for dependencyManagement and
> >>> transitive dependencies which we obviously have to take into account.
> >>>
> >>> I used this one-liner instead:
> >>> |mvn dependency:list | ||grep| |":.*:.*:.*"| || ||grep| |-||v| |-e
> >>> ||"Finished at"| |-e ||"Some problems"| || ||cut| |-d] -f2- | ||sed|
> >>> |'s/:[a-z]*$//g'| || ||sort| |-u
> >>>
> >>> |which I have documented here:
> >>> https://cwiki.apache.org/confluence/display/FLINK/Dependencies
> >>>
> >>> On 08.08.2018 15:06, Aljoscha Krettek wrote:
> >>>> +1
> >>>>
> >>>> - verified checksum and gpg files
> >>>> - verified LICENSE and NOTICE: NOTICE didn't change from 1.5, LICENSE
> >>>> had one unnecessary part removed
> >>>>
> >>>> Side comment: I'm not sure whether the "Verify that the LICENSE and
> >>>> NOTICE file is correct for the binary and source releases" part is
> >>>> valid anymore because we only have one LICENSE and NOTICE file. also
> >>>> "The LICENSE and NOTICE files in flink-dist/src/main/flink-bin refer
> >>>> to the binary distribution and mention all of Flink's Maven
> >>>> dependencies as well" can be dropped because we don't have them
> anymore.
> >>>>
> >>>> I came to the same conclusion on dependencies. I used
> >>>> tools/list_deps.py and diff'ed the output for 1.5 and 1.6, that's
> >>>> probably what Chesnay also did ... :-)
> >>>>
> >>>>> On 8. Aug 2018, at 14:43, Chesnay Schepler <chesnay@xxxxxxxxxx>
> wrote:
> >>>>>
> >>>>> +1
> >>>>>
> >>>>> - verified source release contains no binaries
> >>>>> - verified correct versions in source release
> >>>>> - verified compilation, tests and E2E-tests pass (on travis)
> >>>>> - verified checksum and gpg files
> >>>>>
> >>>>> New dependencies (excluding dependencies where we simply depend on a
> >>>>> different version now):
> >>>>>     Apache licensed:
> >>>>>         io.confluent:common-utils:jar:3.3.1
> >>>>>         io.confluent:kafka-schema-registry-client:jar:3.3.1
> >>>>>         io.prometheus:simpleclient_pushgateway:jar:0.3.0
> >>>>>         various Apache Nifi dependencies
> >>>>>         various Apache Parquet dependencies
> >>>>>         various ElasticSearch dependencies
> >>>>>     CDDL:
> >>>>>         javax.ws.rs:javax.ws.rs-api:jar:2.1
> >>>>>     Bouncycastle (MIT-like):
> >>>>>         org.bouncycastle:bcpkix-jdk15on:jar:1.59
> >>>>>         org.bouncycastle:bcprov-jdk15on:jar:1.59
> >>>>>     MIT:
> >>>>>         org.projectlombok:lombok:jar:1.16.20
> >>>>>
> >>>>> On 08.08.2018 13:28, Till Rohrmann wrote:
> >>>>>> Thanks for reporting these problems Chesnay. The usage string in
> >>>>>> `standalone-job.sh` is out dated and should be updated. The same
> >>>>>> applies to
> >>>>>> the typo.
> >>>>>>
> >>>>>> When calling `standalone-job.sh start --job-classname foobar.Job`
> >>>>>> please
> >>>>>> make sure that the user code jar is contained in the classpath (e.g.
> >>>>>> putting the jar in the lib directory). Documenting this behaviour
> >>>>>> is part
> >>>>>> of the pending issue FLINK-10001.
> >>>>>>
> >>>>>> We should fix all of these issues. They are, however, no release
> >>>>>> blockers.
> >>>>>>
> >>>>>> Cheers,
> >>>>>> Till
> >>>>>>
> >>>>>> On Wed, Aug 8, 2018 at 11:31 AM Chesnay Schepler
> >>>>>> <chesnay@xxxxxxxxxx> wrote:
> >>>>>>
> >>>>>>> I found some issues with the standalone-job.sh script.
> >>>>>>>
> >>>>>>> I ran "./bin/standalone-job.sh start" as described by the usage
> >>>>>>> string.
> >>>>>>>
> >>>>>>>      2018-08-08 09:22:34,385 ERROR
> >>>>>>>      org.apache.flink.runtime.entrypoint.ClusterEntrypoint
>  -
> >>>>>>>      Could not parse command line arguments [--configDir,
> >>>>>>>      /home/zento/svn/flink-1.6.0/flink-1.6.0/conf].
> >>>>>>>      org.apache.flink.runtime.entrypoint.FlinkParseException:
> >>>>>>> Failed to
> >>>>>>>      parse the command line arguments.
> >>>>>>>               at
> >>>>>>>
> >>>>>>>
> org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:52)
> >>>>>>>
> >>>>>>>               at
> >>>>>>>
> >>>>>>>
> org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.main(StandaloneJobClusterEntryPoint.java:143)
> >>>>>>>
> >>>>>>>      Caused by: org.apache.commons.cli.MissingOptionException:
> >>>>>>> Missing
> >>>>>>>      required option: j
> >>>>>>>               at
> >>>>>>>
> >>>>>>>
> org.apache.commons.cli.DefaultParser.checkRequiredOptions(DefaultParser.java:199)
> >>>>>>>
> >>>>>>>               at
> >>>>>>>
>  org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:130)
> >>>>>>>               at
> >>>>>>>
>  org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:81)
> >>>>>>>               at
> >>>>>>>
> >>>>>>>
> org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:50)
> >>>>>>>
> >>>>>>>              ... 1 more
> >>>>>>>
> >>>>>>> The script should fail earlier if no jar is provided, with a better
> >>>>>>> error message.
> >>>>>>> It is also undocumented, and the usage instructions don't appear
> >>>>>>> correct.
> >>>>>>>
> >>>>>>> Passing a jar with the -j option leads to a ClassNotFoundException:
> >>>>>>> "./bin/standalone-job.sh start -j examples/streaming/WordCount.jar"
> >>>>>>>
> >>>>>>>      2018-08-08 09:26:30,562 ERROR
> >>>>>>>      org.apache.flink.runtime.entrypoint.ClusterEntrypoint
>  -
> >>>>>>>      Cluster initialization failed.
> >>>>>>>      java.lang.reflect.UndeclaredThrowableException
> >>>>>>>               at
> >>>>>>>
> >>>>>>>
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1854)
> >>>>>>>
> >>>>>>>               at
> >>>>>>>
> >>>>>>>
> org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
> >>>>>>>
> >>>>>>>               at
> >>>>>>>
> >>>>>>>
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:189)
> >>>>>>>
> >>>>>>>               at
> >>>>>>>
> >>>>>>>
> org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.main(StandaloneJobClusterEntryPoint.java:158)
> >>>>>>>
> >>>>>>>      Caused by: org.apache.flink.util.FlinkException: Could not
> >>>>>>> load the
> >>>>>>>      provied entrypoint class.
> >>>>>>>               at
> >>>>>>>
> >>>>>>>
> org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.createPackagedProgram(StandaloneJobClusterEntryPoint.java:92)
> >>>>>>>
> >>>>>>>               at
> >>>>>>>
> >>>>>>>
> org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.retrieveJobGraph(StandaloneJobClusterEntryPoint.java:75)
> >>>>>>>
> >>>>>>>               at
> >>>>>>>
> >>>>>>>
> org.apache.flink.runtime.entrypoint.JobClusterEntrypoint.createDispatcher(JobClusterEntrypoint.java:107)
> >>>>>>>
> >>>>>>>               at
> >>>>>>>
> >>>>>>>
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startClusterComponents(ClusterEntrypoint.java:353)
> >>>>>>>
> >>>>>>>               at
> >>>>>>>
> >>>>>>>
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runCluster(ClusterEntrypoint.java:232)
> >>>>>>>
> >>>>>>>               at
> >>>>>>>
> >>>>>>>
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.lambda$startCluster$0(ClusterEntrypoint.java:190)
> >>>>>>>
> >>>>>>>               at
> >>>>>>> java.security.AccessController.doPrivileged(Native Method)
> >>>>>>>               at javax.security.auth.Subject.doAs(Subject.java:422)
> >>>>>>>               at
> >>>>>>>
> >>>>>>>
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836)
> >>>>>>>
> >>>>>>>               ... 3 more
> >>>>>>>      Caused by: java.lang.ClassNotFoundException:
> >>>>>>>      examples/streaming/WordCount.jar
> >>>>>>>               at
> >>>>>>> java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> >>>>>>>               at
> >>>>>>> java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> >>>>>>>               at
> >>>>>>>      sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:338)
> >>>>>>>               at
> >>>>>>> java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> >>>>>>>               at
> >>>>>>>
> >>>>>>>
> org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.createPackagedProgram(StandaloneJobClusterEntryPoint.java:89)
> >>>>>>>
> >>>>>>>               ... 11 more
> >>>>>>>
> >>>>>>> So this seems to not work at all, but maybe I'm using it wrong?
> >>>>>>>
> >>>>>>> (There's also typo in "Could not load the provied entrypoint
> class")
> >>>>>>>
> >>>>>>> On 08.08.2018 10:33, Piotr Nowojski wrote:
> >>>>>>>> +1 from my side
> >>>>>>>>
> >>>>>>>> I’ve spent some time playing around with various examples
> (batching,
> >>>>>>> streaming and SQL) on EMR 6 nodes cluster with yarn deployment,
> with
> >>>>>>> different configuration options (number of task
> >>>>>>> managers/memory/Flip6/credit base flow control/metrics) and
> >>>>>>> everything
> >>>>>>> looks now fine (after fixing
> >>>>>>> https://issues.apache.org/jira/browse/FLINK-9969 <
> >>>>>>> https://issues.apache.org/jira/browse/FLINK-9969> ).
> >>>>>>>> Piotrek
> >>>>>>>>
> >>>>>>>>> On 7 Aug 2018, at 17:17, Till Rohrmann <trohrmann@xxxxxxxxxx>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>> Hi everyone,
> >>>>>>>>> Please review and vote on the release candidate #4 for the
> version
> >>>>>>> 1.6.0,
> >>>>>>>>> as follows:
> >>>>>>>>> [ ] +1, Approve the release
> >>>>>>>>> [ ] -1, Do not approve the release (please provide specific
> >>>>>>>>> comments)
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> The complete staging area is available for your review, which
> >>>>>>>>> includes:
> >>>>>>>>> * JIRA release notes [1],
> >>>>>>>>> * the official Apache source release and binary convenience
> >>>>>>>>> releases to
> >>>>>>> be
> >>>>>>>>> deployed to dist.apache.org [2], which are signed with the key
> with
> >>>>>>>>> fingerprint 1F302569A96CFFD5 [3],
> >>>>>>>>> * all artifacts to be deployed to the Maven Central Repository
> [4],
> >>>>>>>>> * source code tag "release-1.6.0-rc4" [5],
> >>>>>>>>> * website pull request listing the new release and adding
> >>>>>>>>> announcement
> >>>>>>> blog
> >>>>>>>>> post [6].
> >>>>>>>>>
> >>>>>>>>> Please use this document for coordinating testing efforts: [7]
> >>>>>>>>>
> >>>>>>>>> The vote will be shortened since we only adde a minor fix on top
> >>>>>>>>> of the
> >>>>>>> RC
> >>>>>>>>> 3. It will close on Wednesday 6:30pm CET. It is adopted by
> majority
> >>>>>>>>> approval, with at least 3 PMC affirmative votes.
> >>>>>>>>>
> >>>>>>>>> Thanks,
> >>>>>>>>> Your friendly Release Manager
> >>>>>>>>>
> >>>>>>>>> [1]
> >>>>>>>>>
> >>>>>>>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12342760
> >>>>>>>
> >>>>>>>>> [2] https://dist.apache.org/repos/dist/dev/flink/flink-1.6.0/
> >>>>>>>>> [3] https://dist.apache.org/repos/dist/release/flink/KEYS
> >>>>>>>>> [4]
> >>>>>>>
> https://repository.apache.org/content/repositories/orgapacheflink-1178
> >>>>>>>
> >>>>>>>>> [5] https://github.com/apache/flink/tree/release-1.6.0-rc4
> >>>>>>>>> [6] https://github.com/apache/flink-web/pull/117
> >>>>>>>>> [7]
> >>>>>>>>>
> >>>>>>>
> https://docs.google.com/document/d/1upBFZQ7tbaSkYvDiLqfUFXKg8Xxs-lVheEfb66e4jpo/edit?usp=sharing
> >>>>>>>
> >>>>>>>>> Pro-tip: you can create a settings.xml file with these contents:
> >>>>>>>>>
> >>>>>>>>> <settings>
> >>>>>>>>> <activeProfiles>
> >>>>>>>>>    <activeProfile>flink-1.6.0</activeProfile>
> >>>>>>>>> </activeProfiles>
> >>>>>>>>> <profiles>
> >>>>>>>>>    <profile>
> >>>>>>>>>      <id>flink-1.6.0</id>
> >>>>>>>>>      <repositories>
> >>>>>>>>>        <repository>
> >>>>>>>>>          <id>flink-1.6.0</id>
> >>>>>>>>>          <url>
> >>>>>>>>>
> >>>>>>>>>
> https://repository.apache.org/content/repositories/orgapacheflink-1178/
> >>>>>>>>>
> >>>>>>>>>          </url>
> >>>>>>>>>        </repository>
> >>>>>>>>>        <repository>
> >>>>>>>>>          <id>archetype</id>
> >>>>>>>>>          <url>
> >>>>>>>>>
> >>>>>>>>>
> https://repository.apache.org/content/repositories/orgapacheflink-1178/
> >>>>>>>>>
> >>>>>>>>>          </url>
> >>>>>>>>>        </repository>
> >>>>>>>>>      </repositories>
> >>>>>>>>>    </profile>
> >>>>>>>>> </profiles>
> >>>>>>>>> </settings>
> >>>>>>>>>
> >>>>>>>>> And reference that in you maven commands via --settings
> >>>>>>>>> path/to/settings.xml. This is useful for creating a quickstart
> >>>>>>>>> based on
> >>>>>>> the
> >>>>>>>>> staged release and for building against the staged jars.
> >>>
> >
>
>