osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [VOTE] Release 1.6.0, release candidate #4


Great Job on Release 1.6!

I just checked it out and still I can see v.1.6-SNAPSHOT on the title of https://ci.apache.org/projects/flink/flink-docs-release-1.6/ <https://ci.apache.org/projects/flink/flink-docs-release-1.6/>

and when I click on any options, it redirects me to master docs 1.7-SNAPSHOT.

I opened this ticket https://issues.apache.org/jira/browse/FLINK-10112 <https://issues.apache.org/jira/browse/FLINK-10112>

Also I don’t see v1.6 on “Pick Docs Version" drop down

Cheers,
Yaz

> On Aug 8, 2018, at 3:24 PM, Timo Walther <twalthr@xxxxxxxxxx> wrote:
> 
> +1
> 
> - successfully run `mvn clean verify` locally
> - successfully run end-to-end tests locally (except for SQL Client end-to-end test)
> 
> Found a bug in the class loading of SQL JAR files. This is not a blocker but a bug that we should fix soon. As an easy workaround user should not use different Kafka versions as SQL Client dependencies.
> 
> Regards,
> Timo
> 
> Am 08.08.18 um 18:10 schrieb Dawid Wysakowicz:
>> +1
>> 
>> - verified compilation, tests
>> - verified checksum and gpg files
>> - verified sbt templates (g8, quickstart) - run assemblies on local cluster
>> 
>> - I could not execute the nightly-tests.sh though. The tests that were
>> failing most often are:
>>     - test_streaming_file_sink.sh
>>     - test_streaming_elasticsearch.sh
>> 
>> Those are connectors though and it might be only tests flakiness so I
>> think it should not block the release.
>> 
>> On 08/08/18 16:36, Chesnay Schepler wrote:
>>> I did not use the tools/list_deps.py script as I wasn't aware that it
>>> existed.
>>> 
>>> Even if I were I wouldn't have used it and in fact would advocate for
>>> removing it.
>>> It manually parses and constructs dependency information which is
>>> utterly unnecessary as maven already provides this functionality, with
>>> the added bonus of also accounting for dependencyManagement and
>>> transitive dependencies which we obviously have to take into account.
>>> 
>>> I used this one-liner instead:
>>> |mvn dependency:list | ||grep| |":.*:.*:.*"| || ||grep| |-||v| |-e
>>> ||"Finished at"| |-e ||"Some problems"| || ||cut| |-d] -f2- | ||sed|
>>> |'s/:[a-z]*$//g'| || ||sort| |-u
>>> 
>>> |which I have documented here:
>>> https://cwiki.apache.org/confluence/display/FLINK/Dependencies
>>> 
>>> On 08.08.2018 15:06, Aljoscha Krettek wrote:
>>>> +1
>>>> 
>>>> - verified checksum and gpg files
>>>> - verified LICENSE and NOTICE: NOTICE didn't change from 1.5, LICENSE
>>>> had one unnecessary part removed
>>>> 
>>>> Side comment: I'm not sure whether the "Verify that the LICENSE and
>>>> NOTICE file is correct for the binary and source releases" part is
>>>> valid anymore because we only have one LICENSE and NOTICE file. also
>>>> "The LICENSE and NOTICE files in flink-dist/src/main/flink-bin refer
>>>> to the binary distribution and mention all of Flink's Maven
>>>> dependencies as well" can be dropped because we don't have them anymore.
>>>> 
>>>> I came to the same conclusion on dependencies. I used
>>>> tools/list_deps.py and diff'ed the output for 1.5 and 1.6, that's
>>>> probably what Chesnay also did ... :-)
>>>> 
>>>>> On 8. Aug 2018, at 14:43, Chesnay Schepler <chesnay@xxxxxxxxxx> wrote:
>>>>> 
>>>>> +1
>>>>> 
>>>>> - verified source release contains no binaries
>>>>> - verified correct versions in source release
>>>>> - verified compilation, tests and E2E-tests pass (on travis)
>>>>> - verified checksum and gpg files
>>>>> 
>>>>> New dependencies (excluding dependencies where we simply depend on a
>>>>> different version now):
>>>>>     Apache licensed:
>>>>>         io.confluent:common-utils:jar:3.3.1
>>>>>         io.confluent:kafka-schema-registry-client:jar:3.3.1
>>>>>         io.prometheus:simpleclient_pushgateway:jar:0.3.0
>>>>>         various Apache Nifi dependencies
>>>>>         various Apache Parquet dependencies
>>>>>         various ElasticSearch dependencies
>>>>>     CDDL:
>>>>>         javax.ws.rs:javax.ws.rs-api:jar:2.1
>>>>>     Bouncycastle (MIT-like):
>>>>>         org.bouncycastle:bcpkix-jdk15on:jar:1.59
>>>>>         org.bouncycastle:bcprov-jdk15on:jar:1.59
>>>>>     MIT:
>>>>>         org.projectlombok:lombok:jar:1.16.20
>>>>> 
>>>>> On 08.08.2018 13:28, Till Rohrmann wrote:
>>>>>> Thanks for reporting these problems Chesnay. The usage string in
>>>>>> `standalone-job.sh` is out dated and should be updated. The same
>>>>>> applies to
>>>>>> the typo.
>>>>>> 
>>>>>> When calling `standalone-job.sh start --job-classname foobar.Job`
>>>>>> please
>>>>>> make sure that the user code jar is contained in the classpath (e.g.
>>>>>> putting the jar in the lib directory). Documenting this behaviour
>>>>>> is part
>>>>>> of the pending issue FLINK-10001.
>>>>>> 
>>>>>> We should fix all of these issues. They are, however, no release
>>>>>> blockers.
>>>>>> 
>>>>>> Cheers,
>>>>>> Till
>>>>>> 
>>>>>> On Wed, Aug 8, 2018 at 11:31 AM Chesnay Schepler
>>>>>> <chesnay@xxxxxxxxxx> wrote:
>>>>>> 
>>>>>>> I found some issues with the standalone-job.sh script.
>>>>>>> 
>>>>>>> I ran "./bin/standalone-job.sh start" as described by the usage
>>>>>>> string.
>>>>>>> 
>>>>>>>      2018-08-08 09:22:34,385 ERROR
>>>>>>>      org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -
>>>>>>>      Could not parse command line arguments [--configDir,
>>>>>>>      /home/zento/svn/flink-1.6.0/flink-1.6.0/conf].
>>>>>>>      org.apache.flink.runtime.entrypoint.FlinkParseException:
>>>>>>> Failed to
>>>>>>>      parse the command line arguments.
>>>>>>>               at
>>>>>>> 
>>>>>>> org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:52)
>>>>>>> 
>>>>>>>               at
>>>>>>> 
>>>>>>> org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.main(StandaloneJobClusterEntryPoint.java:143)
>>>>>>> 
>>>>>>>      Caused by: org.apache.commons.cli.MissingOptionException:
>>>>>>> Missing
>>>>>>>      required option: j
>>>>>>>               at
>>>>>>> 
>>>>>>> org.apache.commons.cli.DefaultParser.checkRequiredOptions(DefaultParser.java:199)
>>>>>>> 
>>>>>>>               at
>>>>>>>     org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:130)
>>>>>>>               at
>>>>>>>     org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:81)
>>>>>>>               at
>>>>>>> 
>>>>>>> org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:50)
>>>>>>> 
>>>>>>>              ... 1 more
>>>>>>> 
>>>>>>> The script should fail earlier if no jar is provided, with a better
>>>>>>> error message.
>>>>>>> It is also undocumented, and the usage instructions don't appear
>>>>>>> correct.
>>>>>>> 
>>>>>>> Passing a jar with the -j option leads to a ClassNotFoundException:
>>>>>>> "./bin/standalone-job.sh start -j examples/streaming/WordCount.jar"
>>>>>>> 
>>>>>>>      2018-08-08 09:26:30,562 ERROR
>>>>>>>      org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -
>>>>>>>      Cluster initialization failed.
>>>>>>>      java.lang.reflect.UndeclaredThrowableException
>>>>>>>               at
>>>>>>> 
>>>>>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1854)
>>>>>>> 
>>>>>>>               at
>>>>>>> 
>>>>>>> org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
>>>>>>> 
>>>>>>>               at
>>>>>>> 
>>>>>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:189)
>>>>>>> 
>>>>>>>               at
>>>>>>> 
>>>>>>> org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.main(StandaloneJobClusterEntryPoint.java:158)
>>>>>>> 
>>>>>>>      Caused by: org.apache.flink.util.FlinkException: Could not
>>>>>>> load the
>>>>>>>      provied entrypoint class.
>>>>>>>               at
>>>>>>> 
>>>>>>> org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.createPackagedProgram(StandaloneJobClusterEntryPoint.java:92)
>>>>>>> 
>>>>>>>               at
>>>>>>> 
>>>>>>> org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.retrieveJobGraph(StandaloneJobClusterEntryPoint.java:75)
>>>>>>> 
>>>>>>>               at
>>>>>>> 
>>>>>>> org.apache.flink.runtime.entrypoint.JobClusterEntrypoint.createDispatcher(JobClusterEntrypoint.java:107)
>>>>>>> 
>>>>>>>               at
>>>>>>> 
>>>>>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startClusterComponents(ClusterEntrypoint.java:353)
>>>>>>> 
>>>>>>>               at
>>>>>>> 
>>>>>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runCluster(ClusterEntrypoint.java:232)
>>>>>>> 
>>>>>>>               at
>>>>>>> 
>>>>>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.lambda$startCluster$0(ClusterEntrypoint.java:190)
>>>>>>> 
>>>>>>>               at
>>>>>>> java.security.AccessController.doPrivileged(Native Method)
>>>>>>>               at javax.security.auth.Subject.doAs(Subject.java:422)
>>>>>>>               at
>>>>>>> 
>>>>>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836)
>>>>>>> 
>>>>>>>               ... 3 more
>>>>>>>      Caused by: java.lang.ClassNotFoundException:
>>>>>>>      examples/streaming/WordCount.jar
>>>>>>>               at
>>>>>>> java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>>>>>>>               at
>>>>>>> java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>>>>>>>               at
>>>>>>>      sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:338)
>>>>>>>               at
>>>>>>> java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>>>>>>>               at
>>>>>>> 
>>>>>>> org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.createPackagedProgram(StandaloneJobClusterEntryPoint.java:89)
>>>>>>> 
>>>>>>>               ... 11 more
>>>>>>> 
>>>>>>> So this seems to not work at all, but maybe I'm using it wrong?
>>>>>>> 
>>>>>>> (There's also typo in "Could not load the provied entrypoint class")
>>>>>>> 
>>>>>>> On 08.08.2018 10:33, Piotr Nowojski wrote:
>>>>>>>> +1 from my side
>>>>>>>> 
>>>>>>>> I’ve spent some time playing around with various examples (batching,
>>>>>>> streaming and SQL) on EMR 6 nodes cluster with yarn deployment, with
>>>>>>> different configuration options (number of task
>>>>>>> managers/memory/Flip6/credit base flow control/metrics) and
>>>>>>> everything
>>>>>>> looks now fine (after fixing
>>>>>>> https://issues.apache.org/jira/browse/FLINK-9969 <
>>>>>>> https://issues.apache.org/jira/browse/FLINK-9969> ).
>>>>>>>> Piotrek
>>>>>>>> 
>>>>>>>>> On 7 Aug 2018, at 17:17, Till Rohrmann <trohrmann@xxxxxxxxxx>
>>>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>> Hi everyone,
>>>>>>>>> Please review and vote on the release candidate #4 for the version
>>>>>>> 1.6.0,
>>>>>>>>> as follows:
>>>>>>>>> [ ] +1, Approve the release
>>>>>>>>> [ ] -1, Do not approve the release (please provide specific
>>>>>>>>> comments)
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> The complete staging area is available for your review, which
>>>>>>>>> includes:
>>>>>>>>> * JIRA release notes [1],
>>>>>>>>> * the official Apache source release and binary convenience
>>>>>>>>> releases to
>>>>>>> be
>>>>>>>>> deployed to dist.apache.org [2], which are signed with the key with
>>>>>>>>> fingerprint 1F302569A96CFFD5 [3],
>>>>>>>>> * all artifacts to be deployed to the Maven Central Repository [4],
>>>>>>>>> * source code tag "release-1.6.0-rc4" [5],
>>>>>>>>> * website pull request listing the new release and adding
>>>>>>>>> announcement
>>>>>>> blog
>>>>>>>>> post [6].
>>>>>>>>> 
>>>>>>>>> Please use this document for coordinating testing efforts: [7]
>>>>>>>>> 
>>>>>>>>> The vote will be shortened since we only adde a minor fix on top
>>>>>>>>> of the
>>>>>>> RC
>>>>>>>>> 3. It will close on Wednesday 6:30pm CET. It is adopted by majority
>>>>>>>>> approval, with at least 3 PMC affirmative votes.
>>>>>>>>> 
>>>>>>>>> Thanks,
>>>>>>>>> Your friendly Release Manager
>>>>>>>>> 
>>>>>>>>> [1]
>>>>>>>>> 
>>>>>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12342760
>>>>>>> 
>>>>>>>>> [2] https://dist.apache.org/repos/dist/dev/flink/flink-1.6.0/
>>>>>>>>> [3] https://dist.apache.org/repos/dist/release/flink/KEYS
>>>>>>>>> [4]
>>>>>>> https://repository.apache.org/content/repositories/orgapacheflink-1178
>>>>>>> 
>>>>>>>>> [5] https://github.com/apache/flink/tree/release-1.6.0-rc4
>>>>>>>>> [6] https://github.com/apache/flink-web/pull/117
>>>>>>>>> [7]
>>>>>>>>> 
>>>>>>> https://docs.google.com/document/d/1upBFZQ7tbaSkYvDiLqfUFXKg8Xxs-lVheEfb66e4jpo/edit?usp=sharing
>>>>>>> 
>>>>>>>>> Pro-tip: you can create a settings.xml file with these contents:
>>>>>>>>> 
>>>>>>>>> <settings>
>>>>>>>>> <activeProfiles>
>>>>>>>>>    <activeProfile>flink-1.6.0</activeProfile>
>>>>>>>>> </activeProfiles>
>>>>>>>>> <profiles>
>>>>>>>>>    <profile>
>>>>>>>>>      <id>flink-1.6.0</id>
>>>>>>>>>      <repositories>
>>>>>>>>>        <repository>
>>>>>>>>>          <id>flink-1.6.0</id>
>>>>>>>>>          <url>
>>>>>>>>> 
>>>>>>>>> https://repository.apache.org/content/repositories/orgapacheflink-1178/
>>>>>>>>> 
>>>>>>>>>          </url>
>>>>>>>>>        </repository>
>>>>>>>>>        <repository>
>>>>>>>>>          <id>archetype</id>
>>>>>>>>>          <url>
>>>>>>>>> 
>>>>>>>>> https://repository.apache.org/content/repositories/orgapacheflink-1178/
>>>>>>>>> 
>>>>>>>>>          </url>
>>>>>>>>>        </repository>
>>>>>>>>>      </repositories>
>>>>>>>>>    </profile>
>>>>>>>>> </profiles>
>>>>>>>>> </settings>
>>>>>>>>> 
>>>>>>>>> And reference that in you maven commands via --settings
>>>>>>>>> path/to/settings.xml. This is useful for creating a quickstart
>>>>>>>>> based on
>>>>>>> the
>>>>>>>>> staged release and for building against the staged jars.
>>> 
>