osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [VOTE] Release 1.5.1, release candidate #3


Hi Till,

Thanks for reply!

For version 1.4.x when Parallelism > Available task Slots has been selected, Flink throw bellow error immediately as you said

NoResourceAvailableException: Not enough free slots available to run the job
 
but for version 1.5.x there are two different behaviors: Sometimes job ran successfully and sometimes it throw error after 5 minutes with this message:

org.apache.flink.runtime.jobmanager.scheduler.NoResourceAvailableException: Could not allocate all requires slots within timeout of 300000 ms. Slots required: 5, slots allocated: 2


I think it might be a bug and created this ticket to keep track of it
https://issues.apache.org/jira/browse/FLINK-9848 <https://issues.apache.org/jira/browse/FLINK-9848>

Please let me know if you think otherwise.


Cheers,
Yazdan 

> On Jul 12, 2018, at 3:54 AM, Till Rohrmann <trohrmann@xxxxxxxxxx> wrote:
> 
> Hi Yazdan,
> 
> that is the expected behaviour. If the system cannot allocate enough slots
> it will fail with a NoResourceAvailableException after
> the slot.request.timeout has been exceeded.
> 
> Cheers,
> Till
> 
> On Wed, Jul 11, 2018 at 7:44 PM Yaz Sh <yazdanc1@xxxxxxxxx <mailto:yazdanc1@xxxxxxxxx>> wrote:
> 
>> +1
>> 
>> - Verified the signatures for all binary artifacts
>> - Verifies Checksum for all binary packages
>> - Ran local cluster with no error on logs and empty *.out
>> - Stop local cluster  with no error on logs
>> - Ran multiple batch and streaming example via WebUI
>> - Rab multiple batch and streaming examples via CLI
>> - Increase number of task managers and ran examples with Parallelism > 1
>> - Ran WebUI on multiple browsers
>> - Check Example folder for all binary packages
>> 
>> Just an observation:
>> When ran a job with Parallelism > Available task slots intermediately job
>> stay in “Running" status for a very long time and neither finish nor throw
>> any errors.
>> 
>> Please check if someone else can reproduce it.
>> 
>> Cheers,
>> Yazdan
>> 
>> 
>>> On Jul 11, 2018, at 11:21 AM, Till Rohrmann <trohrmann@xxxxxxxxxx>
>> wrote:
>>> 
>>> +1 (binding)
>>> 
>>> - Verified the signatures of all binary artifacts
>>> - Verified that no new dependencies were added for which the LICENSE and
>> NOTICE files need to be adapted.
>>> - Build 1.5.1 from the source artifact
>>> - Run flink-end-to-end tests for 12 hours for the 1.5.1 Hadoop 2.7
>> binary artifact
>>> - Run Jepsen tests for 12 hours for the 1.5.1 Hadoop 2.8 binary artifact
>>> 
>>> Cheers,
>>> Till
>>> 
>>> 
>>> On Wed, Jul 11, 2018 at 9:49 AM Chesnay Schepler <chesnay@xxxxxxxxxx
>> <mailto:chesnay@xxxxxxxxxx <mailto:chesnay@xxxxxxxxxx>>> wrote:
>>> Correction on my part, it does affect all packages.
>>> 
>>> I've also found the cause. To speed up the process I only built modules
>>> that flink-dist depends on (see FLINK-9768). However flink-dist depends
>>> on neither flink-examples-batch nor flink-examples-streaming, yet
>>> happily accesses their target directory. The existing build process only
>>> worked since _by chance_ these 2 modules are built before flink-dist
>>> when doing a complete build.
>>> 
>>> I will rebuild the binaries (I don't think we have to cancel the RC for
>>> this) and open a JIRA to fix the dependencies.
>>> 
>>> On 11.07.2018 09:27, Chesnay Schepler wrote:
>>>> oh, the packages that include hadoop are really missing it...
>>>> 
>>>> On 11.07.2018 09:25, Chesnay Schepler wrote:
>>>>> @Yaz which binary package did you check? I looked into the
>>>>> hadoop-free package and the folder is there.
>>>>> 
>>>>> Did you maybe encounter an error when extracting the package?
>>>>> 
>>>>> On 11.07.2018 05:44, Yaz Sh wrote:
>>>>>> -1
>>>>>> 
>>>>>> ./examples/streaming folder is missing in binary packages
>>>>>> 
>>>>>> 
>>>>>> Cheers,
>>>>>> Yazdan
>>>>>> 
>>>>>>> On Jul 10, 2018, at 9:57 PM, vino yang <yanghua1127@xxxxxxxxx <mailto:yanghua1127@xxxxxxxxx>
>> <mailto:yanghua1127@xxxxxxxxx <mailto:yanghua1127@xxxxxxxxx>>> wrote:
>>>>>>> 
>>>>>>> +1
>>>>>>> reviewed [1], [4] and [6]
>>>>>>> 
>>>>>>> 2018-07-11 3:10 GMT+08:00 Chesnay Schepler <chesnay@xxxxxxxxxx <mailto:chesnay@xxxxxxxxxx>
>> <mailto:chesnay@xxxxxxxxxx <mailto:chesnay@xxxxxxxxxx>>>:
>>>>>>> 
>>>>>>>> Hi everyone,
>>>>>>>> Please review and vote on the release candidate #3 for the version
>>>>>>>> 1.5.1,
>>>>>>>> as follows:
>>>>>>>> [ ] +1, Approve the release
>>>>>>>> [ ] -1, Do not approve the release (please provide specific
>> comments)
>>>>>>>> 
>>>>>>>> 
>>>>>>>> The complete staging area is available for your review, which
>>>>>>>> includes:
>>>>>>>> * JIRA release notes [1],
>>>>>>>> * the official Apache source release and binary convenience
>>>>>>>> releases to be
>>>>>>>> deployed to dist.apache.org <http://dist.apache.org/> <http://dist.apache.org/ <http://dist.apache.org/>> [2], which
>> are signed with the key with
>>>>>>>> fingerprint 11D464BA [3],
>>>>>>>> * all artifacts to be deployed to the Maven Central Repository [4],
>>>>>>>> * source code tag "release-1.5.1-rc3" [5],
>>>>>>>> * website pull request listing the new release and adding
>>>>>>>> announcement
>>>>>>>> blog post [6].
>>>>>>>> 
>>>>>>>> This RC is a slightly modified version of the previous RC, with
>> most
>>>>>>>> release testing being applicable to both release candidates. The
>>>>>>>> minimum
>>>>>>>> voting duration will hence be reduced to 24 hours. It is adopted by
>>>>>>>> majority approval, with at least 3 PMC affirmative votes.
>>>>>>>> 
>>>>>>>> Thanks,
>>>>>>>> Chesnay
>>>>>>>> 
>>>>>>>> [1] https://issues.apache.org/jira/secure/ReleaseNote.jspa?proje <https://issues.apache.org/jira/secure/ReleaseNote.jspa?proje> <
>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?proje <https://issues.apache.org/jira/secure/ReleaseNote.jspa?proje>>
>>>>>>>> ctId=12315522&version=12343053
>>>>>>>> [2] https://dist.apache.org/repos/dist/dev/flink/1.5.1/ <https://dist.apache.org/repos/dist/dev/flink/1.5.1/> <
>> https://dist.apache.org/repos/dist/dev/flink/1.5.1/ <https://dist.apache.org/repos/dist/dev/flink/1.5.1/>>
>>>>>>>> [3] https://dist.apache.org/repos/dist/release/flink/KEYS <https://dist.apache.org/repos/dist/release/flink/KEYS> <
>> https://dist.apache.org/repos/dist/release/flink/KEYS <https://dist.apache.org/repos/dist/release/flink/KEYS>>
>>>>>>>> [4]
>>>>>>>> 
>> https://repository.apache.org/content/repositories/orgapacheflink-1171 <https://repository.apache.org/content/repositories/orgapacheflink-1171> <
>> https://repository.apache.org/content/repositories/orgapacheflink-1171 <https://repository.apache.org/content/repositories/orgapacheflink-1171>>
>>>>>>>> 
>>>>>>>> [5] https://git-wip-us.apache.org/repos/asf?p=flink.git;a=tag;h= <https://git-wip-us.apache.org/repos/asf?p=flink.git;a=tag;h=> <
>> https://git-wip-us.apache.org/repos/asf?p=flink.git;a=tag;h= <https://git-wip-us.apache.org/repos/asf?p=flink.git;a=tag;h=>>
>>>>>>>> refs/tags/release-1.5.1-rc3
>>>>>>>> [6] https://github.com/apache/flink-web/pull/112 <https://github.com/apache/flink-web/pull/112> <
>> https://github.com/apache/flink-web/pull/112 <https://github.com/apache/flink-web/pull/112>>