[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: What is Jenkins job "Portable_Python" in PreCommit?

Makes sense. 
I would like to keep this a separate precommit for a few weeks to make sure that it gets enough bake time and its not flaky.
We should merge it with ":pythonPreCommit" in a few weeks. This takes 5 min on avg so it should not add to the overall time for precommit results.

On Fri, Nov 30, 2018 at 2:09 PM Mark Liu <markliu@xxxxxxxxxx> wrote:
Thank you for improving the test coverage. That's very helpful! 

On the other hand, I have a PR in review to make time-consuming integration tests to run parallel in PreCommit. This could benefits tests like portable_python to be enabled in `:pythonPreCommit` without creating a Jenkins branch and run seed job for testing. However, if we have other reasons, then it may not applied.

Would love to hear thoughts on this.


On Fri, Nov 30, 2018 at 12:27 PM Thomas Weise <thw@xxxxxxxxxx> wrote:
This is a very valuable addition! Given the execution times of Python and Java pre-commit, having this run in parallel should not be an issue.


On Fri, Nov 30, 2018 at 11:01 AM Ankur Goenka <goenka@xxxxxxxxxx> wrote:
We added new precommit which tests a wordcount pipeline on portable flink runner. 
In recent time we missed catching some obvious issues which broke potability and could have been caught by this test.
The current test is fairly light weight and executes in ~5min which seems to be reasonable for a precommit test.
This precommit is added just yesterday so we don't actively track it yet. However please let me know if you see any issues with this precommit.

On Fri, Nov 30, 2018 at 3:49 AM Maximilian Michels <mxm@xxxxxxxxxx> wrote:
This was merged with https://github.com/apache/beam/pull/6954.

Eventually we want to run a portable WordCount on PreCommit. We will do
some more testing on Jenkins before it becomes an official PreCommit task.


On 29.11.18 19:03, Mark Liu wrote:
> ah, thanks Boyuan! Probably I created the PR in a bad timing. Looks like
> a new PR will fix it.
> On Thu, Nov 29, 2018 at 9:50 AM Boyuan Zhang <boyuanz@xxxxxxxxxx
> <mailto:boyuanz@xxxxxxxxxx>> wrote:
>     I believe it's in this pending PR:
>     https://github.com/apache/beam/pull/7157.
>     On Thu, Nov 29, 2018 at 8:36 AM Mark Liu <markliu@xxxxxxxxxx
>     <mailto:markliu@xxxxxxxxxx>> wrote:
>         Hi guys,
>         I made some changes <https://github.com/apache/beam/pull/7146>
>         to Python PreCommit Gradle and then Portable_Python is invoked
>         as a PrecCommit test and failed. However, I can't find where
>         it's defined / generated in Gradle or Jenkins groovy. Does
>         anyone know? My branch is synced to master yesterday.
>         Thanks!
>         Mark