[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Apache Beam Python Wheels Repository


Travis for building the Python wheels looks fine to me. Many Apache
projects use Travis in addition to Jenkins. Apache is also invested in
Travis [1] to ensure the build capacity is sufficient. In any case, we
could migrate away from Travis if it doesn't work out as expected.

We don't have to file a ticket with JIRA to create a new repository.
There is now: https://gitbox.apache.org/setup/newrepo.html


[1] https://blogs.apache.org/infra/entry/apache_gains_additional_travis_ci

On 15.08.18 14:25, Robert Bradshaw wrote:
> For background, a separate repository that contains reference to Brett's
> Travis/Appveyer build scripts + project configuration information is the
> de facto standard for out of the box support for a wide variety of
> platform-specific Python wheels (aka binary builds). 
> We also need a secure place to publish the artifacts, e.g. supplying
> (encrypted) credentials for Travis to commit the artifacts directly to
> SVN or any other place we can privately push/publicly pull ephemeral
> blobs as the build environments are torn down after the job completed.
> (Not sure what Apache infra has here that could act like a GCS bucket.)
> On Wed, Aug 8, 2018, 2:23 AM Boyuan Zhang <boyuanz@xxxxxxxxxx
> <mailto:boyuanz@xxxxxxxxxx>> wrote:
>     Hey Pablo,
>     For windows, I think Travis doesn't have a plan coming soon.
>     But  Matthew Brett's build scripts
>     <https://github.com/matthew-brett/multibuild> actually supports
>     windows platform in Appveyor <https://ci.appveyor.com/>, which we
>     doesn't have chance to explore more.
>     Boyuan
>     On Tue, Aug 7, 2018 at 4:50 PM Pablo Estrada <pabloem@xxxxxxxxxx
>     <mailto:pabloem@xxxxxxxxxx>> wrote:
>         Hi Boyuan,
>         I think this is reasonable. I remember Robert supported this
>         approach as well, and it is quite simple to support this. I
>         am +1 on this.
>         Do you know at all if there are any plans from Travis to support
>         windows?
>         Best
>         -P.
>         On Tue, Aug 7, 2018 at 2:04 PM Boyuan Zhang <boyuanz@xxxxxxxxxx
>         <mailto:boyuanz@xxxxxxxxxx>> wrote:
>             Thanks for Davor's suggestions. Following content discusses
>             more about why we need a new repository. Hope they would be
>             helpful!
>             Purpose of this proposal
>             - Facilitate building python wheels against multi-os
>             Why a new repository is needed
>             - Currently we chose Matthew Brett's build scripts
>             <https://github.com/matthew-brett/multibuild> build Linux
>             and macOS python wheels.
>             - This build tool uses travis <https://travis-ci.com/> as
>             Linux and maxOS build platform, which explains why we cannot
>             use Jenkins.
>             - In order to utilize this build tool based on their guide
>             <https://github.com/matthew-brett/multibuild/blob/devel/README.rst>,
>             we need to use a repo wrapper.
>             Alternatives
>             - Unfortunately, I didn't find any alternatives at this moment.
>             Boyuan Zhang
>             On Sat, Aug 4, 2018 at 9:50 AM Davor Bonaci
>             <davor@xxxxxxxxxx <mailto:davor@xxxxxxxxxx>> wrote:
>                 New repository is not a ticket, it is a self-serve thing.
>                 That said, you probably want to develop the proposal a
>                 bit further, understanding/educating others about the
>                 benefits of what you are proposing, any alternatives,
>                 why a repository is needed, why the sample repository
>                 has Travis CI when everything else is on Jenkins, how
>                 this fits into other decisions about repository
>                 management, and so on. Anything can be done, of course,
>                 but I'd suggest developing (or communicating, or
>                 educating) a bit more.
>                 (I'm fine with any approach.)
>                 On Fri, Aug 3, 2018 at 3:29 PM, Ahmet Altay
>                 <altay@xxxxxxxxxx <mailto:altay@xxxxxxxxxx>> wrote:
>                     This LGTM, also greatly simplifies the creation of
>                     wheel files for multiple platforms.
>                     I can file an INFRA ticket to create a new repo to
>                     host wheel setup. Does anybody have experience with
>                     setting up a new repo similar to this?
>                     Ahmet
>                     On Fri, Aug 3, 2018 at 1:16 PM, Boyuan Zhang
>                     <boyuanz@xxxxxxxxxx <mailto:boyuanz@xxxxxxxxxx>> wrote:
>                         Hey all,
>                         I'm Boyuan Zhang from Google Dataflow Team,
>                         currently helping Release Manager(Pablo Estrada)
>                         with 2.6.0 release. Since Beam decided to
>                         release python wheels since 2.5.0, we need to
>                         create a wrapper repository(sample repo
>                         <https://github.com/boyuanzz/apache-beam-wheels>) under
>                         apache to build and stage released python wheels
>                         for each release. Anyone can help to create this
>                         repository?
>                         Thanks for all your help! Happy Friday~
>                         Boyuan Zhang
>         -- 
>         Got feedback? go/pabloem-feedback
>         <https://goto.google.com/pabloem-feedback>