[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Apache Beam Python Wheels Repository

For background, a separate repository that contains reference to Brett's Travis/Appveyer build scripts + project configuration information is the de facto standard for out of the box support for a wide variety of platform-specific Python wheels (aka binary builds). 

We also need a secure place to publish the artifacts, e.g. supplying (encrypted) credentials for Travis to commit the artifacts directly to SVN or any other place we can privately push/publicly pull ephemeral blobs as the build environments are torn down after the job completed. (Not sure what Apache infra has here that could act like a GCS bucket.)

On Wed, Aug 8, 2018, 2:23 AM Boyuan Zhang <boyuanz@xxxxxxxxxx> wrote:
Hey Pablo,

For windows, I think Travis doesn't have a plan coming soon. But  Matthew Brett's build scripts actually supports windows platform in Appveyor, which we doesn't have chance to explore more.


On Tue, Aug 7, 2018 at 4:50 PM Pablo Estrada <pabloem@xxxxxxxxxx> wrote:
Hi Boyuan,
I think this is reasonable. I remember Robert supported this approach as well, and it is quite simple to support this. I am +1 on this.

Do you know at all if there are any plans from Travis to support windows?

On Tue, Aug 7, 2018 at 2:04 PM Boyuan Zhang <boyuanz@xxxxxxxxxx> wrote:
Thanks for Davor's suggestions. Following content discusses more about why we need a new repository. Hope they would be helpful!

Purpose of this proposal
- Facilitate building python wheels against multi-os

Why a new repository is needed
- Currently we chose Matthew Brett's build scripts build Linux and macOS python wheels.
- This build tool uses travis as Linux and maxOS build platform, which explains why we cannot use Jenkins.
- In order to utilize this build tool based on their guide, we need to use a repo wrapper.

- Unfortunately, I didn't find any alternatives at this moment.

Boyuan Zhang

On Sat, Aug 4, 2018 at 9:50 AM Davor Bonaci <davor@xxxxxxxxxx> wrote:
New repository is not a ticket, it is a self-serve thing.

That said, you probably want to develop the proposal a bit further, understanding/educating others about the benefits of what you are proposing, any alternatives, why a repository is needed, why the sample repository has Travis CI when everything else is on Jenkins, how this fits into other decisions about repository management, and so on. Anything can be done, of course, but I'd suggest developing (or communicating, or educating) a bit more.

(I'm fine with any approach.)

On Fri, Aug 3, 2018 at 3:29 PM, Ahmet Altay <altay@xxxxxxxxxx> wrote:
This LGTM, also greatly simplifies the creation of wheel files for multiple platforms.

I can file an INFRA ticket to create a new repo to host wheel setup. Does anybody have experience with setting up a new repo similar to this?


On Fri, Aug 3, 2018 at 1:16 PM, Boyuan Zhang <boyuanz@xxxxxxxxxx> wrote:
Hey all,

I'm Boyuan Zhang from Google Dataflow Team, currently helping Release Manager(Pablo Estrada) with 2.6.0 release. Since Beam decided to release python wheels since 2.5.0, we need to create a wrapper repository(sample repo) under apache to build and stage released python wheels for each release. Anyone can help to create this repository?

Thanks for all your help! Happy Friday~

Boyuan Zhang

Got feedback? go/pabloem-feedback