[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Gandiva snapshot releases

Ohh, just read the thread, sorry!

So crossbow is located here https://github.com/apache/arrow/tree/master/dev/tasks
I suggest to "fork" the python-wheels directory which contains three templated ymls
for osx, win and linux builds. For building on linux something like the following should
be sufficient https://gist.github.com/kszucs/39154876d60c4109ff59b678afd65b19
Then You need another entry in the tasks.yml, for example:
platform: linux
template: gandiva-jars/travis.linux.yml
# arbitrary params which are available from the templated yml
# these are the expected artifacts from the build
- gandiva-SNAPSHOT-{version}.jar

Of course crossbow is wired towards the current packaging requirements, so likely
We need to adjust it to the newly appearing requirements.

Feel free to reach me on gitter @kszucs.
On Oct 4 2018, at 2:02 pm, Wes McKinney <wesmckinn@xxxxxxxxx> wrote:
> hi Praveen,
> Probably the best way to accomplish this is to use our new Crossbow
> infrastructure for task automation on Travis CI and Appveyor rather
> than trying to do all of this within the CI entries. This is how we
> are producing all of our binary artifacts for releases now --
> presumably in future ASF releases, we will want to include a
> platform-independent Gandiva JAR in our release votes, so this all
> needs to end up in Crossbow anyway. The intent is for the Crossbow
> system to take on responsibility for all packaging automation rather
> than using the normal CI for that.
> Krisztian, do you have time to help Praveen and the Gandiva crew with
> this project? This will be an important test to document and improve
> Crossbow for such use cases
> Thanks
> Wes
> On Thu, Oct 4, 2018 at 7:14 AM Praveen Kumar <praveen@xxxxxxxxxx> wrote:
> >
> > Hi Folks,
> > As part of https://issues.apache.org/jira/browse/ARROW-3385, we are
> > planning to perform a snapshot release of the Gandiva Jar on each commit to
> > master. This would be a platform independent jar that contains the core
> > gandiva library and its jni bridge packaged for Mac, Windows and *nix
> > platforms.
> >
> > The current plan is to deploy separate snapshot jars for each OS through
> > entries in the Gandiva CI matrix and then have a combine step that pulls in
> > each OS specific jar and builds a jar that has all the native libraries.
> > This build/deploy would happen only for commits on master branch and not
> > for PR requests
> >
> > Does the plan sound ok (or) please let us know if there is a better way to
> > achieve the same.
> >
> > If it sounds ok, can someone please help with the following
> > 1. It looks like we only do travis builds and not appveyor for master in
> > arrow. Any reason for this?
> > 2. Even if we did appveyor is there a way to sequence the builds. Like wait
> > for appveyor to complete before kicking off travis? Since we would need the
> > dll to be pre-built.
> > 3. Someone would need to configure the credentials to use for the ossrh
> > deployment. The credentials would need access to deploy to org.apache.arrow.
> >
> > Thanks ahead!