osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [DISCUSS] Publish vendored dependencies independently


In case any of this affects how artifacts are published in general, please make sure that publishing to 3rd party repo continues to work.

For example: ./gradlew :beam-runners-flink_2.11-job-server:publish -PisRelease -PnoSigning -PdistMgmtSnapshotsUrl=https://mycustomrepo/libs-release

Thanks,
Thomas


On Thu, Nov 15, 2018 at 11:27 AM Kenneth Knowles <kenn@xxxxxxxxxx> wrote:
Agree on the low bar. We should just make them all 0.x releases to send the right message (don't use, and no compatibility) and not worry as much about bad releases, which we 
would never actually depend on in the project.

QQ: What does the new -P flag do? I was also hoping to eliminate the redundant -PisRelease flag, especially for vendored deps that should really be straight line.

Kenn

On Wed, Nov 14, 2018 at 12:38 PM Lukasz Cwik <lcwik@xxxxxxxxxx> wrote:
Its a small hassle but could be checked in with some changes, my example commit was so that people could try it out as is.

I'll work towards getting it checked in and then start a release for gRPC and guava.

On Wed, Nov 14, 2018 at 11:45 AM Scott Wegner <scott@xxxxxxxxxx> wrote:
Thanks for pushing this forward Luke.

My understanding is that these vendored grpc artifacts will only be consumed directly by Beam internal components (as opposed to Beam user projects). So there should be a fairly low bar for publishing them. But perhaps we should have some short checklist for releasing them for consistency.

One item I would suggest for such a checklist would be to publish artifacts from checked-in apache/beam sources and then tag the release commit. Is it possible to get your changes merged in first, or is there a chicken-and-egg problem that artifacts need to be published and available for consumption?

On Wed, Nov 14, 2018 at 10:51 AM Lukasz Cwik <lcwik@xxxxxxxxxx> wrote:
Note, I could also release the vendored version of guava 20 in preparation for us to start consuming it. Any concerns?

On Tue, Nov 13, 2018 at 3:59 PM Lukasz Cwik <lcwik@xxxxxxxxxx> wrote:
I have made some incremental progress on this and wanted to release our first vendored dependency of gRPC 1.13.1 since I was able to fix a good number of the import/code completion errors that Intellij was experiencing. I have published an example of what the jar/pom looks like in the Apache Staging repo:
https://repository.apache.org/content/groups/snapshots/org/apache/beam/beam-vendor-grpc-1_13_1/

You can also checkout[1] and from a clean workspace run:
./gradlew :beam-vendor-grpc-1_13_1:publishToMavenLocal -PisRelease -PvendoredDependenciesOnly
which will build a vendored version of gRPC that is published to your local maven repository. All the projects that depended on the gradle beam-vendor-grpc-1_13_1 project are now pointing at the Maven artifact org.apache.beam:beam-vendor-grpc-1_13_1:0.1

I was planning to follow the Apache Beam release process but only for this specific artifact and start a vote thread if there aren't any concerns.



On Thu, Oct 25, 2018 at 10:59 AM Lukasz Cwik <lcwik@xxxxxxxxxx> wrote:
Thats a good point Thomas, hadn't considered the lib/ case. I also am recommending what Thomas is suggesting as well.

On Thu, Oct 25, 2018 at 10:52 AM Maximilian Michels <mxm@xxxxxxxxxx> wrote:
On 25.10.18 19:23, Lukasz Cwik wrote:
>
>
> On Thu, Oct 25, 2018 at 9:59 AM Maximilian Michels <mxm@xxxxxxxxxx
> <mailto:mxm@xxxxxxxxxx>> wrote:
>
>     Question: How would a user end up with the same shaded dependency
>     twice?
>     The shaded dependencies are transitive dependencies of Beam and thus,
>     this shouldn't happen. Is this a safe-guard when running different
>     versions of Beam in the same JVM?
>
>
> What I was referring to was that they aren't exactly the same dependency
> but slightly different versions of the same dependency. Since we are
> planning to vendor each dependency and its transitive dependencies as
> part of the same jar, we can have  vendor-A that contains shaded
> transitive-C 1.0 and vendor-B that contains transitive-C 2.0 both with
> different package prefixes. It can be that transitive-C 1.0 and
> transitive-C 2.0 can't be on the same classpath because they can't be
> perfectly shaded due to JNI, java reflection, magical property
> files/strings, ...
>

Ah yes. Get it. Thanks!


--