Thanks for pushing this forward Luke.
My understanding is that these vendored grpc artifacts will only be consumed directly by Beam internal components (as opposed to Beam user projects). So there should be a fairly low bar for publishing them. But perhaps we should have some short checklist for releasing them for consistency.
One item I would suggest for such a checklist would be to publish artifacts from checked-in apache/beam sources and then tag the release commit. Is it possible to get your changes merged in first, or is there a chicken-and-egg problem that artifacts need to be published and available for consumption?
Note, I could also release the vendored version of guava 20 in preparation for us to start consuming it. Any concerns?
I have made some incremental progress on this and wanted to release our first vendored dependency of gRPC 1.13.1 since I was able to fix a good number of the import/code completion errors that Intellij was experiencing. I have published an example of what the jar/pom looks like in the Apache Staging repo:
You can also checkout and from a clean workspace run:
./gradlew :beam-vendor-grpc-1_13_1:publishToMavenLocal -PisRelease -PvendoredDependenciesOnly
which will build a vendored version of gRPC that is published to your local maven repository. All the projects that depended on the gradle beam-vendor-grpc-1_13_1 project are now pointing at the Maven artifact org.apache.beam:beam-vendor-grpc-1_13_1:0.1
I was planning to follow the Apache Beam release process but only for this specific artifact and start a vote thread if there aren't any concerns.
Thats a good point Thomas, hadn't considered the lib/ case. I also am recommending what Thomas is suggesting as well.
On Thu, Oct 25, 2018 at 10:52 AM Maximilian Michels <mxm@xxxxxxxxxx
On 25.10.18 19:23, Lukasz Cwik wrote:
> On Thu, Oct 25, 2018 at 9:59 AM Maximilian Michels <mxm@xxxxxxxxxx
> <mailto:mxm@xxxxxxxxxx>> wrote:
> Question: How would a user end up with the same shaded dependency
> The shaded dependencies are transitive dependencies of Beam and thus,
> this shouldn't happen. Is this a safe-guard when running different
> versions of Beam in the same JVM?
> What I was referring to was that they aren't exactly the same dependency
> but slightly different versions of the same dependency. Since we are
> planning to vendor each dependency and its transitive dependencies as
> part of the same jar, we can have vendor-A that contains shaded
> transitive-C 1.0 and vendor-B that contains transitive-C 2.0 both with
> different package prefixes. It can be that transitive-C 1.0 and
> transitive-C 2.0 can't be on the same classpath because they can't be
> perfectly shaded due to JNI, java reflection, magical property
> files/strings, ...
Ah yes. Get it. Thanks!