[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [VOTE] Community Examples Repository

Charles, I agree with your comments and questions. I want to add one more benefit that was mentioned earlier:

A place for people to contribute example that is not tied to the beam release cycle. Such a repository could be a place for casual contributors to add examples over time.

On Wed, Aug 8, 2018 at 2:25 PM, Charles Chen <ccy@xxxxxxxxxx> wrote:
It looks like the main claim is that 1 and 2 have the benefit of increasing visibility for examples on the Beam site.  I agree with Robert's comments on the doc which claim that this is orthogonal to whether a separate repository is created (the comments are unresolved: https://docs.google.com/a/google.com/document/d/1vhcKJlP0qH1C7NZPDjohT2PUbOD-k71avv1CjEYapdw/edit?disco=AAAABzifZxY).

I would add that the maintenance and testing burden has not been adequately addressed in the proposal (i.e. are we creating new Jenkins jobs?; will postcommits on the main Beam repo run examples tests?; are we releasing artifacts--if so, is this together with the main package or separately in new packages?).  If we go with the half-way solution in (2), there is also the issue of where the threshold is--for example, if a user-contributed example is particularly useful, do we move it to the main repo?

On Wed, Aug 8, 2018 at 1:35 PM Griselda Cuevas <gris@xxxxxxxxxx> wrote:
I'd vote for 2. 

Giving independence to an example repository and creating the right infrastructure to maintain them will give visibility to the efforts our users are creating to solve their uses cases with Beam. I also want to make the process of sharing common work more easily. 

Re:The examples that will remain in core, I agree that it's crucial to keep some examples for testing. 

On Wed, 8 Aug 2018 at 11:44, Lukasz Cwik <lcwik@xxxxxxxxxx> wrote:
I would vote for 3.

My reasoning is that Java has a good mechanism to get a starter/example project going by using the the maven archetypes already. Our quickstart guide for Apache Beam for the Java SDK already covers generating the examples archetype.
We could point users to the starter project at the end of the java quickstart.

If python/go have a similar mechanism that is commonly used, I would go with those over creating a separate repo for examples and adding the maintenance burden involved.

On Wed, Aug 8, 2018 at 11:01 AM Rui Wang <ruwang@xxxxxxxxxx> wrote:
2 - examples that rely on experimental API can still stay in where they are because such examples could be changed.


On Wed, Aug 8, 2018 at 10:52 AM Charles Chen <ccy@xxxxxxxxxx> wrote:
3 - We benefit from increased test coverage by having examples together with the rest of the code.  As Robert mentions in the doc, hosting the Beam examples in the main repository is the best way to keep the examples visible, tested and maintained.  Given that we recently moved to a single repository for the website since that previously caused a lot of pain, it makes sense to be consistent here.

On Wed, Aug 8, 2018 at 10:27 AM Ahmet Altay <altay@xxxxxxxxxx> wrote:
2 - Similar to Huygaa, I see value in keeping a core set of examples tested and maintained against head. At the same time I understand the value of a growing set of community grown examples that are targeted against a pre-defined versions of Beam and not necessarily updated at every release.

On Wed, Aug 8, 2018 at 10:22 AM, Huygaa Batsaikhan <batbat@xxxxxxxxxx> wrote:
2 - I like the idea of having a separate repo where we can have more freedom to check in examples. However, we benefit from having immediate core examples in Beam for testing purposes.

On Wed, Aug 8, 2018 at 9:38 AM David Cavazos <dcavazos@xxxxxxxxxx> wrote:
Hi everyone!

We discussed several options as well as some of the implications of each option. Please vote for your favorite option, feel free to back it up with any reasons that make you feel that way.

1) Move all samples to a new examples repository
2) Move some samples to a new examples repository
3) Leave samples where they are

Some implications to creating a new repository:
- Every example would be independent from every other example, so tests can be run in parallel
- Examples would now show how to use Beam externally
- The examples repository would need a testing infrastructure
- Decoupling makes examples easier to test on different versions
- Easier to copy-paste an existing example and start from there, almost like a template
- Smaller size for the core Beam library
- Two different repositories to maintain
- Versioning could mirror Beam's current version