osdir.com

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [VOTE] Community Examples Repository


Hi guys,

For this kind of discussion, I would prefer to avoid Google Doc and
directly put the point/proposal on the mailing list.

It's easier for the community to follow.

The statement is more for 3 because it's more convenient for users to
easily find the examples and include in the distribution.

Regards
JB

On 08/08/2018 23:25, Charles Chen wrote:
> It looks like the main claim is that 1 and 2 have the benefit of
> increasing visibility for examples on the Beam site.  I agree with
> Robert's comments on the doc which claim that this is orthogonal to
> whether a separate repository is created (the comments are unresolved:
> https://docs.google.com/a/google.com/document/d/1vhcKJlP0qH1C7NZPDjohT2PUbOD-k71avv1CjEYapdw/edit?disco=AAAABzifZxY).
> 
> I would add that the maintenance and testing burden has not been
> adequately addressed in the proposal (i.e. are we creating new Jenkins
> jobs?; will postcommits on the main Beam repo run examples tests?; are
> we releasing artifacts--if so, is this together with the main package or
> separately in new packages?).  If we go with the half-way solution in
> (2), there is also the issue of where the threshold is--for example, if
> a user-contributed example is particularly useful, do we move it to the
> main repo?
> 
> On Wed, Aug 8, 2018 at 1:35 PM Griselda Cuevas <gris@xxxxxxxxxx
> <mailto:gris@xxxxxxxxxx>> wrote:
> 
>     I'd vote for 2. 
> 
>     Giving independence to an example repository and creating the right
>     infrastructure to maintain them will give visibility to the efforts
>     our users are creating to solve their uses cases with Beam. I also
>     want to make the process of sharing common work more easily. 
> 
>     Re:The examples that will remain in core, I agree that it's crucial
>     to keep some examples for testing. 
> 
> 
>     On Wed, 8 Aug 2018 at 11:44, Lukasz Cwik <lcwik@xxxxxxxxxx
>     <mailto:lcwik@xxxxxxxxxx>> wrote:
> 
>         I would vote for 3.
> 
>         My reasoning is that Java has a good mechanism to get a
>         starter/example project going by using the the maven archetypes
>         already. Our quickstart guide for Apache Beam for the Java SDK
>         already covers generating the examples archetype.
>         We could point users to the starter project at the end of the
>         java quickstart.
> 
>         If python/go have a similar mechanism that is commonly used, I
>         would go with those over creating a separate repo for examples
>         and adding the maintenance burden involved.
> 
> 
> 
>         On Wed, Aug 8, 2018 at 11:01 AM Rui Wang <ruwang@xxxxxxxxxx
>         <mailto:ruwang@xxxxxxxxxx>> wrote:
> 
>             2 - examples that rely on experimental API can still stay in
>             where they are because such examples could be changed.
> 
>             -Rui
> 
>             On Wed, Aug 8, 2018 at 10:52 AM Charles Chen <ccy@xxxxxxxxxx
>             <mailto:ccy@xxxxxxxxxx>> wrote:
> 
>                 3 - We benefit from increased test coverage by having
>                 examples together with the rest of the code.  As Robert
>                 mentions in the doc, hosting the Beam examples in the
>                 main repository is the best way to keep the examples
>                 visible, tested and maintained.  Given that we recently
>                 moved to a single repository for the website since that
>                 previously caused a lot of pain, it makes sense to be
>                 consistent here.
> 
>                 On Wed, Aug 8, 2018 at 10:27 AM Ahmet Altay
>                 <altay@xxxxxxxxxx <mailto:altay@xxxxxxxxxx>> wrote:
> 
>                     2 - Similar to Huygaa, I see value in keeping a core
>                     set of examples tested and maintained against head.
>                     At the same time I understand the value of a growing
>                     set of community grown examples that are targeted
>                     against a pre-defined versions of Beam and not
>                     necessarily updated at every release.
> 
>                     On Wed, Aug 8, 2018 at 10:22 AM, Huygaa Batsaikhan
>                     <batbat@xxxxxxxxxx <mailto:batbat@xxxxxxxxxx>> wrote:
> 
>                         2 - I like the idea of having a separate repo
>                         where we can have more freedom to check in
>                         examples. However, we benefit from having
>                         immediate core examples in Beam for testing
>                         purposes.
> 
>                         On Wed, Aug 8, 2018 at 9:38 AM David Cavazos
>                         <dcavazos@xxxxxxxxxx
>                         <mailto:dcavazos@xxxxxxxxxx>> wrote:
> 
>                             Hi everyone!
> 
>                             We discussed several options as well as some
>                             of the implications of each option. Please
>                             vote for your favorite option, feel free to
>                             back it up with any reasons that make you
>                             feel that way.
> 
>                             1) Move *all* samples to a *new
>                             *examples*repository*
>                             2) Move *some* samples to a *new
>                             *examples*repository*
>                             3) Leave samples where they are
> 
>                             Some implications to creating a new repository:
>                             - Every example would be independent from
>                             every other example, so tests can be run in
>                             parallel
>                             - Examples would now show how to use Beam
>                             /externally/
>                             - The examples repository would need a
>                             testing infrastructure
>                             - Decoupling makes examples easier to test
>                             on different versions
>                             - Easier to copy-paste an existing example
>                             and start from there, almost like a template
>                             - Smaller size for the core Beam library
>                             - Two different repositories to maintain
>                             - Versioning could mirror Beam's current version
> 
>                             Link to proposal
>                             <https://docs.google.com/document/d/1vhcKJlP0qH1C7NZPDjohT2PUbOD-k71avv1CjEYapdw/edit?usp=sharing>
> 
> 

-- 
Jean-Baptiste Onofré
jbonofre@xxxxxxxxxx
http://blog.nanthrax.net
Talend - http://www.talend.com