OSDir


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [VOTE] Community Examples Repository


I'd also vote for 3: I don't see much added value in separating the repos and I see much additional effort to be done in maintaining extra repo(s) (updating examples when new version of beam sdk comes out) and their infrastructure (jenkins, etc). What Lukasz Cwik said about mvn archetypes and how easy the examples can be to get starter examples from a common repo only strengthens my opinion. 

Regarding 2: I think it's not good to have some official examples here and some there - IMO it can make a false impression (user experience) that some examples are less important than the others. Maybe a good idea is to encourage users to share their (independent, non official) examples and create a list of such on the beam site instead of 2?

Łukasz

czw., 9 sie 2018 o 11:35 Alexey Romanenko <aromanenko.dev@xxxxxxxxx> napisał(a):
3 - I agree with JB, Charles and Lukasz arguments above saying why we need to have examples and main code in the same repository (+ website code base will move there soon). I don’t see any huge benefits to have examples aside and, at the same time, it will bring additional complexity and burden for project support.  

On 9 Aug 2018, at 08:18, Jean-Baptiste Onofré <jb@xxxxxxxxxxxx> wrote:

Hi guys,

For this kind of discussion, I would prefer to avoid Google Doc and
directly put the point/proposal on the mailing list.

It's easier for the community to follow.

The statement is more for 3 because it's more convenient for users to
easily find the examples and include in the distribution.

Regards
JB

On 08/08/2018 23:25, Charles Chen wrote:
It looks like the main claim is that 1 and 2 have the benefit of
increasing visibility for examples on the Beam site.  I agree with
Robert's comments on the doc which claim that this is orthogonal to
whether a separate repository is created (the comments are unresolved:
https://docs.google.com/a/google.com/document/d/1vhcKJlP0qH1C7NZPDjohT2PUbOD-k71avv1CjEYapdw/edit?disco=AAAABzifZxY).

I would add that the maintenance and testing burden has not been
adequately addressed in the proposal (i.e. are we creating new Jenkins
jobs?; will postcommits on the main Beam repo run examples tests?; are
we releasing artifacts--if so, is this together with the main package or
separately in new packages?).  If we go with the half-way solution in
(2), there is also the issue of where the threshold is--for example, if
a user-contributed example is particularly useful, do we move it to the
main repo?

On Wed, Aug 8, 2018 at 1:35 PM Griselda Cuevas <gris@xxxxxxxxxx
<mailto:gris@xxxxxxxxxx>> wrote:

   I'd vote for 2. 

   Giving independence to an example repository and creating the right
   infrastructure to maintain them will give visibility to the efforts
   our users are creating to solve their uses cases with Beam. I also
   want to make the process of sharing common work more easily. 

   Re:The examples that will remain in core, I agree that it's crucial
   to keep some examples for testing. 


   On Wed, 8 Aug 2018 at 11:44, Lukasz Cwik <lcwik@xxxxxxxxxx
   <mailto:lcwik@xxxxxxxxxx>> wrote:

       I would vote for 3.

       My reasoning is that Java has a good mechanism to get a
       starter/example project going by using the the maven archetypes
       already. Our quickstart guide for Apache Beam for the Java SDK
       already covers generating the examples archetype.
       We could point users to the starter project at the end of the
       java quickstart.

       If python/go have a similar mechanism that is commonly used, I
       would go with those over creating a separate repo for examples
       and adding the maintenance burden involved.



       On Wed, Aug 8, 2018 at 11:01 AM Rui Wang <ruwang@xxxxxxxxxx
       <mailto:ruwang@xxxxxxxxxx>> wrote:

           2 - examples that rely on experimental API can still stay in
           where they are because such examples could be changed.

           -Rui

           On Wed, Aug 8, 2018 at 10:52 AM Charles Chen <ccy@xxxxxxxxxx
           <mailto:ccy@xxxxxxxxxx>> wrote:

               3 - We benefit from increased test coverage by having
               examples together with the rest of the code.  As Robert
               mentions in the doc, hosting the Beam examples in the
               main repository is the best way to keep the examples
               visible, tested and maintained.  Given that we recently
               moved to a single repository for the website since that
               previously caused a lot of pain, it makes sense to be
               consistent here.

               On Wed, Aug 8, 2018 at 10:27 AM Ahmet Altay
               <altay@xxxxxxxxxx <mailto:altay@xxxxxxxxxx>> wrote:

                   2 - Similar to Huygaa, I see value in keeping a core
                   set of examples tested and maintained against head.
                   At the same time I understand the value of a growing
                   set of community grown examples that are targeted
                   against a pre-defined versions of Beam and not
                   necessarily updated at every release.

                   On Wed, Aug 8, 2018 at 10:22 AM, Huygaa Batsaikhan
                   <batbat@xxxxxxxxxx <mailto:batbat@xxxxxxxxxx>> wrote:

                       2 - I like the idea of having a separate repo
                       where we can have more freedom to check in
                       examples. However, we benefit from having
                       immediate core examples in Beam for testing
                       purposes.

                       On Wed, Aug 8, 2018 at 9:38 AM David Cavazos
                       <dcavazos@xxxxxxxxxx
                       <mailto:dcavazos@xxxxxxxxxx>> wrote:

                           Hi everyone!

                           We discussed several options as well as some
                           of the implications of each option. Please
                           vote for your favorite option, feel free to
                           back it up with any reasons that make you
                           feel that way.

                           1) Move *all* samples to a *new
                           *examples*repository*
                           2) Move *some* samples to a *new
                           *examples*repository*
                           3) Leave samples where they are

                           Some implications to creating a new repository:
                           - Every example would be independent from
                           every other example, so tests can be run in
                           parallel
                           - Examples would now show how to use Beam
                           /externally/
                           - The examples repository would need a
                           testing infrastructure
                           - Decoupling makes examples easier to test
                           on different versions
                           - Easier to copy-paste an existing example
                           and start from there, almost like a template
                           - Smaller size for the core Beam library
                           - Two different repositories to maintain
                           - Versioning could mirror Beam's current version

                           Link to proposal
                           <https://docs.google.com/document/d/1vhcKJlP0qH1C7NZPDjohT2PUbOD-k71avv1CjEYapdw/edit?usp=sharing>



-- 
Jean-Baptiste Onofré
jbonofre@xxxxxxxxxx
http://blog.nanthrax.net
Talend - http://www.talend.com