osdir.com

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Community Examples Repository


Agreed with Rui. We could also add more SQL examples (like, different IOs ) for everyone to get started with.

Best,
Kai

On 2018/08/02 17:40:32, Rui Wang <ruwang@xxxxxxxxxx> wrote: 
> I might miss it: are examples to be moved including those which are not
> under example/? For example there are some BeamSQL examples in
> org/apache/beam/sdk/extensions/sql/example
> <https://github.com/apache/beam/tree/master/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/example>
> .
> 
> 
> It's better to keep BeamSQL examples in where it is because related API
> might still change.
> 
> -Rui
> 
> On Thu, Aug 2, 2018 at 8:58 AM Ahmet Altay <altay@xxxxxxxxxx> wrote:
> 
> > Robert, I agree with you in general. However there is also a second
> > motivation. There is an increase in new PRs that are coming to add new
> > examples. This is great however the core code (including distributions) is
> > not a great place to host such examples. An examples repo would help in
> > this case. It could also serve as an entry point for new contributors.
> >
> >
> >
> > On Thu, Aug 2, 2018 at 12:40 AM, Robert Bradshaw <robertwb@xxxxxxxxxx>
> > wrote:
> >
> >> I have to admit I'm generally -1 on moving examples to a separate
> >> repository. In particular, I think it would actually inhibit the
> >> stated goals of increasing visibility and better keeping them up to
> >> date, and for all the reasons we just migrated the beam-site directory
> >> in. It seems the primary motivation is that it's difficult in Java to
> >> have a portion of the repo that depends on another as if it were
> >> "external" (i.e. the way others would use Beam) rather than being a
> >> sub-project of Beam. Is this not doable?
> >> On Wed, Aug 1, 2018 at 10:59 PM Charles Chen <ccy@xxxxxxxxxx> wrote:
> >> >
> >> > I would also prefer that examples be linked to releases so that we can
> >> build and test them during development; i.e. if your commit breaks
> >> wordcount, we want to know right away so we can revert.  Perhaps we can
> >> keep these in the repo but more clearly modularize the artifacts we release?
> >> >
> >> > For the Python SDK, if we separate this out in any way, there is the
> >> separate issue of dealing with namespace packages (which are kind of broken
> >> and poorly supported:
> >> https://github.com/pypa/python-packaging-user-guide/issues/265), if we
> >> want to keep the examples under the apache_beam.examples module path.  See
> >> also https://packaging.python.org/guides/packaging-namespace-packages/.
> >> >
> >> > On Wed, Aug 1, 2018 at 9:29 PM jb@xxxxxxxxxxxx <jb@xxxxxxxxxxxx> wrote:
> >> >>
> >> >> Hi,
> >> >>
> >> >> I don't have problem to move the examples in a dedicated repository.
> >> However, IMHO, we have to:
> >> >>
> >> >> 1. Keep a build of examples linked to latest core release/SNAPSHOT
> >> >> 2. Include the examples in the distribution (convenient for the users)
> >> >>
> >> >> On another topic, I think it would be better to avoid usage of Google
> >> Doc for such kind of discussion and directly share on the mailing list (at
> >> least a summary/light details).
> >> >>
> >> >> Regards
> >> >> JB
> >> >>
> >> >> On Thursday, August 02, 2018 00:12 CEST, David Cavazos <
> >> dcavazos@xxxxxxxxxx> wrote:
> >> >>
> >> >>
> >> >> Hi everyone!
> >> >>
> >> >> We wanted to migrate the examples from the core repository to a new
> >> Beam community examples repository. As the number of examples grow, it
> >> makes sense to modularize and decouple the core functionality from the
> >> examples.
> >> >>
> >> >> We will also create some guidelines with the best practices for new
> >> examples to be submitted.
> >> >>
> >> >> For more details, feel free to take a look and comment on the proposal.
> >> >>
> >> >> Cheers,
> >> >> David
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> >>
> >
> >
>