OSDir


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Community Examples Repository


I might miss it: are examples to be moved including those which are not under example/? For example there are some BeamSQL examples in org/apache/beam/sdk/extensions/sql/example


It's better to keep BeamSQL examples in where it is because related API might still change.

-Rui

On Thu, Aug 2, 2018 at 8:58 AM Ahmet Altay <altay@xxxxxxxxxx> wrote:
Robert, I agree with you in general. However there is also a second motivation. There is an increase in new PRs that are coming to add new examples. This is great however the core code (including distributions) is not a great place to host such examples. An examples repo would help in this case. It could also serve as an entry point for new contributors.



On Thu, Aug 2, 2018 at 12:40 AM, Robert Bradshaw <robertwb@xxxxxxxxxx> wrote:
I have to admit I'm generally -1 on moving examples to a separate
repository. In particular, I think it would actually inhibit the
stated goals of increasing visibility and better keeping them up to
date, and for all the reasons we just migrated the beam-site directory
in. It seems the primary motivation is that it's difficult in Java to
have a portion of the repo that depends on another as if it were
"external" (i.e. the way others would use Beam) rather than being a
sub-project of Beam. Is this not doable?
On Wed, Aug 1, 2018 at 10:59 PM Charles Chen <ccy@xxxxxxxxxx> wrote:
>
> I would also prefer that examples be linked to releases so that we can build and test them during development; i.e. if your commit breaks wordcount, we want to know right away so we can revert.  Perhaps we can keep these in the repo but more clearly modularize the artifacts we release?
>
> For the Python SDK, if we separate this out in any way, there is the separate issue of dealing with namespace packages (which are kind of broken and poorly supported: https://github.com/pypa/python-packaging-user-guide/issues/265), if we want to keep the examples under the apache_beam.examples module path.  See also https://packaging.python.org/guides/packaging-namespace-packages/.
>
> On Wed, Aug 1, 2018 at 9:29 PM jb@xxxxxxxxxxxx <jb@xxxxxxxxxxxx> wrote:
>>
>> Hi,
>>
>> I don't have problem to move the examples in a dedicated repository. However, IMHO, we have to:
>>
>> 1. Keep a build of examples linked to latest core release/SNAPSHOT
>> 2. Include the examples in the distribution (convenient for the users)
>>
>> On another topic, I think it would be better to avoid usage of Google Doc for such kind of discussion and directly share on the mailing list (at least a summary/light details).
>>
>> Regards
>> JB
>>
>> On Thursday, August 02, 2018 00:12 CEST, David Cavazos <dcavazos@xxxxxxxxxx> wrote:
>>
>>
>> Hi everyone!
>>
>> We wanted to migrate the examples from the core repository to a new Beam community examples repository. As the number of examples grow, it makes sense to modularize and decouple the core functionality from the examples.
>>
>> We will also create some guidelines with the best practices for new examples to be submitted.
>>
>> For more details, feel free to take a look and comment on the proposal.
>>
>> Cheers,
>> David
>>
>>
>>
>>
>>