[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: The full list of proposals / prototype documents

Thank you for this link, Etienne. 
I agree that it doesn’t fit well for design documents page. So, I think it makes sense to add either on wiki or as a part of Nexmark documentation on web site: https://beam.apache.org/documentation/sdks/java/nexmark/.

On 11 Jul 2018, at 11:16, Etienne Chauchot <echauchot@xxxxxxxxxx> wrote:


It is the doc I wrote when I submitted Nexmark PR to ease the reading of the code.
It is not a design doc, I don't know if it belongs to the website page or to the wiki for beam devs.


Le mercredi 06 juin 2018 à 17:48 +0200, Alexey Romanenko a écrit :
FYI: Finally, it was merged and you can find this page here:

Thank you everybody who helped me to compile this list! 
I’ll try to do my best to update this with new coming docs. In the same time, please, feel free to add your new docs (or notify me if I missed this) once they are finished and ready to be published.


On 31 May 2018, at 18:52, Eugene Kirpichov <kirpichov@xxxxxxxxxx> wrote:

Thank you!

On Thu, May 31, 2018 at 8:30 AM Alexey Romanenko <aromanenko.dev@xxxxxxxxx> wrote:
Thank you everybody for provided links. I collected all of them (please, correct me if I missed something), categorized and created a dedicated page for Beam website.

Here is a PR for that (please, review):


On 30 May 2018, at 13:17, Łukasz Gajowy <lukasz.gajowy@xxxxxxxxx> wrote:

2018-05-29 22:42 GMT+02:00 Lukasz Cwik <lcwik@xxxxxxxxxx>:
Providing ownership to the PMC account allows others to take over ownership of the document once a contributor stops being active. This allows docs to be updated (even if just to point to a newer doc).

On Tue, May 29, 2018 at 1:20 PM Kenneth Knowles <klk@xxxxxxxxxx> wrote:
My position on ownership is design docs are really documents "of the moment" and authored by a particular individual or group. Experience shows that even if you try, keeping it fresh is not likely to happen. Anything that needs freshness (like end-user docs) should be in a different medium. I would just date the gdoc so readers know how to interpret it (the automated "last edit" date is not sufficient for understanding how stale something is). 

So it seems like it makes little difference if the project or PMC has ownership or even write access. Of course I have no objections if someone wants to transfer ownership, but is there a reason to encourage it?


On Tue, May 29, 2018 at 1:11 PM Lukasz Cwik <lcwik@xxxxxxxxxx> wrote:
I transferred ownership of the docs that I owned to the apachebeam@xxxxxxxxx PMC account and put the ones that I owned into the drive folder.

Would it be a good idea for others to follow suit?

On Tue, May 29, 2018 at 11:23 AM Lukasz Cwik <lcwik@xxxxxxxxxx> wrote:
I created a PR for the beam-site to link to the design docs and template from the contribution guide:

On Fri, May 25, 2018 at 10:23 AM Lukasz Cwik <lcwik@xxxxxxxxxx> wrote:

On Thu, May 24, 2018 at 2:11 PM Scott Wegner <swegner@xxxxxxxxxx> wrote:
Thanks for sharing these. I also put together a design doc template based on common styling / sections I saw in the docs listed above. Others are free to use it as they'd like.

On Thu, May 24, 2018 at 6:23 AM Kenneth Knowles <klk@xxxxxxxxxx> wrote:

On Thu, May 24, 2018 at 6:08 AM Etienne Chauchot <echauchot@xxxxxxxxxx> wrote:
Great that you take this action Alexey !
Here are the links I have, there is duplicates with the ones you already received and maybe old docs as well:


Le jeudi 24 mai 2018 à 14:53 +0200, Alexey Romanenko a écrit :
Thank you everybody for positive feedback and sending me links to design docs. I’m going to complete the full list and create PR for review.

Perhaps, it was a bit of misunderstanding here, let me explain what I assume under creating of such page. 

Actually, for now I’m going to collect all known design docs under one umbrella (the page on Beam web site) to make it easy to discover. For example, as a new contributor, I’d like to know more details what was initial design of new FileIO, SDF, Metrics, etc. 

I'm not going to copy the content of this docs to one page or even web site, let’s keep this as it is, no changes here for the moment. I think, moving to something else than Google docs is a tough question and requires another discussion. 

So, in this case, this task seems not so hard since we don’t add such docs too often - I'll just have to update this index page on web site. In addition, the authors will be always welcomed to update this page by themselves. In my turn, I’ll try to keep an eye on this to keep it synced. And of course, any help will be welcomed too =)


On 24 May 2018, at 00:01, Griselda Cuevas <gris@xxxxxxxxxx> wrote:

Hi Everyone, 

@Alexey, I think this is a great idea, I'd like to understand more of the motivation behind having all the designs doc under a single page. In my opinion it could become a challenge to maintain a page, so knowing what you want to accomplish could help us think of alternative solutions? 

On Wed, 23 May 2018 at 14:08, Daniel Oliveira <danoliveira@xxxxxxxxxx> wrote:
+1 to web site page (not Google Doc).

Definitely agree that a common entry point would be excellent. I don't like the idea of the Google Doc so much because it's not very good for having changes reviewed and keeping track of who added what, unlike Github. Adding an entry to the list in the website would require reviews and leave behind a commit history, which I think is important for an authoritative source like this.

PS: I also have a doc I proposed that I didn't see in the lists:

On Wed, May 23, 2018 at 12:52 PM Lukasz Cwik <lcwik@xxxxxxxxxx> wrote:
+1, Thanks for picking this up Alexey

On Wed, May 23, 2018 at 10:41 AM Huygaa Batsaikhan <batbat@xxxxxxxxxx> wrote:
+1. That is great, Alexey. Robin and I are working on documenting some missing pieces of Java SDK. We will let you know when we create polished documents.

On Wed, May 23, 2018 at 9:28 AM Ismaël Mejía <iemejia@xxxxxxxxx> wrote:
+1 and thanks for volunteering for this Alexey.
We really need to make this more accesible.
On Wed, May 23, 2018 at 6:00 PM Alexey Romanenko <aromanenko.dev@xxxxxxxxx>

> Joseph, Eugene - thank you very much for the links!

> All, regarding one common entry point for all design documents. Could we
just have a dedicated page on Beam web site with a list of links to every
proposed document? Every entry (optionally) might contain, in addition,
short abstract and list of author(s). In this case, it would be easily
searchable and available for those who are interested in this.

> In the same time, using a Google doc for writing/discussing the documents
seems more than reasonable since it’s quite native and easy to use. I only
propose to have a common entry point to fall of them.

> If this idea looks feasible, I’d propose myself to collect the links to
already created documents, create such page and update this list in the

> WBR,
> Alexey

> On 22 May 2018, at 21:34, Eugene Kirpichov <kirpichov@xxxxxxxxxx> wrote:

> Making it easier to manage indeed would be good. Could someone from PMC
please add the following documents of mine to it?

> SDF related documents:
> http://s.apache.org/splittable-do-fn
> http://s.apache.org/sdf-via-source
> http://s.apache.org/textio-sdf
> http://s.apache.org/beam-watch-transform
> http://s.apache.org/beam-breaking-fusion

> Non SDF related:
> http://s.apache.org/context-fn
> http://s.apache.org/fileio-write

> A suggestion: maybe we can establish a convention to send design document
proposals to dev+designs@xxxxxxxxxxxxxxx? Does the Apache mailing list
management software support this kind of stuff? Then they'd be quite easy
to find and filter.

> On Tue, May 22, 2018 at 10:57 AM Kenneth Knowles <klk@xxxxxxxxxx> wrote:

>> It is owned by the Beam PMC collectively. Any PMC member can add things
to it. Ideas for making it easy to manage are welcome.

>> Probably easier to have a markdown file somewhere with a list of docs so
we can issue and review PRs. Not sure the web site is the right place for
it - we have a history of porting docs to markdown but really that is high
overhead and users/community probably don't gain from it so much. Some have
suggested a wiki.

>> Kenn

>> On Tue, May 22, 2018 at 10:22 AM Scott Wegner <swegner@xxxxxxxxxx> wrote:

>>> Thanks for the links. Any details on that Google drive folder? Who
maintains it? Is it possible for any contributor to add their design doc?

>>> On Mon, May 21, 2018 at 8:15 AM Joseph PENG <josephtengpeng@xxxxxxxxx>

>>>> Alexey,

>>>> I do not know where you can find all design docs, but I know a blog
that has collected some of the major design docs. Hope it helps.

>>>> https://wtanaka.com/beam/design-doc

>>>> https://drive.google.com/drive/folders/0B-IhJZh9Ab52OFBVZHpsNjc4eXc

>>>> On Mon, May 21, 2018 at 9:28 AM Alexey Romanenko <
aromanenko.dev@xxxxxxxxx> wrote:

>>>>> Hi all,

>>>>> Is it possible to obtain somewhere a list of all proposals /
prototype documents that have been published as a technical / design
documents for new features? I have links to only some of them (found in
mail list discussions by chance) but I’m not aware of others.

>>>>> If yes, could someone share it or point me out where it is located in
case if I missed this?

>>>>> If not, don’t you think it would make sense to have such index of
these documents? I believe it can be useful for Beam contributors since
these proposals contain information which is absent or not so detailed on
Beam web site documentation.

>>>>> WBR,
>>>>> Alexey