Re: Housing longer-term Arrow development, design, and roadmap documents
I would prefer Confluence over GitHub pages because I would hope that one can integrate the ASF JIRA via widgets into the wiki pages. The vast amount of issues should all be categorizable into some topic. Once these are triaged, they should pop up in the respective wiki pages that could form a roadmap. That way, newcomers should get a better start to find the things to work on for a certain topic.
On Sun, Jun 24, 2018, at 7:02 PM, Antoine Pitrou wrote:
> Hi Wes,
> I wonder if GitHub wiki pages would be an easier-to-approach alternative?
> Le 24/06/2018 à 08:42, Wes McKinney a écrit :
> > hi folks,
> > Since the scope of Apache Arrow has grown significantly in the last
> > 2.5 years to encompass many programming languages and new areas of
> > functionality, I'd like to discuss how we could better accommodate
> > longer-term asynchronous discussions and stay organized about the
> > development roadmap.
> > At any given time, there could be 10 or more initiatives ongoing, and
> > the number of concurrent initiatives is likely to continue increasing
> > over time as the community grows larger. Just off the top of my head
> > here's some stuff that's ongoing / up in the air:
> > * Remaining columnar format design questions (interval types, unions, etc.)
> > * Arrow RPC client/server design (aka "Arrow Flight")
> > * Packaging / deployment / release management
> > * Rust language build out
> > * Go language build out
> > * Code generation / LLVM (Gandiva)
> > * ML/AI framework integration (e.g. with TensorFlow, PyTorch)
> > * Plasma roadmap
> > * Record data types (thread I just opened)
> > With ~500 open issues on JIRA, I have found that newcomers feel a bit
> > overwhelmed when they're trying to find a part of the project to get
> > involved with. Eventually one must sink one's teeth into the JIRA
> > backlog, but I think it would be helpful to have some centralized
> > project organization and roadmap documents to help navigate all of the
> > efforts going on in the project.
> > I don't think documents in the repository are a great solution for
> > this, as they don't facilitate discussions very easily --
> > documentation or Markdown documents (like the columnar format
> > specification) are good to write there when some decisions have been
> > made. Google Documents are great, but they are somewhat ephemeral.
> > I would suggest using the ASF's Confluence wiki for these purposes.
> > The Confluence UI is a bit clunky like other Atlassian products, but
> > the wiki-style model (central landing page + links to subprojects) and
> > collaboration features (comments and discussions on pages) would give
> > us what we need. I suspect that it integrates with JIRA also, which
> > would help with cross-references to particular concrete JIRA items
> > related to subprojects. Here's an example of a Confluence landing page
> > for another ASF project:
> > https://cwiki.apache.org/confluence/display/Impala
> > What do others think?
> > Thanks,
> > Wes