osdir.com

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Recruiting more maintainers for Apache Arrow


One of the things I’ve started doing in the Spark project is live code
reviews to encourage other folks to get involved in the review process and
help it seem more achievable (see
https://www.youtube.com/playlist?list=PLRLebp9QyZtYF46jlSnIu2x1NDBkKa2uw )
.

Another that I think has helped us is making it clear one of the steps to
becoming a committer (something often valued by corporate employers) is
being involved in the review process.

I don’t know how much this applies, but some of the committees have also
found our PR dashboard which gives a view of PRs that are ready to merge
and organized by area to be helpful (see
http://spark-prs.appspot.com ).

YMMV of course, but this is a problem with I spend a lot of time thinking
about (only sometimes with answers) so really interested to see where the
discussion goes.

I gave a somewhat related talk: (Dealing with Contributor Overload) at FOSS
backstage recently
https://youtu.be/XS8cTLAuHUw

I’m not really all that involved with the Arrow project but if folks would
be open to it I’d be happy to add it to my list of projects I do livestream
reviews with.

On Sat, Jun 30, 2018 at 7:58 AM Wes McKinney <wesmckinn@xxxxxxxxx> wrote:

> hi folks,
>
> Arrow has grown by leaps and bounds over the last 2.5 years. We are
> approaching our 2000th patch and on track to surpass 200 unique
> contributors by year end.
>
> All this contribution growth is great, but it has a hidden cost: the
> maintenance. The burden of maintaining the project: particularly
> reviewing and merging patches, has fallen on a very small number of
> people. From the commit logs, we can see how many patches each
> committer has merged:
>
> $ git shortlog -csn d5aa7c46692474376a3c31704cfc4783c86338f2..master
>   1289  Wes McKinney
>    268  Uwe L. Korn
>     74  Korn, Uwe
>     54  Antoine Pitrou
>     52  Julien Le Dem
>     39  Philipp Moritz
>     18  Kouhei Sutou
>     18  Steven Phillips
>     13  Bryan Cutler
>     11  Jacques Nadeau
>     10  Phillip Cloud
>      8  Brian Hulette
>      5  Robert Nishihara
>      5  adeneche
>      4  GitHub
>      3  Sidd
>      3  siddharth
>      1  AbdelHakim Deneche
>      1  Your Name Here
>
> So Uwe and I have merged ~84% of the patches in the project so far.
> This isn't a completely accurate reflection of the maintainer burden,
> since many others contribute to code reviews and other aspects of
> patch maintenance, and you have to be a committer to earn a place on
> this list.
>
> I'm not sure what's the best way to address this problem. The quality
> of our code review has declined at times as we struggle to keep up
> with the flow of patches -- I don't think this is good. Having the
> patch queue pile up isn't great either. Personally, I'm having a
> difficult time balancing project maintenance and patch authoring,
> particularly in the last 6 months.
>
> Unfortunately, many people believe that writing patches is the primary
> mode of contribution to an open source project. Apache projects
> explicitly state that non-patch contributions are valued in earning
> karma (committership and PMC membership). We're starting to have more
> corporate contributors come out of the woodwork, and while it's great
> for contributors to be paid to write patches for the project, they are
> rarely given the time and space to contribute meaningfully to
> maintenance.
>
> Any thoughts about how we can grow the maintainership? Somehow we need
> to reach ~5-6 core maintainers over the next year.
>
> Thanks,
> Wes
>
-- 
Twitter: https://twitter.com/holdenkarau