[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

A renewed plea for help [was Re: Recruiting more maintainers for Apache Arrow]

hi folks,

It's been a few months, but as Apache Arrow is rapidly becoming a
critical dependency of next-generation data applications (see, for
example, RAPIDS just launched by NVIDIA http://rapids.ai/), we are
quite seriously in need of more project maintainers, or in lieu of new
individual contributors, additional direct funding. We are especially
in need of corporations dependent on this software to help carry the
load of JIRA gardening, code review, build and CI tooling, packaging
automation, developer workflow tools, and so on.

One of the casualties of the growing maintenance burden of this
project is that it's increasingly difficult for people like me who
know the project internals very well to allocate time to working on
new functionality. When I talk to people about the project they often
ask me things like "When will X, Y, or Z functionality be ready?" and
my answer is often "I don't know, it depends on whether more people
show up to help with the maintenance workload so people can spend more
time building new things". This is coupled with the frustration that
newcomers can experience where the learning curve is very steep to be
able to contribute significantly to new functionality. The only way
out is to recruit more people to help keep things orderly, take out
the proverbial garbage, and keep the project healthy.

If anyone reading has the bandwidth to help with maintaining the
project, or to contribute funds to support maintenance, please let us

Special thanks to Antoine, Kou, Kristztian, Phillip, and Uwe for their
work on tooling, packaging, and other development processes for the
0.10 and 0.11 releases.


On Mon, Jul 2, 2018 at 11:40 AM Antoine Pitrou <antoine@xxxxxxxxxx> wrote:
> Hi,
> Le 02/07/2018 à 15:58, Wes McKinney a écrit :
> > * http://ivory.idyll.org/blog/2018-how-open-is-too-open.html
> > * http://ivory.idyll.org/blog/2018-oss-framework-cpr.html
> Very good articles, but I would stress that some of the mechanisms
> proposed lack metrics in their favour.  Two particular examples that I
> know about:
> 1)
> """ I seem to recall Martin van Loewis offering to review one externally
> contributed patch for every ten other patches reviewed by the submitter.
> (I can’t find the link, sorry!) This imposes work requirements on
> would-be contributors that obligate them to contribute substantively to
> the project maintenance, before their pet feature gets implemented. """
> Martin's offer was almost never taken up, although he expressed it many
> times during many years.  I think there are two factors to it:
> a) Cost.  As an occasional contributor, I could understand having to do
> a review before contributing a patch of mine, but not having to do 5 or
> more reviews for each patch I contribute.  The effort asked is much too
> high, and you're probably discouraging people who are discovering the
> project, even before they could get hooked on it.
> b) Difficult.  It's much more difficult and intimidating to review
> someone else's PR, than to propose your own changes knowing that it will
> be reviewed by (you are assuming) competent people.  So this mechanism
> is excluding first-time contributors, which is probably *not* what you want.
> 2)
> """ Some projects have excellent incubators, like the Python Core
> Mentorship Program, where people who are interested in applying their
> effort to recruiting new contributors can do so. """
> Actually, it doesn't seem to me that a significant proportion of
> frequent Python contributors have gone through the core mentorship
> process.  It probably got us a handful of one-time contributions.
> Pointing to the Python core mentorship program as an "excellent
> incubator" sounds rather far-fetched to me.
> Generally speaking, there's a limit to the usefulness of hand-holding
> contributors, especially if your project is rather complex (as Python
> is), because the blocking point for contributors is *not* that the
> development mailing-list is a bit intimidating (as was claimed by the
> people who founded the Python core mentorship program).
> PS : as a matter of fact, the general rate of contributions to Python
> has been *decreasing* for years.
> Regards
> Antoine.