[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Recruiting more maintainers for Apache Arrow

Hi folks,

I would like to highlight that the challenges we are having are
endemic to many parts of the open source world right now. A colleague
of mine in the Python world wrote some pieces about this recently:

* http://ivory.idyll.org/blog/2018-how-open-is-too-open.html
* http://ivory.idyll.org/blog/2018-oss-framework-cpr.html

Here are some quotes from those pieces:

"This need for constant attention to projects, the sprawling ecosystem
of amazing scientific software packages, and the relatively small
community of actual maintainers, when combined, lead to the open
source sustainability problem in science: we do not have the person
power to keep it all running without heroic efforts. And when you
couple this with the lack of clear career paths for software
maintenance in science, it is clear that we cannot ethically and
sustainably recruit more people into open source maintainership."

I would say that "heroics" does describe some of the occasional
behavior of Arrow maintainers. The trouble with "heroics" (which
translates practically speaking to "overwork") is that if sustained
for a long period of time, it surely leads to burnout and depression.
I can speak from personal experience.

On a later point in this quote about "lack of clear career paths for
software maintenance", rather than griping about the problem, I
decided to do something about it. I have recently created a new
organization so that I can

a) enable organizations to directly fund Arrow maintenance and
b) provide secure full-time employment to Arrow maintainers

"Second, the cost of the constant maintenance needs (code,
documentation, installation, etc.) on the pool of available effort
needs to be taken into account. Contributions of new features that do
not come with effort applied to maintenance should be carefully
considered - is this new contributor likely to stick around? Can they
and will they devote some effort to maintenance? If not, maybe those
contributions should be deferred in favor of contributions that add
maintenance effort to the project, e.g. via partnerships."

I see both sides of this argument. I think we need to be more
proactive about requesting maintenance help from "extractive"
contributors who are mostly "taking" from the project and giving
relatively little to support the overall health of the project.

"Fourth, there are some interesting governance implications around
allowing all or most of the resource appropriators to participate in
decision making. I need to dig more into this, but, briefly, I think
projects should formally lay out what level of investment and
contribution is rewarded with what kind of operational, policy making,
and constitutional decision making authority."

Apache governance already provides a framework for obtaining decision
making authority in a project. Suffice to say, I would be hesistant to
support a new PMC member who has not engaged on project maintenance.

- Wes

On Mon, Jul 2, 2018 at 7:03 AM, Antoine Pitrou <antoine@xxxxxxxxxx> wrote:
> Hi Dimitri,
> Le 02/07/2018 à 12:46, Dimitri Vorona a écrit :
>> Hi Wes,
>> to contribute an outsiders POW: while it is clear, what's expected if you'd
>> like to make a PR, it's not at all clear to me, where would I start if I
>> wanted to help with PR reviews without being heavily involved with the
>> community/being a full maintainer. Should I just grab a PR, test it,
>> comment on changes? I wouldn't be sure if I were stepping on someone's
>> feet, tbh.
> You don't have to manually test a PR, unless you want to be sure about
> semantics that are not part of the tests added in the PR (but then it
> would be a good idea to mention that the tests don't exercise the
> semantics enough :-)).
> From my point of view (generally as an open source developer and
> maintainer, this isn't specific to Arrow), reviewing is:
> * checking for soundness of concepts (if the PR adds any of them)
> * checking for maintainability and readability of code
> * checking for smelly coding patterns, possible sources of bugs etc.
> * depending on the context, checking for possible performance issues
> * any potential problem that your personal expertise may help you detect
> If you're not sure about a comment and hesitate posting it, a good
> solution is to phrase it as a question.
> Regards
> Antoine.