Re: Summary of Paul's post with some comments

Walter Chang wrote:
> I didn't want to butcher your wonderful post so I figured I'd
> "summarize" it in my reply :)
> Please let me know if I missed anything !

Thanks.

Here are a few elaborations or clarifications (not really anything you 
missed, just more thoughts).

I should have said at the start, as others said on the Jython lists about 
a year ago, that one of the reasons Jython hasn't progressed much from 2.1 
is that it hasn't had to. In other words, since Java is so backwardly 
compatible, Jython 2.1 is so good for so many uses that while a later 
Jython version tracking CPython is a nice to have thing, lack of it often 
isn't any sort of show stopper. Now, this lack of upgrades may make people 
think Jython is not used or is no longer useful, but another way to look 
at it is how great Jython already is and continues to be. Still, from a 
marketing standpoint, not bumping up version numbers obviously is a 
problem, and CPython itself had it when people years ago were thinking a 
Python 1.3 must naturally be less capable than, say, Perl 4.x, whereas the 
numbers didn't really mean anything you could easily compare.

Another issue is my own motivation to use Jython; I should have added that 
a major reason for me to have used Jython over CPython in the first place 
recently is commercial, as in a client wanted a solution using "Java" 
(although we would have rather used CPython, including from reusing 
existing wxPython code). So, from an open source developer standpoint, 
when you may still have one foot each stuck in both free and commercial 
worlds you may look for technologies that are useful in both. And Jython 
leverages on CPython experience, plus Java experience, so it makes sense 
in that context for many people. Anyway, that becomes another factor in 
evaluating what technology to develop or deploy free stuff on -- does the 
platform also have commercial usefulness and so skills honed on it have 
some potential financial payoff? And Jython clearly is valuable on that 
basis since Java is popular in the commercial realm (although so is 
CPython). So for purely free projects, one may still end up weighing "does 
this technology add to a commercial skill set" versus "is this the best 
free technology"?

I didn't spell out all of the negatives of injecting money into a free 
software project. People who might contribute can get put off for various 
reasons when money is on the table, either not liking a commercial 
atmosphere, getting a sense that someone else is already on top of the 
proposed changes, or developing a fear that small changes may not fit into 
some larger plan and will get lost or ignored. But the biggest may be 
simply having a mixed set of feelings about seeing others paid to do what 
you might do for free (as a demotivator). Now obviously, as it suggested 
by the following link's title: "Studies Find Reward Often No Motivator: 
Creativity and intrinsic interest diminish if task is done for gain"
  http://www.gnu.org/philosophy/motivation.html
money often hurts intrinsic motivation to do a creative task, but be that 
as it may, we are still humans living in a certain culture which requires 
money to survive. But perceptions of fairness and so on also have an 
impact on motivation (e.g. if you have two kids, try giving one of them 
half a glass of orange juice and one a full glass of orange juice without 
comment and see what happens, even if they both don't like orange juice. 
:-). This issue of perceived fairness is true even if ultimately almost 
all creative work is in a sense voluntary (since most bureaucracies doling 
out rewards can't tell the difference between masterpieces and junk in a 
time frame to issue rewards that have any "operant conditioning" effect). 
Still, one can't discount the need for some money in our society, since 
otherwise people end up doing other things with most of their waking 
hours; the issue is more if this is a reward or is just what it takes to 
have an otherwise average life in our culture. Anyway, when you mix money 
into free software, all sorts of strange things can happen, and one would 
hope to do it in in such a way that most of them are strangely good. :-) 
One approach if the money is limited is outlined below -- focus on 
supporting tasks other than coding.

Elaborating on the previous issue, here are some musings on how the notion 
of commercial-type "project management" fits in with open source / free 
software. Please forgive me if this is stuff you know already. I won't say 
project management can't mesh with "open source", because many project 
management concerns remain important, like prioritizing or triaging bug 
reports, modularizing tasks, tracking defects, collecting user feedback, 
reviewing quality, examining regression test results, and so forth (many 
of which are supported by aspects of a system like Sourceforge, but still 
take quite a bit of work to make happen). But here is a big difference 
between conventional project management and open source projects: without 
the power of command a typical project manager has (from wielding the 
carrot of monetary rewards and the stick of firing), it is just not 
possible to get developers attention to *your* priorities in the same way 
in a free project as it is in a commercial one. So, the more successful 
project managers of open source may end up looking more like the 
successful higher level *consultants* in a commercial enterprise, as 
opposed to the commercial *project managers*. To elaborate, beyond raw 
technical ability, a consultant succeeds in an enterprise typically by the 
power of persuasion coupled with a host of other socially oriented skills 
(not to get to far into the dark side of sausage making here, but these 
include at the very least sometimes getting other people to take ownership 
and get the credit of good ideas as their own. :-) As soon as a consultant 
starts trying to *command* results, then they don't usually last very long 
(unless of course they were actually hired as a project manager in the 
first place, which is rare.) Of course, really good commercial project 
managers may also do some of these consulting things when working with 
creative types, so the line blurs, but I think the basic distinction of 
command versus persuasion still holds.

So, the dynamics of a free or open source software project like Jython are 
just going to be different from a commercial project by default, and 
commercial project management approachs are used only at some risk. One 
aspect of that difference is also that doing things in a way that might 
seem terribly wasteful by any sort of sensible commercial development 
strategy (as if that really happens anyway in large companies :-) may in 
the free software world make a strange sort of sense. Specifically, free 
software projects often pursue many redundant approaches at once, but 
perhaps only one out of a thousand starts might go somewhere. For a 
commercial IT department, a 0.1% success rate for projects might seem like 
grounds for dissolving the whole department, but in an open source arena, 
with perhaps millions of developers overall doing stuff, that might 
translate into a lot of continual progress for the overall community 
(especially as the false starts or abandoned forks are soon forgotten or 
just not even seen). In fact, it maybe the only way to explore a large 
space of possibilities. That's one reason why it is hard to schedule 
results on an open source project (although even for many paid commercial 
projects often the less of a schedule the faster the results. :-) From:
  http://squizlog.keithpitty.org/archives/000046.html
  http://www.amazon.com/gp/product/0932633439/102-7822609-1235344?n=283155
"It is interesting to note that a University of NSW study, quoted in 
Peopleware: Productive Projects and Teams, concluded that "projects on 
which the boss applied no schedule pressure whatsoever ("Just wake me up 
when you're done.") had the highest productivity of all.""

So for anyone with commercial project management experience approaching an 
open source project, it is important to think about what parts of that 
skill apply in the non-commercial realm, and what parts may need to be 
changed to be more effective when "herding feral free-range cats".
  http://www.orient-lodge.com/index.php?q=node/view/68
From that link: "I’ve managed programming efforts before, and it has 
often been said that managing programmers is like herding cats. Open 
source programmers are the feral free-range cats. Herding them is even 
more of a challenge. Open source programmers work on projects because they 
are cool, fun, and achieve some sort of goal that the open source 
programmer likes. If the open source programmer gets bored or loses 
interest he just stops programming, and gaps can be left in the 
functionality. This was a problem in the early days of DeanSpace, but now, 
with funding, CivicSpace can hire people to work on the most crucial 
elements. During my years managing technology projects, I’ve gotten very 
interested in organizational dynamics. One of the first things I’ve always 
tried to focus on with any group I’ve been part of is defining the primary 
task. I was somewhat frustrated by a lack of clarity in terms of a primary 
task or specific sets of apparent goals for the summit. There wasn’t a 
clear definition of what CivicSpace is. There wasn’t a clear understanding 
of how all the different people involved in CivicSpace interact, or how we 
would define success. Yet in many ways, this was highly appropriate. 
CivicSpace seems committed to a bottom-up, emergent approach to activism. 
People work on what is important to them, and from that a true vision 
emerges. In many ways, the summit exemplified this approach. Tying it to 
my organizational dynamics perspective, it reminded me of the article, 
"Our Best Work Happens When We Don't Know What We're Doing".
  http://www.ispso.org/Symposia/Toronto/1999french-simpson.htm
"

The concept I like to think of for organizing a open source or free 
software project is "stigmergy".
  http://en.wikipedia.org/wiki/Stigmergy
"Stigmergy is a method of communication in emergent systems in which the 
individual parts of the system communicate with one another by modifying 
their local environment. Stigmergy was first observed in nature - ants 
communicate to one another by laying down pheromones along their trails, 
so where ants go within and around their ant colony is a stigmergic 
system. Similar phenomena are easily seen in many (all?) eusocial 
creatures, such as termites, who use pheromones to build their very 
complex nests by following a simple decentralized rule set. Each insect 
scoops up a 'mudball' or similar material from its environment, invests 
the ball with pheromones, and deposits it on the ground. Termites are 
attracted to their nestmates' pheromones and are therefore more likely to 
drop their own mudballs near their neighbors'. Over time this leads to the 
construction of pillars, arches, tunnels and chambers. The term was 
introduced by French biologist Pierre-Paul Grassé in 1959 to refer to 
termite behavior. He defined it as: "Stimulation of workers by the 
performance they have achieved." Stigmergy is not restricted to eusocial 
creatures, or even to physical systems. On the internet there are many 
emergent phenomena that arise from users interacting only by modifying 
local parts of their shared virtual environment. Wikipedia is a perfect 
example of this. The massive structure of information available in a wiki 
could be compared to a termite nest; one initial user leaves a seed of an 
idea (a mudball) which attracts other users who then build upon and modify 
this initial concept eventually constructing an elaborate structure of 
connected thoughts."

So, from a "project management" perspective on free and open source 
software, a good thing to think about may be, how can we set things up to 
make this stigmergic process happen? Licenses play a role (thankfully the 
CPython and Jython ones are good in that regard). But other factors play a 
role as well, including coordination with other projects, a level of fun 
or coolness, and some level of modularity or other approach to make 
complexity manageable. If there is one thing I think Jython might suffer 
from right now it is that the Jython codebase is fairly difficult to 
approach, making it harder for worker termites to get in there and make 
small improvements, especially because of the possibility of breaking 
something (including agreement with CPython). It's in Java after all, and 
there are good reasons why Java should be avoided (including so much 
boilerplate the essential functionality is obscured). Anyway, that is one 
thing that motivates some of my interest in a PyPy or Grammar driven 
approach that could be shared to some extent with CPython. But, it could 
also be not shared, that is, Jython could be written in Jython, or some of 
Jython could be generated from a grammar using ANTLR even if CPython is 
not -- all efforts to minimize the amount of Java code that needs to be 
maintained. (Though whether they would pan out in practice is a different 
issue).

Anyway, when looking at Jython as a stigmergic process, if someone has 
dollars to invest in it, then possibly the best way to invest those 
dollars isn't even in core development -- but more in areas of 
communications, coordination, maintaining and running test suites 
(especially relative to CPython), integrating in patches, producing up to 
date user and developer documentation, or other "boring stuff" for most 
developers (but perhaps quite interesting to some people who like that 
sort of stuff). Likewise, dollars could be spent on getting developers 
together physically at conferences and so on if that made any sense. I 
think what just was discussed here about the web site was an example -- 
why put limited skilled Jython internals developer time into web site 
design when other people could handle that? I'm not saying as things stand 
now getting the web site updated wasn't the best use of a developer's time 
(especially as an investment to get more help down the road), just that if 
one is looking at Jython as a process down the road and what parts could 
be offloaded to differently skilled people while avoiding some of the 
conflicts from paying for only some coding when the budget is limited, it 
would seem that documentation is one of the offloadable parts.

As far as "offshoring", for one example, this group
    "Minciu Sodas Laboratory"
    http://www.ms.lt/
based loosely in Lithuania lead by Andrius Kulikauskas, who seems sincere 
and is someone I might consider if I was hiring people to create an event 
or support structure around a project (not necessarily to do any coding, 
though they seem to do it too). From their web page: "Minciu Sodas helps 
your enterprise work openly to integrate constructive people around your 
purposes." [Disclaimer: I have no connection with these guys, but they did 
offer me a free membership years ago which I declined; I don't know that 
much about them, but from what I read they seem pretty cool.]
I'm sure there are lots of other possibilities too. And certainly the big 
names mentioned on this list have their own offshore resources and sites too.

But obviously it would seem the best place to start is with the people 
already sweating it out to make Jython work, and find out what they want 
to do and what they don't want to do, and what they think they think the 
bottlenecks are.

So to summarize, from a stigmergic point of view for investment strategy 
by an open source project manager, you may want to target investments to 
supporting a stigmergic coding process, and that doesn't necessarily mean 
paying for coding but rather removing roadblocks to letting the process 
happen. Or to quote AA Milne through Winnie the Pooh: "Because Poetry  and 
 Hums aren't things which you get, they're things which get you. And all 
you can do is to go where they can find you."

Obviously though, if your only goal for Jython was to track CPython 
(including, say, getting twisted to work out of the box), then one can 
just use a generic waterfall development model and some hired hands, and 
that may be all your group may think it wants. :-) And I can't say that 
would not just by itself be a very nice thing.

Anyway, just some more thoughts. :-)

--Paul Fernhout



-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid0944&bid$1720&dat1642