Where Squeak is Headed [was: Module discussion]

Folks -

Just for purposes of my own planning, I have been chatting with others at SqC 
recently about where we are and where we should be heading.  Historically I 
have written messages with this same subject header intended as sort of a 
declaration.  This time the intention is simply to participate in a discussion 
that has already begun (as in Andrew Greenberg's original message in the 
"Uncomfortable Question" thread, and Mike Rueger's message summarizing a 
discussion on modules and updates last night among those at OOPSLA).  In what 
follows, I believe I am speaking for most of SqC.

Whither SqC
As background to some of what follows, I thought it might be useful to talk 
about some of the changes at SqC.  The most significant change is the birth of 
Croquet, an event of which many of you are now aware.  In terms of 
personalities, Alan is promoting and critiquing this project, while the 
technical core consists of Andreas Raab, David Smith, and David Reed.  Kim and 
Pat continue to help Alan hold the fort at Viewpoints Research.

John Maloney, who had stayed on at Disney after the rest of us left, has 
finally left also and is now working with Mitch Resnick's group at MIT.  They 
are planning interesting projects using Squeak, and also watching the evolution 
of Croquet for opportunities to participate.  Hopefully some time in the near 
future John or someone else in that group will post a little overview of what's 
going on there and their plans for the future.

Scott, Ted, Michael and I are all doing various separate Squeak-related 
projects on our own.

Croquet
The whole Squeak Central team continues to stay in touch and we work together 
off and on as the occasion arises (like fixing bugs for Alan ;-).  We are all 
intrigued with Croquet, but at this formative stage it is clear that the core 
group there needs above all to keep its focus, independent of the rest of us 
and the Squeak community in general.  Apart from the fact that there isn't 
money for others, magical things are happening as they take the "end users 
first" philosophy into the world of collaborative 3-D.

That said, it is my hope that, by tracking the Croquet work as closely as 
possible, we may be able to co-evolve Squeak so that it will continue to be the 
environment of choice for future generations of Croquet.

Jitter
One of my goals over the next few months is to continue to encourage Ian to 
complete J5, the latest incarnation of his dynamic translation technology 
applied to Squeak.  On the last serious go-round, it helped a lot to have 
Marcus Denker tracking down bugs, beating compilers into line, etc, and Ian may 
well need this kind of help with J5.  I'm not the right person, but I would 
certainly lobby to hook people up at any point where Ian could use help.  I 
haven't seen anything about it, but I'm assuming Ian will be showing and 
talking about his work at OOPSLA.  Jitter is a prime example of why I hope we 
can keep the Croquet and Squeak kernels in sync:  I doubt that Ian wants to 
maintain more than one version of J5, and I know both Squeak in general and 
Croquet in particular, could always use an extra 2-5x in performance.

Looking at the next six months, I feel that Ian's design for J5 goes beyond 
Anthony's work on the BCVM and, if possible, I would like to see J5 completed 
and made part of the mainstream Squeak VM support.  At the same time, Anthony 
has done all the hard work of converting Squeak to use proper closures, and it 
would be ideal if that experience could be salvaged and recreated in the 
context of Squeak with J5.  It's conceivable that this could be done in a way 
that the resulting image could be run with the BCVM or J5 interchangeably -- 
only Anthony and Ian could say if this makes sense.

Kernel
There is a lot of separate interest in kernels which I believe can be 
convergent.  Andreas has a draft in his 400k SqueakScript kernel, I'm just now 
getting my 240k tiny.image out of mothballs, and various people are playing 
with modules and SqueakMap in the hope of decomposing Squeak into an image 
built up cleanly from a small kernel.  This has been on our agenda for several 
years, hopefully now is the time to make it really happen.

Modules
I have just read the summary of the SqueakBOF discussion sent out by Michael 
Rueger.  Interestingly, this reflects the sentiments of Squeak Central as well. 
 Moreover, as Michael mentioned, Scott Wallace, on whom the burden of the fork 
fell most heavily within SqC, has dealt quite carefully with most of the 
incompatibilities and, I believe, even has a working draft for a 3.4 image.

Something needs to be said at this juncture.  There are a number of reasons 
that the 3.3 run at modularization did not succeed.  It retained the name space 
experiment from my work on Environments, it forced some changes that could have 
remained backward compatible, it was invasive against Les Tyrell's exhortation, 
it was not embraced by the rest of the community in spite of a courageous and 
intense push by Henrik Gedenryd.  If any fault is to be ascribed for this 
particular false start, it should rest with me and not with Henrik.  I have 
made mistakes before and I intend to go on making them, hopefully not all the 
time.  Henrik put a huge amount of work into the project, for which he can feel 
proud, and I don't think anyone would fault the job he did within the context 
of the project.  There is much to be salvaged both from the code and from the 
early experience of trying to modularize the Squeak image.

Having learned a lesson, we have the opportunity to again choose a path to a 
more modular world.  Examples exist in nearly every other mature Smalltalk, 
both of simple modules and more comprehensive packages.  I would like to see a 
module system that was completely independent of the rest of Squeak, along the 
lines that Les Tyrell suggests.  A move to life with SqueakMap should greatly 
encourage this property.

Many people in our community have had experience with modules, even the chance 
to compare life with several different systems.  Now would be a good time to 
recommend (possibly for the second time) any approaches known to be simple and 
effective.

"Under new management"
Michael shared with me one other topic raised at the OOPSLA BOF, but not 
included in the public report.  Here's the wording I saw:

        "The suggestion is to hand management of the update stream over to a 
group
        of experienced Squeakers. This group will manage the review and 
publishing
        process and have SqC as advisors in the background.
        Candidates right now are Göran, Doug, Ned and Tim (you did volunteer,
        didn't you? ;-) ). Volunteers, comments, vetos are welcome."

Believe it or not, this, too, agrees with current SqC thinking.  I think 
nothing could be more invigorating to our process going forward.  Presumably 
the migration of essentially everything but the kernel (and potentially even 
that) into SqueakMap will lead to territories responsibly managed by those who 
know the most about them.  Beyond the update process, some attention needs to 
be paid to the identification of stable releases.  It is my experience that 
there are "propitious" times for stable releases (generally on the eve of 
significant changes), and I think it will behoove us to evolve an informal 
mechanism for picking these times and a formal process for checking that 
SqueakMap packages sync'ed to a stable release get some decent QA.

Looking forward to more reports from OOPSLA, to further comments on this 
thread, and to a winter of exciting new progress.

        - Dan