Re: against the components utopia

On Thu, Feb 09, 2006 at 01:05:44PM +0100, Lex Spoon wrote:
> Hey, Jamie, I will probably just have to say that I find the strong
> form of components utopia to be an unobtainable grail.  Arguing about
> it seems about as effective as arguing about mandantory static typing.

Certainly similar. Arguing that components are useless because
sometimes they break each other seems to me rather like arguing that
static type systems are useless because they can't prevent all runtime
errors. :-)

In both cases, I'd say that while they can't be relied on, they are
still very helpful as far as they go.

[ Snip long bit about components needing to interact, and needing a way
to override that interaction. ]

That method of overriding is exactly what I started by arguing that
sbaz doesn't have! It's what I'm trying to design for Kitten. I'm also
trying the minimise the need for it.

Suppose we have a universe of packages that all play nicely together.
Now we want introduce a problem package that has dependencies which
conflict with those of some other packages.

AIUI, in sbaz there are two options: modify the source of the package
to work in the universe, or create a new universe. The first is of
course the ideal solution, but it may not be feasible, either because
the source is not available, or because there is not development time
spare to do the conversion. The second option I consider unworkable:
what if I need one of the packages that can't fit into this new
universe?

The package system for Kitten should add some more options: if the
conflicting dependency is internal (i.e. other packages don't need to
talk to it), we can just isolate it in its own classloader. Problem
solved. If it's external, we'll have to do more work: we can create a
new component (probably corresponding to the GoFs Adaptor, Mediator,
or Facade patterns), behind which the problem package can be isolated.
That's an undesirable route, but it may be necessary in some
circumstances.

So, the key difference between Kitten and sbaz is that Kitten has
configurable levels of isolation, whereas AFAICT sbaz does not have
any.

> I don't think that source recompilation at install time changes any of
> this.  So, I don't expect that Gentoo has managed to avoid the above
> fundamental problems.  But this email is too long already, so I'll
> leave that for another post.

In Gentoo, compiling from source has several advantages:
- Conditional compilation based on detected dependencies, i.e.
  typically autoconf.
- Conditional patching by the Gentoo packager if required.
- Avoidance of ABI breakage.

For Scala, the (smaller) advantages would be:
- Correct mixin versions.
- Possibly improved robustness against changes to Scala.
- Possibility of conditional compilation (most likely of whole files,
  since we don't have a preprocessor).

I'm not really suggesting that sbaz should be source-based (though the
mixin problem could bite). My main point about Gentoo is that it
assumes by default that arbitrary versions of packages will just get
along (and takes a few steps to encourage that), and for the most part
they do. Debian (primarily due to C ABI issues) has to assume that
mismatched versions of software will not work together, and thus has
to define separate universes.

Put another way, I think sbaz can afford to act more like Gentoo than
Debian, because Java ABIs are more robust. But, it needs to deal
gracefully with conflicts when they do occur.

> Just by the way, you can perfectly well statically link in sbaz if you
> want.  In fact, the sbaz package itself does that.  In that case,
> though, you should probably not put the libraries in /lib.

I'm not sure what you mean by static linking. Java doesn't have such a
concept. Unless you explicitly use separate ClassLoaders (which I'm
arguing is necessary), everything goes into the same namespace,
regardless if where it's physically located. Unless you mean defining
the whole thing as a mixin?

One trick I understand some people use is to rename the packages of
internal dependencies, so if I use Log4J and want to avoid conflicts,
I can go through the Log4J sources and change all the package
declarations to e.g. com.sygneca.org.apache.log4j. That's pretty evil
though.

> One concrete example would be a 3-person team developing the software
> infrastructure for a small business.  They work with 10-ish packages
> that are specific to the business.  Scala bazaars supports that, just
> as Debian's APT does (though, I think, more clumsily).
> 
> Now, yes, those 10 packages do not do everything they need.  They
> would not include data structure libraries, GUI frameworks, CORBA
> integration, etc.  Thus, the 3-person team would work within the
> *union* of the 10-package universe plus a larger one that includes all
> the reusable stuff.  The rules for the 10-package universe could be
> essentially nil, because it's just 3 people who know each other well.

But in that case, how is the little universe a separate universe at
all? Isn't it just a private corner of the larger one? And if it needs
to avoid conflicts with the larger universe, isn't it going to have to
follow the same rules?

-- Jamie Webb