Re: against the components utopia

Hey Jamie, 

I see no conflict between sbaz and Kitten.  Sbaz itself is agnostic
about what kind of components are shipped around and the mechanism by
which they are linked into a running program.  So, by all means feel
free to post Kitten tools and Kitten-packaged components to the
scala-dev bazaar!

It is true that, in the lack of any fancy linker, scala-dev has a
*culture* of accumulating jars into /lib that can all be
simultaneously loaded.  That strikes me as a good thing to push even
if there is a linker available that does not need it.  For example,
there may as well be a standard log4j in scala-dev's /lib, and
programs that don't care should use that version when possible, and
the linker can then make that case less hassle than the case where you
use a custom version...

But there is no reason that a better component system could not be
posted to scala-dev as well....  The only real "problem" would be if
we get so overloaded with different component systems that no ones
components can be used with anyone elses.  That's a bit of a
theoretical problem at the moment!


> Suppose we have a universe of packages that all play nicely together.
> Now we want introduce a problem package that has dependencies which
> conflict with those of some other packages.
> 
> AIUI, in sbaz there are two options: modify the source of the package
> to work in the universe, or create a new universe. The first is of
> course the ideal solution, but it may not be feasible, either because
> the source is not available, or because there is not development time
> spare to do the conversion. The second option I consider unworkable:
> what if I need one of the packages that can't fit into this new
> universe?


With sbaz, the straightforward approach would be to use a custom
CLASSPATH.  For politeness, you should not, in that case, put the
conflicting jar's into /lib.


Notice that it's hard for a completely new package to conflict with
other stuff except with filenames.  For a conflict to occur, there
must be two packages that try to work but fail.  Thus, we should
probably talk about incompatibility unless its about filenames.



> The package system for Kitten should add some more options: if the
> conflicting dependency is internal (i.e. other packages don't need to
> talk to it), we can just isolate it in its own classloader.

Cool!

Have you looked at "Jiazzi" at all?





> > I don't think that source recompilation at install time changes any of
> > this.  So, I don't expect that Gentoo has managed to avoid the above
> > fundamental problems.  But this email is too long already, so I'll
> > leave that for another post.
> 
> In Gentoo, compiling from source has several advantages:
[...]

Yes, it has advantages.  It does, however, not help you with the
classic problems from RPM hell:

 - API's change between versions

 - Packages are split, merged, and renamed, thus invalidating
   declared dependencies (I depend-on "apache", but now the
   package is called "apache2")

 - Packages put files into different places in the filesystem
   (I install into /home/apache/modules, but the new version
   wants me to put them in /www/modules)

 - Packages push the limits of API's, e.g. using NULL instead
   of the empty string



This said, it may well be that Scala Bazaars should rely on source
distribution instead of distribution of jar's.  That, however, is
independent of the basic mechanism for dealing with conflicts like the
above.



> One trick I understand some people use is to rename the packages of
> internal dependencies, so if I use Log4J and want to avoid conflicts,
> I can go through the Log4J sources and change all the package
> declarations to e.g. com.sygneca.org.apache.log4j. That's pretty evil
> though.

Yes.  This can also be done with binary-rewrites, which I am under the
impression that Jiazzi would sometimes use.



> > One concrete example would be a 3-person team developing the software
> > infrastructure for a small business.  They work with 10-ish packages
> > that are specific to the business.  Scala bazaars supports that, just
> > as Debian's APT does (though, I think, more clumsily).
> > 
> > Now, yes, those 10 packages do not do everything they need.  They
> > would not include data structure libraries, GUI frameworks, CORBA
> > integration, etc.  Thus, the 3-person team would work within the
> > *union* of the 10-package universe plus a larger one that includes all
> > the reusable stuff.  The rules for the 10-package universe could be
> > essentially nil, because it's just 3 people who know each other well.
> 
> But in that case, how is the little universe a separate universe at
> all? Isn't it just a private corner of the larger one? And if it needs
> to avoid conflicts with the larger universe, isn't it going to have to
> follow the same rules?

On the first question: not necessarily.  While for many people it is
possible to work in a corner of an existing universe, for others
obviously it is not.  Google is not going to post their code to a
public Gentoo server even if they use the Gentoo tools.

On the second question, they do indeed have to stay consistent with
the bigger universe.  However, they do not have to follow all of the
bureaucracy and rules of the bigger universe in their own.  They do
not, for example, have to name their packages with DNS names.  They do
not have to follow all the details of the filesystem policy.  They do
not have to subject each other to whatever new-committer process is
used by PUB (probably not *everyone* can post without being screened).

That these questions are hard to answer is precisely the reason I
don't want the system to impose any particular answer.  I'd like to
allow people to organize themselves in a wide variety of ways.  Sbaz
commits to the package universes approach.  That's a large design
space, but it already kicks out some possibilities.  For example,
Eclipse's plugin-updating system is not a package-universes solution.



-Lex