Re: SVN externals handling

Renaud Bruyeron wrote:
> I have an issue with a couple of projects that use svn:externals.
> The use-case looks like this:
> Project A
> - build.xml
> - src
> - project B (via external)
> -- build.xml
> -- src
> - project C (via external, requires B.jar)
> -- build.xml
> -- src
>
>
> If project B or project C are modified, project A should rebuild. 
> However the SCM trigger does not know anything about externals because 
> svn info does not recurse into externals.

I didn't know anything about "svn:externals", so I read the document. I 
can see why polling wouldn't pick that up.

> How do people handle this case (I saw at least one other email on this 
> list mention this problem) ?

One approach is to rely on push, not polling. See 
https://hudson.dev.java.net/build.html  While easy to set up, polling is 
not a wise approach anyway. It wastes network and server resource, and 
there tends to be a time lag between the change and the build. "Push" 
for svn works by setting up a post commit script to invoke the "schedule 
a build" URL.

Another approach is to have Hudson understands svn:externals. I don't 
know if there are any cheap way to find out where externals are, but 
maybe it's possible by looking at .svn files somewhere.

A variation of the above is to fix issue #49, which would most certainly 
allow Hudson to learn all svn:externals.


> I tried to check out several modules in the same job, but this does not 
> work in some cases because the urls of the various projects may look 
> like this: http://svn/projectA/branches/7.2.1, 
> http://svn/projectB/branches/7.2.1, http://svn/projectC/branches/7.2.1
> When trying to use all these as modules, hudson barfs because it tries 
> to check them out in the same folder 7.2.1...

This is an issue of its own (it happens when you have multiple 
"trunk"s.) This is issue #153. If you can ping crashedworld@xxxxxxxxxxxx 
to see where he is on this issue, that would be great.

> Another approach would be to have 3 projects in Hudson and cascade the 
> dependencies.
> However, project B or C cannot build on their own, therefore the 2 
> projects would only be simple checkouts and act as external triggers for 
> project C...
> Moreover, project B or C could be multi-gigabyte folders that are 
> already checked out by project A's external setup, leading to a lot of 
> wasted space.

Ah, this is interesting. I guess it works, except the waste you 
mentioned, and the branch issue.


> I looked into the subversion SCM provider, and it could support 
> externals natively by changing a few things (it looks like a bit of work 
> though):
> * after svn checkout or svn update, run svn status --xml to pick up the 
> locations of the externals

Does "svn status -xml" work recursively? Or will Hudson need to run this 
for each sub-directory?

> * if (externalSupportEnabled), run svn info --xml on the root AND on the 
> externals location
> * somehow merge change info from the different externals

Once we know where svn:externals are, I think the rest is probably not 
too hard. SubversionSCM class already supports multiple modules, so in a 
way this is just a slight variation to that.

> Another approach would be to change the SCM poller only, and be able to 
> specify additional svn urls to poll for changes. This change looks 
> simpler, however it is less transparent.

Yeah. I'd prefer to keep the configuration simpler, so I think the 
earlier approach you suggested sounds more promising.


--
Kohsuke Kawaguchi
Sun Microsystems                   kohsuke.kawaguchi@xxxxxxx

smime.p7s
Description: S/MIME Cryptographic Signature