Re: State of Development

On Oct 26, 2007, at 2:27 AM, David Schmitt wrote:
>
> That's good to hear!  My regular tests are already running again on
>
> http://puppettests.black.co.at/
>
> The testresults are put into a directory named after the time the  
> test ran.
> I run the following tests:
>
> * rake test, output in /rake_tests

You're definitely getting failures I'm not getting, at least on Debian.

I've only got one failure on Debian and none on OS X; I'm about to  
test my other OSes.

> * puppetmaster + puppetd on a empty site.pp. config, output and vardir
> in /minimal
>
> * puppetmaster + puppetd on my full configuration. config, output  
> and vardir
> in /trunk

Can we work at getting these tests, at least the minimal one, into  
the test framework?  Otherwise it's difficult for me to work at  
reproducing them.

> These tests are currently run 06:30 UTC+2, which should be around  
> the end of
> your workday? Please tell me if you'd like them run more often or at a
> different time.

The time is fine.  Now we just need a script that emails failures to  
puppet-dev (preferably only if the failures are different than the  
day before).  Care to take responsibility for that?

> I even whipped up a little munin plugin which tracks rake test  
> results.
> http://www.edv-bus.at/munin/black.co.at/git-test.black.co.at- 
> puppettests.html

That's pretty darn slick.

We really need to set up some kind of autobuild system, preferably  
backed by multiple OS instances.  Maybe something on EC2, with a  
community member taking responsible for each instance?  E.g., have a  
Solaris image maintained by someone familiar with it, same for  
Debian, Ubuntu, etc.

Anyone want to volunteer to head up this effort?  Anyone know much  
about autobuild systems, especially those that will generate nice  
graphs like this?

> As you said, the git repo currently is a bit off track. Also the  
> tests are run
> within a vserver container, so some tests (e.g. the mount stuff)  
> will always
> fail due to missing permissions.

We need to get the tests to the point where they always pass on all  
systems.  If you've got tests that never pass, then the tests need to  
be fixed so they pass or they need to be changed so they aren't run.

Are you in a position to take a crack at your consistently-failing  
tests?

  --
  Talent hits a target no one else can hit; Genius hits a target no one
  else can see.  -- Arthur Schopenhauer
  ---------------------------------------------------------------------
  Luke Kanies | http://reductivelabs.com | http://madstop.com