Recent experiences with Bamboo, which is for the most part an excellent tool, have inspired me to play around with various CI systems. I tried out Continuum, which I found to be both too clever and too stupid about Maven builds (which we use), and the old standard, CruiseControl, which I found frankly annoying to configure and manage. Annoying tools get ignored, not used to their fullest. Someone pointed me at Hudson, which is even newer than Bamboo, so I tried it out.

I think I’m in love.

The Hudson guys have taken a lot of hints from other CI tools. The builder configuration is Maven-aware, but not clever about it, which makes it very comfortable to use for Maven projects. The configuration is very straightforward, well documented, and handled entirely inside the webapp, which is in turn capable of self-hosting as a standalone program if you don’t have or don’t want to run a java web container like Tomcat or JBoss. The Hudson site does something exceptionally smart with this: they package the self-hosting version as a Java Web Start application, which means you can click a button in your browser and have a working build server in a matter of seconds.

However, a shiny demo system and simple configuration are not the only key features. I’m more interested in how Hudson behaves under a variety of “evil” build situations: does it fail sanely if the disk is full, and can it recover from non-responsive builds? In order to do this I put together a set of Maven build plugins that do the following things:

• One writes an infinite number of zeros to a file, filling the disk.
• Another spawns an unlimited number of threads, all of which live forever and none of which ever sleep.
• A third consumes the entire Java heap by creating and retaining objects in a loop.

Each of these “problem” builds can be run independantly of any others. I’ve intentionally skipped over idcmp’s fourth case: none of these explicitly ignores signals, since that’s hard to do from Java and unlikely. However, the thread-spawning build tends to ignore signals as a side-effect of swamping the CPU with context switches.

I set up the latest Hudson stable build (at the time of this writing, 1.156) as a self-hosting application in an Ubuntu 7.04 VMWare instance along with Maven 2.0.7 and ran each build individually as a Maven build.

Heaphell

The Heaphell build ran mvn hellbuild:heaphell, which causes Maven to exhaust its stack. If Maven runs in its own process, this is fine: Maven fails the build with an OutOfMemoryError and the build server itself is unaffected. On the other hand, if the build server is “clever” and tries to run Maven directly, the OutOfMemoryError may take out part of the build server as well.

Hudson handled this well. The build failed, but Hudson remained rock-solid, even after several heap exhaustion failures. On a lark I set MAVEN_OPTS to ‘-Xmx2G’ for the build and rebuilt it; it took a little longer to fail, but Hudson didn’t seem to mind.

Threadhell

The Threadhell build ran mvn hellbuild:threadhell, which causes maven to spawn threads calculating x = sin (x) indefinitely. This build gave me trouble on my Mac: the java process Maven ran in started ignoring signals and had to be killed the hard way (kill -9 or the Force Quit button in Activity Monitor).

Since this build will never end on its own, I forcibly killed it after a minute of activity. Or rather, I tried to: even the instance’s console was having trouble coping with the CPU load. Ten minutes after I stopped the build, Hudson’s Maven subprocess was still chewing up 99.7% of the CPU. Hudson did survive the anvil I dropped on the agent subprocess without complaint, but it wasn’t able to rein in its builder on its own.

Diskhell

Potentially the most destructive of the hellbuild plugins, and the reason I ran this in VMWare in the first place, Diskhell ran mvn hellbuild:diskhell, which fills an arbitrary file with zeros until it can’t fill it any more. The Hudson installation I tested this on intentionally had the work directory on the same partition as, well, the entire rest of the system.

Since Hudson and its build agents ran as a normal user, the resulting mess left the system operable. Hudson was even able to perform further builds using the few pages the Diskhell build left available. Once the disk really filled up, and df reported zero pages available, Hudson still remained operable (and even attempted builds), which impresses me a bit. It wasn’t able to do much useful work; eventually even its ability to remember that it was planning to build something failed as the build queue log filled up, but it never crashed and it recovered gracefully after I deleted the gigantic file the Diskhell build left behind. Unlike Bamboo, I wasn’t able to convince Hudson to mangle its own config files: attempting to change the configuration caused an ugly stack trace instead.

Science: It Works.

Overall, Hudson can be recovered from a few weird states without restarting the build server. However, it doesn’t seem to be able to recover itself: it won’t delete the build results on its own, for example, and it can’t actually kill off an unresponsive build agent. Since I consider Bamboo’s configuration-eating behaviour a bug, that puts Bamboo and Hudson about on par with each other for features; the choice between them comes down to price and preference.

The sources for the ‘hellbuild’ maven plugin and the example project are available under the MIT license from our public Mercurial server, or you can use it in your own builds by adding our public Maven repository as a plugin repository in your own build. If you cook up something useful, please submit a patch or a Mercurial bundle back to us at patches@unreasonent.com.