Hudson

There has been some amount of discussion on the Hudson user's list recently about the status of the "Locks and Latches" plugin. The plugin allows for one to create "locks" for Jobs in a similar manner to how "locks" work in a multithreaded programming environment. The need for such a plugin becomes very clear once you start to run multiple jobs that depend on some set of shared resources, take the following example:

• Jobs A,B,C must run unit tests that fetch data from a test site
• Slave #1 can only run one instance of Apache at a time

How one would accomplish this with the Locks and Latches plugin would be to create a lock like "Site Lock" in the Hudson configuration, and then bind Jobs A, B, C to that Lock. Making the (large) assumption that the plugin works correctly and locks properly in order to prevent A and B from running concurrently, this would be enough to satisfy the requirements we have for the scenario above. Unfortunately it seems the plugin is largely unmaintained and buggy; in the past couple weeks of experimenting with such a set up on a variety of different slaves we've noticed that the locks aren't always respected, causing some locked jobs to execute in parallel spewing bad test results and build failures (the crux of this issue seems ot have been reported by Sergio Fernandes in #2450).

The Loopback Slave

During the usual friday-frenzy I sat down and wrote a quick 10 minute little bookmarklet to start a Hudson job. Unlike most bookmarklets that "do things" this one actually "does things" but doesn't take you away from your current page. Using the Hudson Remote Access API you can query information from Hudson programmatically, but you can also kick off builds remotely with nothing more than a simple HTTP request to the properly formed URL.

By dragging the link below to your bookmark bar, and updating the URL within ("http://hudson/") to the URL of your Hudson instance, you can queue a Hudson build from any page at any time (without leaving the page).

The Bookmarklet

The Code

<a href="javascript:var b = prompt('Enter a job you would like to build:', 'main'); if (!b) { } else { var i = document.createElement('img'); i.src = 'http://hudson/job/' + b + '/build'; i.height = 1; i.width = 1; document.body.appendChild(i);}; alert('Build ' + b + ' queued');">Build Hudson Job</a>

I've been using a Gnome-based desktop for about the past 8-9 months and one of the things I've come to really appreciate is that most Gnome applications integrate with "libnotify". Libnotify is a simple Windows taskbar-like notification system that presents status messages at the bottom of your screen. Like all great pieces of software, it has a solid Python interface which allows for incorporating it in those little 10-minutes scripts I find myself writing every now and again.

One of the things I wanted to script was the notification of the build status of the numerous jobs that we're running in our Hudson instance here at Slide. Using the Universal Feed Parser and pynotify (listed under "notify-python"), I had a good little Gnome Hudson Notifier running in less than 10 minutes.

Source code after the jump.

I recently wrote about "one-line automated testing" by way of Hudson, a Java-based tool that helps to automate building and test processes (akin to Cruise Control and Buildbot). If you were to read this blog regularly, you'd be well aware that I work primarily with Python these days, at a web company no less! What does a web company need with a continuous integration tool? Especially if they're not using a compiled language like Java or C# (heresy!).

As any engineering organization grows, it's bound to happen that you reach a critical mass of developers and either need to hire an equitable critical mass of QA engineers, or start to approach quality assurance from all sides. That is to say, automated unit testing and automated integration testing becomes a requirement for growing both as a engineering organization but as a web application provider (user's don't like broken web applications). With web products like Top Friends, SuperPoke! and Slide FunSpace we have a large amount of ever-changing code, that has been in a constant state of flux for the past 16-18 months. We can accomodate for ever-changing code on the backend for the past year and half with PyUnit and development discipline.

For about as long as my development team has been a number larger than one, I've been on a relatively steady "unit test" kick. With the product I've worked on for over a year gaining more than one cook in the kitchen, it became time to start both writing tests to prevent basic regressions (and save our QA team tedious hours of blackbox testing), but also to automate those tests in order to quickly spot issues.

While I've been on this pretty steadily lately, I'm proud to say that automated testing was one of my first pet projects at Slide. If you ever crack into the Slide corporate network you can find my workstation under the name "ccnet" which is short for Cruise Control.NET, my first failed attempt at getting automated testing going on our now defunct Windows desktop client. As our development focus shifted away from desktop applications to social applications the ability to reliably test those systems plummeted; accordingly our test suite for these applications became paltry at best. As the organization started to scale, this simply could not stand much longer else we might not be able to efficiently push stable releases on a near-nightly schedule. As we've started to back-fill tests (test-after development?) the need to automate these tests has arisen to which I started digging aronud for something less painful to deal with than Cruise Control, enter Hudson.