In recent months, I’ve had the opportunity to set up two build servers; one using Cruise Control.NET and the other using Team Build (part of Team Foundation Server). In this article, I’d like to chronicle the strengths and weaknesses I’ve found in both systems in my particular environments. My experience with each build system is admittedly limited, but I hope the information below might prove useful to someone. Note that the source control system in both environments was Team Foundation Server 2008.
Why Have a Build Server?
Before I compare the two build systems, I should first point out the advantages of having a build server. In my case, the build server provides three essential services:
- Continuous Integration. As soon as a developer checks in some new code, a build is performed to ensure that the new code doesn’t break the build. If it does, everyone knows about as soon as possible so it can be fixed. No more waiting for integration builds to find out that there’s a problem.
- Automated nightly builds. Nightly builds provide your QA people (if you’re lucky enough to HAVE QA people) with a regular, reliable heartbeat of incremental builds. QA knows that every morning there will be a new build they can test.
- Reliable, repeatable installers. Our build server also outputs an MSI installer that allows the software to be installed much as if it was a shipping product. If someone needs a new installer or wants to search for a particular build, the build server is the place to go. If you don’t HAVE a build server, you need a developer to perform a build on their workstation instead and if there’s no developer available, you’re out of luck.
I believe build servers are an essential part of modern software engineering – if you don’t have one yet, get one set up as soon as you can.
Cruise Control is certainly the most popular build system available. It’s an open-source solution that has been around for years, works with all kinds of source control and repository systems, and has a huge community of users and plug-in developers. A spin-off version especially for .NET developers (herein called CC.NET) is managed by ThoughtWorks and is available from their site. CC.NET has a nice web-based dashboard for viewing the status and history of builds, as well as allowing control over the build process. A single CC.NET server can be configured to build multiple projects.
I think one of the strongest reasons to pick CC.NET is its maturity and resulting pervasiveness. It’s been around for what seems like forever, and if you’ve ever had to deal with a build server in either a Java or .NET environment, chances are you’ve used it already. In addition, the CC.NET plug-in developer community has written solutions for just about every conceivable problem you might have. There is a wealth of knowledge on the Internet on how to set it up and get it working, so it’s hard to go wrong with CC.NET.
Something that’s both a strength AND a weakness with CC.NET is that it's not part of the Team Foundation Server (TFS) ecosystem. This is a strength in the sense that it allows you some flexibility with the CC.NET server’s configuration. A single CC.NET server can work with multiple TFS installations (and/or other source control providers) and support multiple projects. In my particular case this was a huge reason to use CC.NET. As you’ll see below, Team Build has some restrictions on the account used to run the Team Build service, while CC.NET allows you to communicate with the target TFS server by simply specifying a username and password with TFS access. This “disconnection” with TFS is also a weakness in that the central TFS server has no idea what build environments are out there. There is is no automatic central storage of build profiles and definitions, and no way to automatically interact with build information from within the development environment. Once, one of my colleagues modified his build script to delete all files in a particular directory. Unfortunately, when it ran, it resolved that directory to the C:\ drive root folder, completely obliterating his build environment. The problem was exacerbated by the fact that the CC.NET build configuration had never been backed up so he had to start all over from scratch!
Another thing that I would list as a weakness for CC.NET is the fact that it uses its own XML schema for controlling builds. This means that you have to learn this schema inside and out in order to make maximum use of all CC.NET has to offer. While many may see this as a minor point (and it really is) it is nonetheless an additional barrier to getting CC.NET installed and configured to perform your builds.
Team Build 2008
Team Build is a service that comes with Team Foundation Server. Having used CC.NET before, I initially found the architecture of Team Build very confusing. Instead of setting up a build server, creating a build definition ON the build server, and pointing it at your source control server, Team Build handles things very differently. First, you install Team Build on a computer. That computer then becomes a “Build Agent” that the master TFS server can talk to. This is a major architectural difference: with CC.NET, the CC.NET server initiates communication with TFS during a build; with Team Build, it’s TFS that initiates the communication with a build agent of your choosing.
Because of this architecture, your build definitions don’t actually live on the build server. The build definitions are part of your TFS project and are under source control themselves. This centralizes the build definitions for the project where they can be deployed to any build agent in your environment simply by selecting that agent when you perform a build. Well, when I say “any build agent” I mean any build agent that has any dependencies installed on it that your particular project might need. In my case, that means a couple of MSBuild extentions, which isn’t very heavy-weight as far as dependencies go – they can be installed in about 30 seconds.
So, the centralized storage of build definitions can be a very good thing. Looking back at the problem my colleague encountered when his C:\ drive was deleted, well, that could still happen with Team Build but at least the build definition would still be intact since it is under source control. However, this centralized storage of build definitions has a downside, too – all build agents (i.e., build servers) must be on the same Windows domain as the TFS server, or at least on a trusted domain. In addition, the Team Build service must run under a domain account (so you need the password for this domain account when you set up Team Build). This can present some problems if you’re in an environment, like mine, where IT controls the TFS server and the domain it’s attached to, and getting your build server attached to the same domain requires requisitions, approvals, security checks, etc. To be sure, this can a royal pain in the neck and is where I see one of CC.NET’s biggest advantages. You can put CC.NET ANYWHERE and have it talk to TFS with no problem. Of course, you’ll need to put a TFS username and password in clear text in your CC.NET configuration file and that or may not be a cause for concern in your environment.
Another advantage of Team Build (sort of) is that it uses the same XML language to control builds as is used in your existing Visual Studio projects. Specifically, it uses MSBuild to define and control the build definition. So, if you’re already familiar with MSBuild, you can figure how to configure Team Build fairly easily. I was NOT familiar with MSBuild and I had a helluva time getting my builds to work the way I wanted, but then again, I was trying to do some fancy things with automatic assembly version number increments and integrating a couple of WiX installer projects. Like CC.NET, MSBuild has a community of extension developers and while I haven’t performed a formal comparison of the two communities, I would expect CC.NET to have a lot more available simply because it’s been around so much longer.
A major disadvantage of Team Build is a pretty heavy dependency in some cases. If you have created unit tests using the built-in features of Visual Studio, and you want to run those unit tests as part of your build, well you have to install Visual Studio on your build machine. Yuck. The same is true if you have a Visual Studio Database Edition project – you must install that edition of Visual Studio too if you want to build that project. I’m really hoping for a solution to this requirement in the next version of Visual Studio and TFS.
Lastly, while CC.NET has a pretty spiffy web dashboard for status display and build control, Team Build has its status and control built directly into Visual Studio. While this means that you can interact with your builds without leaving the development environment, it also means that anyone WITHOUT Visual Studio can’t see anything (unless your TFS server has Team System Web Access installed).
There’s obviously a lot more involved in the comparison of these two systems, but I hope this blog post gives you someplace to start. Check out the resources below for more information.
If you’re trying to decide which build system to go with, I suggest you check out the following resources which contain some additional helpful information:
Cruise Control .Net vs Team Foundation Build on StackOverflow.com
Buck Hodges’ Blog (TFS Development Manager)
Jim Lamb’s Blog (TFS Program Manager for Team Build)
MSBuild Extension Pack
MSBuild Community Tasks Project