Andre's Blog

Personal blog of Andre Perusse

Build Engineering = Sorcery

Most of my software engineering career has been involved with internal intranet applications. Since applications were typically only deployed once on one application server, I’ve never had the opportunity to explore the wondrous world of build engineering. For the last two years, however, I’ve been working for a software vendor and build engineering is very much an important part of the development process. And until recently we’ve had a dedicated build engineering department that looked after most of the nasty details involved with this tricky voodoo science. But that department is now gone, and my team has to look after its own build requirements.

For the past 4 to 6 weeks, I’ve been delving deep into the bowels of MSBuild, the technology provided by Microsoft to perform builds of .NET projects. When you select “Build” from within Visual Studio, it’s calling MSBuild to do the work and generally Visual Studio shields you from all the intricacies of this build engine. However, when you need to perform automated builds, properly version your output assemblies, and automatically create MSI installers, you really need to learn how MSBuild works. And let me tell you, it is pure sorcery. You have to a be a bloody wizard to understand how all of these pieces interact with one another and how to control them to do your bidding.

At the root of the problem is the fact that the build engine and pretty much every artifact that interfaces with it is represented by an XML file. I used to love XML files – it solved so many of my problems trying to store and retrieve structured data where a SQL database was not available or suited to the task at hand. Now I’m starting to develop a true resentment of this once heroic technology that has been twisted to perform evil. It pretty much boils down to this: XML is being used to specify the granular tasks that must be performed to compile and output your entire solution, including all the projects, and all the classes and other files included in those projects. XML in this case is not being used to represent data storage, but instead is being used as a primitive command and control language. Individual XML nodes can be used to set variables, to check conditions, and to execute tasks. To create these XML files, you must be a master of both the XML schema available to you, as well as how the actual build engine interprets the schema. Though Visual Studio provides some rudimentary Intellisense when editing XML files, all it can show you is a list of valid elements and/or attributes. And there is NO compiler for these XML files – the only way to see if what you’ve written is syntactically valid is to actually run it! Holy smurf, I feel like I’m back in 1983 or something.

To complicate matters further, MSBuild and its related technologies (Visual Studio, WiX, Team Build Server) just LOVE to scatter all kinds of environment and preprocessor variables throughout these XML files. While extremely helpful when performing tasks, these variables are damn near inscrutable – the development environment provides you little assistance with determining what variables are available in any given location, what they mean, where they’re coming from, and what process is setting their values. You must scour through reference documentation, online forums, and the occasional helpful blog to find out just what the hell is going on at any point in time. Writing XML code to control your build and your installer is very much like reciting some obscure Latin incantation to perform some magic. You just have to hope you’ve said it in just the right way and with the proper inflections so that that the build gods will oblige your request. It’s enough to drive anyone completely batty.

Now, where did I leave my wizard’s hat?