I would advise any software development department to make use of virtualisation – especially for appliances such as Build Agents. The choice of virtualisation product in a commercial environment typically comes down to one of two choices. Microsoft offer Hyper-V for free as a role on Windows 2008 and newer and VMWare provide ESX (both free and paid versions). Both are excellent solutions though at this time my preference is with VMWare – it’s simply a more mature product with a greater feature set. I have no experience of ESX setup/configuration however and whilst this is an area where Hyper-V scores well I would still put ESX ahead.

Both Hyper-V and ESX have similar features and either product will support a TFS build environment without issue.

One particular feature that is must have [for build eco-systems] is the ability to create template virtual machines. A template VM is a machine that is pre-configured and ready to run but repackaged as a template. The purpose of a template is to ensure successive deployments of machines based on the template will, obviously, inherit the template’s configuration and be to all intents and purposes ‘the same’.

Using a pre-configured template as a base for a machine should save considerable time and ensure consistency in the resulting machine. Build Agents are commodity appliances that simply ‘drop in’ to the TFS build eco-system or removed at will. They benefit from the template in both speed of deployment (subject to disk/network constraints) and consistency of content.

The first template to build is a base operating system and I follow a pre-defined set of actions when building one.

Create machine

Resources such as CPU, disk, memory should be sized appropriately. Whilst these are usually editable when a machine is deployed from the template, historical evidence shows that they are seldom adjusted. Resource over-allocation may soon become a problem if the template has excessive demands.
Virtualisation products typically provide support for fixed or expanding disk storage formats. If disk capacity is at a premium where the templates are stored it may help to use an expanding format.

Install Operating System

With recent versions of Windows there should be little to do bar map the DVD drive/mount the ISO and install. For those versions of Windows that support it, do not enter a license key – a template need not be licensed at this stage.

Install Operating System Service Pack

Where appropriate, insert CD/DVD/mount ISO and/or run the Service Pack installer. I perform this 1st as it generally provides the fastest way to get the most stable copy of the Operating System.

Install tools, customisations, etc.

Installation of system tools such as 7-Zip, .NET Framework, Silverlight, etc. Any system level customisations should also be applied here.

I do not usually add a template to a domain as templates have no purpose on a domain. I also name each template with a fixed naming convention (T-’OS’-'TYPE’). For example, a Windows 2008 R2 French template machine would have the following name: T-WIN2K8R2-FR.

Configure Windows Update

Even when the latest Service Pack has been recently released it’s normal for Windows Update to have an additional set of updates to apply. Sadly this can amount to significant download and installation time. I also enable Microsoft Update to allow updates for other Microsoft products (such as Visual Studio, SQL Server, Silverlight, etc.). Even though Microsoft are steadily improving the experience, it’s usually necessary to run this several times in between reboots.

Once complete I disable automatic updates on all virtual machine templates. In the context of a Build Agent, automatic updates that yield later versions of the C++ development runtime can cause issues.

Clean up machine content

Delete temporary folder files, any local downloads, internet history, etc. Not always necessary but I like to have my templates with as little context data as possible.

Convert the machine to a Template

Most virtualisation products contain inbuilt support for deploying templates. What I mean by that is they will handle machine identification changes such as SID regeneration and network name change when a machine is created from a template. If that’s the case, simply convert the machine as is. If not, Sysprep should be used to manipulate the machine state into something that will support being rolled out. Once Sysprep has completed (the machine should have shutdown) the machine can be converted to a template.

Hopefully the benefit of creating template machines with a ‘bare’ operating system installation should be obvious. It will be quicker to deploy the template in a consistent manner rather than repeat the above actions per machine. The next and final post of builds covers the Build Agent template virtual machine configuration and how components are built without Visual Studio.

If you’ve read the previous articles you will know that I consider TFS to have an excellent build eco-system. Other application suites offer similar services but somehow TFS manages to do exactly what’s expected with the minimum of hassle. Whilst many of the concepts are technology agnostic please be warned I’ll be using TFS as my reference point.

I’d first like to touch on how Visual Studio solution (.sln) files are typically used as it has an impact on how each build definition may be configured. Microsoft’s ‘guidance‘ is that everyone should work with a large solution file and it is that that is added as the component build item. This has some advantages:

• the developers always build all of the software to the extent as controlled by the solution file
• the solution file will define the build order using project dependencies so this need not be understood by everyone
• changes to the build order made in a solution are immediately reflected in the next build

Potential drawbacks are that:

• all developers will always have to pull out the latest source and build all components in order to start a task – if the task is small this could use a significant portion of the task’s allocated time
• Visual Studio solutions files with large amounts of projects are [still] slow – even after the improvements Microsoft has made in Visual Studio 2010
• changes to the solution must be carefully considered and controlled – build output may be affected
• finally, as all components are built on each developer workstation, all developers will typically need a full set of licensed controls even though their daily task output has no need for them thus increasing the potential cost and servicing impact

What’s my advice? Consider the options before choosing to use large solution files and if you can, don’t use them.

I practise a “work against the latest nightly build and only get the minimum code out to complete the feature” development style. All developers will be required to know the system composition so that projects may be built individually in the correct order. However, knowledge such as this being shared across the team should be considered beneficial to the ongoing project rather than detrimental.

A TFS build definition consists of the following major attribute groups:

• When – The choice of trigger (Manual, CI, Scheduled, etc. all covered in the last post)
• How – Build process template and associated configuration
• What – Location of what to monitor; list of components to build

TFS provides a default build process template that performs standard actions suitable for out of the box build operation. It’s normal to customise this process to suit the needs of the project. Additional actions may include resource version number updating, code signing binary output, etc. As the TFS build process template is built using Microsoft Workflow it’s extremely easy to customize.

Other aspects of ‘How’ include whether tests are executed, the build configuration type, source and symbol server integration, etc. These are fairly self explanatory and exposed in the build definition editor using simple UI options.

There are 2 aspects to ‘What’ is built:

1. the location of the source code repository that the build definition will work against – typically the root of a branch
2. the complete set of components that will be built from that location

With a single large solution file in use, the build definition will refer to it alone. If individual project files are used, the build definition will be more complex referring to the project file for each component in an appropriate order.

Whilst it’s not a difficult task to configure a build definition I would advocate the use of an assigned resource, a “build engineer” if you will, so that at least the responsibility lies with someone. This role can be shared with a developer or be completely separate depending on the size/structure of the company/product build requirement.

With TFS, builds are remotely executed on Build Agents that are managed by Build Controllers. There are typically many Agents to a single Controller and a single Controller per TFS Team Collection. As the product is highly configurable it could equally be multiple Controllers each with a single Agent. Whilst there is no right or wrong configuration the former provides a pattern that is probably better understood.

Each Build Agent uses a distinct working folder thus allowing for multiple builds to be running under different Agents on the same machine. Custom Tags are used to define a Build Agent that is configured for specific build duties, such as being hosted on a platform with a particular configuration. A build definition can use these tags to indicate to the Build Controller what type of Build Agent is required. Builds are then scheduled on the next available Agent that matches the Tag criteria. Should no Agent exist or be available the build will fail. A build definition that specifies no Tags will be scheduled on any Agent.

With the correct build definitions in place, the following key drivers for using any form of automated build should be realised:

• Traceability
• Repeatability
• Reliability

Execution of builds remotely means that the code is taken from the developer [as part of a Check-in] and built within the confines of the controlled build eco-system using one of the pre-defined build processes. The oft abused “it works on my machine” issue is minimised as builds are not performed on a developer’s machine where a potentially variable set of control and runtime versions may have yielded unpredictable results. Code contained in each and every build is also only ever taken from the content of a TFS branch (development/main/release) and is therefore fully traceable.

Build status reporting is via e-mail, desktop notification application or custom event subscription. Completed builds can then be assigned a quality indicator that may be used further in the development process.

After a very short time frame it’ll be hard to imagine returning to the ways of old!

Most developers will perceive running a build as the execution of Visual Studio in much the same way as a local build action is performed. It is also true that most small development teams simply install a full development environment on the Build Agent and queue builds using it and depending on the project type, that actually may be necessary. As I believe there is a better way of achieving the same result, that will be the subject of my next post.

A build is the term given to the process by which content stored in a code repository is transformed into deliverable artifacts. Most source code is compiled/processed into binary artifacts such as DLLs or EXEs using a known toolset initiated and controlled by a pre-defined build process. Each process usually follows the ‘get latest, build components, run tests and drop build output to deployment folder’ sequence. TFS provides an adequate starter in the shape of the ‘default process’.

Build output can be anything from loose files right up to an installation package. Whilst there is no ‘one size fits all’ answer I would always advocate the provision of an installation package to deploy an application. Using the right technology this can also created as part of the default build process.

Continuing on the branch structure introduced in the last post, there is usually more than one build definition configured (each may also use a distinct process):

• Development branches would typically use Continuous Integration (CI) builds
• Main branch would typically use Nightly builds
• Release branches would typically use Manual builds

Continuous Integration (CI)

CI builds are executed when content is “touched”. The purpose of this build type is that it ensures the quality of code repository content is kept high. For basic implementations the measure of quality may merely reflect how compilable the content is rather than be a measure of its functional quality. However, most CI build systems provide for the execution of tests as part of the build. With an appropriate set of tests, the functional quality of the build can also be measured. A build that compiles but fails testing is likely to be as undesirable as one that fails compilation, perhaps even more so!

TFS 2010 offers a number of different CI build options with one that is possibly still unique: ‘Gated Check-in’. The concept behind this build type is that it operates directly on a check-in. Only when the check-in content builds successfully will the check-in be completed. This is in contrast to most CI builds where the check-in is performed prior to build execution. The advantage of the Gated Check-in is that it’s not possible to check in code that does not compile thus ensuring a higher level of confidence in the stability of the code repository content. A disadvantage is that a single check-in must include all appropriate changes and if the check-in becomes dependent on another it may be impossible to submit a build for successful compilation. In this case I would still advocate the mechanism but allow the build action to be overridden. Different teams have different ways of working so a more standard CI build or a ‘roll-up’ CI build may be chosen instead.

I consider CI build output very transient so the build definition in TFS is set to keep output only from the last few successful.

Nightly

Building the content of the code repository on a nightly basis should be considered the minimum for any project. A nightly build simply provides confidence that the code repository content will support the following day’s code changes. A failed build should be addressed as soon as practicable.

I execute the nightly build against the Main branch as Main is considered stable and it is the baseline for other branches (development and release). Main is updated (merged) with from content from Development branches so code churn is less than that of the source branch. The Development branch content is already considered high quality (it executes Gated Check-in builds with test execution) so any issues should only arise from the result of the merge operation.

Nightly build output is considered worthwhile. I leave the TFS defaults of ‘last 10 successful builds kept’.

Manual

With TFS any build may be executed manually. With both CI and Nightly builds in place there’s less requirement to run a build process manually but the option remains.

However, Release builds are, by comparison to the others described here, executed far less frequently and almost without exception executed manually. A release build will usually have a different process to support the official software release. This may be the in the form of version resource management, code signing of output, release packaging, etc.

Successful release builds are always kept as they form a record of what official releases have been made.

One of the key objectives in investing in this type of build configuration is to minimise the ‘it works when I build it on my machine’ scenarios that are often prevalent in small development teams. Knowing exactly what was included in a release package and how it was generated is key to understanding how situations arise when the application is in front of a customer.

It should be understood that this phase requires a resource that has a good range of experience. Building an application (even a simple one) usually requires an understanding of the software structure, the intended composition and the expected platforms. Whoever is managing this should have ideally been involved in the previous phases on this or other projects so that the task is not performed blind.

The next post will delve into how to configure TFS build ecosystem to support the builds described.

The assumption is you’ve had your application composition and software structure defined and everyone is desperate to get stuck in to code generation. After all, we’re all doing Agile and the business expects results [because of it]!

Developers typically submit code to the repository and move on to the next feature. After all a developer’s purpose is to develop features and the job of pulling the source together is not usually a shared responsibility.

For small development teams, < 5 co-located for example, it’s perfectly possible to get away with the bare minimum. That is Microsoft Visual SourceSafe with the benefit being that it takes no time to install, there’s not a lot of configuration to it and it [mostly] works. There are many horror stories on the Internet regarding VSS but in reality you have to do something to break it, it doesn’t just break itself.

However, even Microsoft have decreed that VSS is dead and the new solution going forward is Team Foundation Server. Going from VSS to TFS 2010 is daunting to say the least and it must be said that it might be better to consider alternatives such as Git or SVN. For the 2010 release, Microsoft provided a variant of TFS called “TFS Basic 2010″. Providing the full feature set of TFS bar Sharepoint and Reporting Services integration it’s described as “TFS 2010 for SourceSafe Users”.

Unlike TFS 2010, installation of TFS Basic 2010 is supported on Windows 7 machines so there’s not even a requirement to be running it on a server. As the product is quite mature now I’ll simply point readers to the web for details on how to install TFS Basic 2010. Note that out of the box it’s licensed [not limited to] for 5 users; additional users require a CAL and those costs may add up.

The Code Repository store for TFS is a SQL Server database. The developer need not be aware of that of course but the system administrator should ensure the TFS databases are backed up appropriately. Since release a Backup Wizard has been added to TFS to make this as simple as possible.

Small development teams with VSS probably rarely made use of Branching – almost certainly in stark contrast to teams who used open source equivalents. For those readers wondering what Branching is, it’s a mature topic and whilst the question has already been asked, a one liner from me is:

a branch is a separate code stream created from a base in order to apply a distinct set of changes.

Branching therefore allows multiple streams of work to be active simultaneously and should increase productivity by actively supporting the development process. Whilst VSS branching always seemed to lead to problems, both technical and user orientated, TFS offers a robust solution and all development teams large or small should make use of it.

I would advocate the following branch configuration as a minimum:

• Main (contains stable code)
• Development [n] (work in progress branch, created from Main – effectively v-next)
• Release [n] (official releases, created from Main)

Other patterns exists – branch on feature, branch per developer, branch for serviced release, etc. An existing development process is more likely to define and drive the chosen pattern rather than there be any considerable technical advantage in choosing one above another.

For reference, Microsoft keep an active project on Codeplex called Visual Studio TFS Branching Guidance that is now in its third iteration.

What’s this got to do with builds, are we not way off track? Not at all, Branching should be integral to all development teams – especially Agile ones – and therefore must be implicitly supported by the build ecosystem. My example branching configuration defines a number of areas where builds may be executed and the next post will describe each in some detail.

Before anyone can get to building components there’s usually a large amount of [largely unseen] effort required to define application composition, software structure, code repository configuration and deployment plans. I’ll set the scene by defining what I consider those terms to mean:

Application Composition

This is just a fancy way of saying “how is the solution going to be shaped”. Almost always the classic onion metaphor will be used. Each layer may have contractual dependency on others but know nothing about the internal implementation. This phase is key – even in an “Agile” world. Don’t start coding without a pretty good idea of where you’re going and what technology (not technique) you’ll use to get there. To succeed in this phase you need experience, plain and simple.

Software Structure

Standards, patterns, components, etc. to be used in production of the solution. Some assets may be carried over/re-used/enhanced from previous projects, such as standards, whereas others may be specific. Some developers will probably do a good job here though it’s often best left to the architect.

Code Repository

A place where a developer stores code that is considered ‘complete’. It is not a backup medium (though of course it is imperative that it is covered by a very good backup strategy). Although the repository can be simply a shared folder on a disk it makes little sense as there are so many better options such as GIT or SVN, both of which are well respected open source projects. My experience is with Microsoft TFS so I’ll be using this as the basis for my posts.

Deployment

To me, defines both the process and anything considered a deliverable generated from the process. Binary deliverables are the set of assets generated from source code that would be presented to appropriate platform(s) for the purposes of achieving the goals of the solution. Deliverables could equally be miscellaneous data files. Typically an installer package would be used to contain and manage deliverable assets. In summary, deployment covers the bit after the Developer and before the Tester. Generally speaking this is perhaps the least considered phase of those I’ve listed here and is the purpose behind the post.

The next post will backtrack to the code repository stage so that I can define how a developer interacts with code and what affect that may have.

Before anyone asks “where’s the source code?”, I need to do a clean-up of a couple of the classes and when it’s in reasonable shape, I’ll upload it to the WordPress plugin repository.

It uses as custom mySQL table to store entries, WordPress options table to store plugin option data and pChart 2.0 for graphical presentation.

Inspiration for the plugin was the “Runners Log” plugin that I had used for a number of months. One thing that I didn’t like with that was the need to add a new post just to get an entry logged. Whilst I could have hacked around that I felt that I would ultimately be happier with something specific to my needs. I’ll be adding a number of output features over time so consider this release a version 0.1!

Any comments please let me know via the contact page.

Anyway, back to the PC… it actually performed its last calculation a few weeks back! Failing to boot and eventually corrupting the disk once again I decided to retire it. The empty shell of my Suntek Grace aluminium case has been sat by the side of my desk since.

I don’t have a need for a gaming machine (reasons that should be obvious when considering this is a PC!) so I went for something reasonable rather than spectacular. The CPU is a top of the line SandyBridge (2600K) as it was listed on the Scan Today Only page with what should be a decent motherboard (Asus P8Z68-V).

Hopefully all the bits will be here by the week end and I’ll be with something more useful than the laptop I’m currently using (its good but it’s certainly no desktop replacement). I’m still hopeful Duke Nukem Forever will provide a few hours of good old fashioned fun. If not I can continue to marvel at my adventure game collection running under ScummVM

History of my Babbage Engines:

• 286
• Intel DX-33
• Intel DX4-100
• Pentium 133
• Pentium 2-350 -> later upgraded to 550Mhz
• Pentium 3-1Ghz
• Pentium M760 2ghz

The later machines lasted significantly longer in the purchased form than those earlier although the internals of each PC was gradually upgraded over its lifespan. Hopefully the main components of the new machine will last as long as the previous.

I’ve verified that the EA Protocol Handler works with this version without change. Should anyone find anything untoward then consider dropping a post to the Sourceforge project page’s forum.

$user can be very useful when tracing multi-tier calls with delegated tokens.

In all honesty I have very little time and interest to play games on my PC these days. My PC became nothing more than a Visual Studio workhorse – a sad fact that I do way too much unpaid work at home.  Over the years it has also started to become unreliable so a return to gaming will definitely require a kit refresh. Intel have a solid product in their “Sandy Bridge” CPUs although marred by stupid CPU/motherboard combination restrictions.

You never know, hell might even freeze over and I will install Windows 7 It didn’t last more than a weekend when I tried it out a year back (it does pretty much nothing for a developer and is best on hardware more recent than my current PC).

When the day comes I’ll post about the beating my wallet will have taken… It will quite possibly be the last “PC” I buy as the PC games industry is dead [to me] and the chance of Windows severely outstripping hardware is likely to be a thing of the past.