Simple Template For Tracking OKRs Weekly And Quarterly

Objectives and Key Results (or more commonly knowns as OKRs) are a well-defined way to codify a company's strategy along with tracking progress towards the intended results. John Doerr introduced OKRs at Google to great effect and has since written an insightful book called Measure What Matters that goes into the OKR framework and how it's successfully being used at various companies.

The OKR process can become overly complex rife with over analysing the data behind the numbers, poorly quantifying the key results and/or poorly communicating the ongoing impact they're having. I have seen many companies abandon the initiative part way through a quarter or even after a few quarters of failed attempts. 

I've distilled the OKR process for product engineering teams and the wider company down to a simple tracker that can be used on a weekly basis. Below I will explain the "team tracker" along with some examples, followed by how each team's OKRs come together into a simple view across the company that anyone can understand.

Team OKR Tracker

The team OKR tracker is the starting point for the manager of each team. OKRs don't have a mandated length of time but I find doing them on a quarterly cadence allows enough time to make the upfront planning worth the time invested while still allowing for regular points throughout the year where strategic initiatives can be adjusted. Here is what a fresh version of the template looks like prior to a team's manager populating it with objectives, key results, owners and their associated weekly progress tracking:

There are several parts of the above sheet template which are worth digging into further:

• Objective: This is what you want to accomplish. An objective should be aspirational, significant to the company, personally meaningful and ambitious. I'd recommend having no more than 5 but ideally 3 or less objectives per team (the old adage of less is more can't be said enough when setting objectives).

• Key Result: This is how you will track the achievement of the objective. Key results should be easily measurable and limited in number. This is how the company measures progress towards its objectives while remaining flexible on the exact tactics to implement.

• Owner: There should be a single owner for any given key result who is responsible for driving its success. In a product team this is often the CTO, Head of Product/Engineering/Design, Product Owner, Scrum Master, Tech Lead, etc.

• Weekly Percentage Tracking: Each week throughout the quarter the owners of the OKRs should get together to discuss the progress of each one based on the percentage in the spreadsheet. Similar to a daily standup within a product team, any blockers or impediments should be raised and decisions made to resolve them. 

• Final: This is simply the final result of each OKR at the end of the quarter. Note that sometimes it may take a week or two in order to determine the steady state of actual metrics associated with a given OKR (e.g. waiting for A/B test results for a feature deployed on the last day of the quarter).

• Forecast: This should be regularly discussed and re-forecasted if necessary to get an accurate picture of where the progress toward an OKR might end up at the end of a quarter. This is obviously hard to achieve with complete certainty but even when accounting for the usual margin of error, it's still a valuable way to indicate to everyone which OKRs are on track and which ones are struggling.

• Notes: Along with updating the weekly percentage of the OKRs, each owner can add a few notes about what is happening. This can be particularly valuable when a key result is not moving (aka still at 0%) but the team is making good progress with the build which will be deployed in the future and which will subsequently increase the key result pecentage.

• Averages: This is simply an average across all the OKRs for the team. If teams are aiming for the usual ~70% achievement rate (aka the OKRs taken together were ambitious but not impossible) this section highlights how the team is tracking on that overall goal.

Team OKR Tracker Examples

The team OKR tracker below has been populated with examples of potential objectives, key results, owners and associated weekly progress tracking for a fictional product engineering team:

Company "Roll Up" OKR Tracker

Each team within the company (e.g. Sales, Product, Marketing, People, etc) has their own OKR tracking sheet which is then "rolled up" into an overall high-level view. This view shows progress across the company against the previously-defined OKRs. It is able to give the executive team, the board and/or the rest of the company (via a monthly all hands meeting or weekly email) a simple view of which teams are doing well and which teams may need some additional help or resources in order to achieve their goals before the OKR period comes to an end.

Google Sheet Template

I've provided the OKR tracker as a Google Sheet which can be easily copied and used within your own company: https://docs.google.com/spreadsheets/d/1aEpVezdaaxrDv7hxex25ozS6QD_deP5cUcoKLn35aww

My podcast with AWS Startup Stories about hiring for startups

On June 23rd, 2017 AWS released the final podcast in a 6 episode series titled AWS Startup Stories. The series of podcasts cover a range of topics from startup fundraising (Crowdcube, LocalGlobe) to scaling a startup (Monzo, Starling Bank, Once dating app) and growing and retaining a team (Makers Academy, Yoyo Wallet).

In episode 6 Will Bentinck, Head of Careers at Makers Academy and myself, VP Engineering at Yoyo Wallet discuss the ins and outs of hiring for startups and how to retain talent long-term. The 30 minute podcast was hosted by Darren Mowry, Head of Startup Business Development and Venture Capital at AWS EMEA.

10 Steps To Priming Kanban

Jesper Boeg has written one of the best and most succinct PDF docs on Kanban (43 pages in all). He breaks Kanban down into 10 steps that can be implemented piecemeal until your team has a comprehensive Kanban process governing all aspects of software development.

Priming Kanban by Jesper Boeg from Andrew Thompson

3 Responsibilities of Product Management

Adam Nash (prior VP of Product Management at LinkedIn) describes what it takes to be a great product leader. He boils down a product manager's job description to three key responsibilities.

Responsibility #1: Product Strategy

Adam describes product strategy this way: "it’s the product manager’s job to articulate two simple things":

1. What game are we playing?

• What is the vision of the product?
• What value does it provide customers?
• What is the differentiated advantage over competitors?

2. How do we keep score?

Clearly answering these questions synchronizes teams across the organization and helps them understand how to win effectively in the market. 

Responsibility #2: Prioritization

Different processes exist to handle the prioritization of features and tasks: Waterfall, RUP, Agile, Kanban.. etc. But without a solid product strategy, prioritization becomes very difficult to do effectively. Adam describes a framework for product planning which he calls the Three Feature Buckets:

1. Customer Requests: These are features that your customers are actively requesting. 
2. Metrics Movers: These are features that will move your target business & product metrics significantly.
3. Customer Delight: These are features that customers haven’t necessarily asked for, but literally delight them when they see them.

Features may fit into more than one bucket but rarely fit in all three. The benefit of classifying each feature this way is so that a team can be intellectually honest with themselves about why they should implement a particular feature.

Responsibility #3: Execution

Execution is all about shipping the product and getting it in front of users so that they can derive value from it. Sometimes this means ASAP and sometimes it requires timing the market effectively. Teams have many different approaches on how they execute ranging from light weight idea, test and deploy methodologies to full-blown specifications, sign-off, development, QA and release cycles. Adam describes the 4 parts of execution that are critical to its success:

1. Product specification: The necessary level of detail to ensure clarity about what the team is building.
2. Edge case decisions: Very often, unexpected and complicated edge cases come up. Typically, the product manager is on the line to quickly triage those decisions for potentially ramifications to other parts of the product.
3. Project management: There are always expectations for time/benefit trade-offs with any feature. A lot of these calls end up being forced during a production cycle, and the product manager has to be a couple steps ahead of potential issues to ensure that the final product strikes the right balance of time to market and success in the market.
4. Analytics: In the end, the team largely depends on the product manager to have run the numbers, and have the detail on what pieces of the feature are critical to hitting the goals for the feature. They also expect the product manager to have a deep understanding of the performance of existing features (and competitor features), if any.

Appointed as VP of Engineering at RESAAS

I am thrilled to accept my new positon as the VP of Engineering at RESAAS. I joined the company 8 months ago as the first full-time software engineer and with the help of Tom Rossiter, CTO, we have grown the development team to 8 full-time backend, frontend and quality assurance engineers all in 6 short months. Our team has been able to successfully expand in such a short time-frame by utilizing Agile methodologies, test automation, continuous integration and database change management tools along with the dedication of some amazingly talented engineers who work very hard to continuously improve the processes, the tooling and most importantly the product RESAAS is offering.

The enterprise social network that our team is building has evolved considerably over that time period from a single .NET and SQL Server based system hosted in a data center to a scalable platform hosted in the cloud (Windows Azure). Our core platform has been able to leverage Worker roles, Web roles, SQL Azure and Azure Table and Blob Storage along with various other 3rd party services to connect real estate professionals and their clients together in real-time via their laptop or chosen mobile device.

I am looking forward to working hard and developing the full potential of both our engineering team and the RESAAS core platform in the coming months.

Note that RESAAS is a public company currently listed on the CNSX under the symbol: RSS. Here is the press release on the CNSX of my appointment as Vice President of Engineering at RESAAS. 

Data-tier Applications, DAC Framework and SQL Azure Integration with a Continuous Integration (CI) Process

:: Introduction

Leveraging the efficiency and cost savings of the "cloud" within an enterprise requires the use of specific tools applicable to the PAAS provider used. When it comes to working with SQL Azure (Microsoft's cloud based relational database engine), I have found that data-tier applications along with some supporting deployment infrastructure (PowerShell Scripts) work particularly well to synchronize both the database changes across a development team and to deploy those changes to other testing environments as they make their way to a production system.

:: Data-tier Applications

Data-tier applications are a project type within Visual Studio 2010 that allows development teams to manage database changes across ALL of their environments whether in the cloud or on-premiss. Developers work with individual SQL scripts within the project which define anything from table schema's to indexes to stored procedures. Once a change has been made to a data-tier application's scripts, the entire project can be compiled and a DAC Package (called DACPAC from here on out) is then created that can be deployed to a SQL Server 2008 R2 or SQL Azure database instance.

For details on how to create and setup a data-tier application for the first time see this link. 

:: DACPAC Files

A DACPAC is simply a file with a ".dacpac" extension that contains all the schema, stored procedures, users and policies required to setup a new database or upgrade an existing one. Note that DACPACs should not be confused with BACPACs. BACPACs are a newer format put forward by Microsoft and the file format can contain everything a DACPAC can but can also include data. 

There is also a version number (with a format of x.x.x.x) embedded within the DACPAC file which can be used to version the DACPAC. Unfortunately this version number is not updated automatically by project when the data-tier application is modified. Therefore a developer will need to increment it manually before deployment. To access and modify this version number right-click on the data-tier application project and find it under the "Project Settings" tab.

:: Deploying Data-tier Applications

Before a data-tier application can be continuously upgraded with a newer version (the typical workflow of any development team), it must first be deployed. The deployment essentially creates the initial database, sets it up as a data-tier application within the database engine (either SQL Server 2008 R2 or SQL Azure) and then goes about deploying all the schema, store procedures, users... etc that are contained within the DACPAC.

The DACPAC can be easily deployed from within Visual Studio by right-clicking the data-tier application project and selecting deploy (note that the connection string within the data-tier application project will need to be setup before doing this). Alternatively you can use SQL Server Management Studio (SSMS) or some PowerShell commands: How to: Deploy a Data-tier Application

Note that the above deployment is really just a one time operation. Once the initial setup has taken place and the data-tier application is deployed, you'll only need to upgrade the data-tier application from then on (unless you delete the database and start again which is handy for certain developers who might break things and need to start afresh).

:: Upgrading Data-tier Applications

There are a variety of mechanisms available to deploy a DACPAC file to a SQL Server 2008 R2 or SQL Azure database instance. Unfortunately the documentation available for these mechanisms is fairly minimal and therefore it makes it difficult to assess what mechanisms to use for each deployment scenario you may have within your enterprise. Here are the ones I have come across, the way to invoke them and the scenario(s) I think they are applicable to:

• Within Visual Studio
• Invoke: Within Visual Studio right-click on the data-tier application project and select "Deploy". Note that Visual Studio figures out
• Details: Depending on the current configuration (e.g. Debug/Release) the appropriately set connection string with be used when the project is deployed. Therefore the connection string can be changed to access a local SQL Server 2008 R2 database instance or a SQL Azure database instance
• Scenario: This deployment scenario is most applicable to developers working locally during  a typical development workflow of repeatedly modifying code, compiling and deploying it and then debugging the result. I highly recommend that developer's use a local instance of SQL Server 2008 R2 for two reasons:
• No data transfer costs are accrued as ALL data transfer occurs on a developer's local machine and not back and forth between SQL Azure and their local machine.
• DACPAC deployments are much faster and more reliable when performed locally since deploying a DACPAC to SQL Azure can take minutes/hours depending on the connection and the database connections can be dropped mid-deployment (this won't cause an error but the deployment will need to be restarted)
• This 20min video, by a program manager on the Microsoft SQL Server Developer Manageability team, explains a range of information about data-tier applications and their use within Visual Studio.
• Within SQL Server Management Studio (SSMS)
• Invoke: Within SSMS open the "Management" folder and then the "Data-tier Applications" node. Right-click the data-tier application you would like to upgrade and select "Upgrade Data-tier Application..."
• Details: Just follow the on-screen instructions making sure to save the SQL Script that the DACPAC uses to apply the changes to the database instance. On some occasions the upgrade wizard has failed but because I saved the SQL Script I could execute it myself within SSMS.
• Scenario: This is the easiest way to get started when deploying a DACPAC to SQL Azure since SSMS provides a easy to follow wizard. Note however that I have seen this entire process take 1-2 hours to upgrade a database (40 tables + 50 stored procedures). Being able to automate this step is very helpful which the next section on PowerShell scripting will explain.
• Using a DAC Framework enabled PowerShell Script
• Invoke: You'll need to create your own PowerShell script to manage the upgrade process of one or more of your data-tier applications residing on SQL Server 2008 R2 or within SQL Azure. Note that if you are unfamiliar with PowerShell and the DAC Framework creating a robust and reliable script will take significant development effort.
• Details: I created my own parameterized PowerShell script that accepts a DACPAC file and some credentials to a SQL Azure database, subsequently connects to it and then upgrades the existing data-tier application. Two sources of information that I used when creating my PowerShell script were the following:
• How to: Upgrade a Data-tier Application
• SQL DAC Examples
• Scenario: This is the most appropriate method for building an automated process for managing data-tier applications that can be used by a continuous integration (CI) system.  
• Note that side-by-side upgrades are the older way to upgrade a data-tier application and were used with the DAC Framework v1.0. There is a newer upgrade process called in-place upgrades and is supported by the DAC Framework v1.1. I decided to use the in-place upgrade since it doesn't create copies of the "pre-upgrade" database. In order to perform an in-place upgrade using PowerShell make sure to use the DacStore.IncrementalUpgrade Method and not the DacStore.Upgrade Method which is now obsolete. Also note that the DacUpgradeOptions passed into the DacStore.IncrementalUpgrade Method control things like whether or not to ignore data loss, ignore drift, skip policy validation and/or perform rollbacks on failure. Make sure to select these options properly for each environment that you're upgrading (i.e. you may not care about drift on your local or CI databases but in your production database database drift should be a cause for concern).

:: Data-tier Application Upgrade Architecture across Environments

The following diagram shows how I used a data-tier application's DACPAC output to upgrade various SQL Server 2008 R2 and SQL Azure database instances across various environments (from local development through to production).

Note that only the changes to the SQL scripts within the data-tier application in Visual Studio are committed (not the DACPAC output that is built locally). The DACPAC is rebuilt on CI using NAnt and MSBuild and is used to immediately upgrade the CI's SQL Azure database instance. The same DACPAC is then archived and used in subsequent deployments for Staging, UAT and Production. The reason behind this build architecture of "build once and deploy to all subsequent environments" is outlined in the book called Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation.

Also note that the "Upgrade" step in both the CI and Staging/UAT/Production environments is performed via the parameterized PowerShell script mentioned in a section above.

Agile Contracts

:: Questions

Lately I've been curious about how one would structure an agile development team within a consulting organization which typically sells fixed-priced projects.

• Can an Agile development team really work effectively when the fixed-priced contract, which governs the deliverables produced, is signed upfront before the project has started and therefore before the requirements have been adequately understood by either the consultant or the client?
• Should a fixed-price contract be avoided entirely or can the incentives somehow be lined up properly such that both the consultant and the client have a large enough stake in producing the best possible outcome together?

:: Resources

I have scoured the web and found the following articles which deal with Agile development and how to structure contracts around it's development methodology:

• InfoQ's Agile Contracts
• Alistar Cockburn's collection of ideas on Agile contracts

:: Conclusions

In an ideal world one can try to structure an agile development methodology around a fixed-priced contract in various ways but most of them only work well when the project is running smoothly. As soon problems start occurring most of the contracts start looking like the wrong approach. Eventually when the rubber hits the road and scope starts screeping, money runs out and/or deadlines loom, something has to give and invariable its the agile methodology because fixed-price contracts are less malleable than the underlying development process. This would be all well and good unless the abandonment of core agile practises during an emergency period of the project did not have adverse effects... but unfortunately time and time again I have seen adverse effects push projects even further into trouble.

-------------------------------

:: Update

I recently came across a 44 page PDF titled Agile Contracts Primer. 

Technical Debt

:: Definitions

Technical debt is a high-level metaphor used to describe the quality of software source code. It was first introduced by Ward Cunningham back in 1992 and has since take shape in many different forms. Today it can be used informally to describe the state of a software application to a non-technical audience or it can be used more formally for measuring technical aspects related to software source code or specific tasks required to "clean" it up. Given that the term "technical debt" can have a wide array of definitions, the following sources provide some deeper insight about the term: 

• Wikipedia Article on Technical Debt
• Technical Debt Blog Post by Steve McConnell (Nov 1, 2007)
• On Technical Debt by Philippe Krutchten, Prof. of Software Engineering at UBC (March 21, 2011)

A software application is only as good as the team that designs, builds and tests it. Therefore, technical debt has many sources and can be introduced by architects, developers or testers. It usually starts out being just a small issue but if left unmanaged, can grow to become very debilitating for a development team to produce software in a timely manner. Technical debt can be associated at any level of the software application stack whether embedded in the underlying architecture, classes, methods, algorithms, documentation and/or build systems. It is therefore important in software projects to not only know what requirements have yet to be implemented fully but also to know what technical debt lurks within the software so that it can be managed effectively.

:: Measurement

The holy grail for measuring technical debt would be to measure the impact of a single software change in terms of both the value it produces (most likely profitability) and the risk it either increases or mitigates. The ideal software change would maximize overall value while reducing overall risk. As an industry, software development is a very long way from this fine grained approach to measuring technical debt but there are many techniques that can be used (either individually or combined together) to measure the potential burden that a piece of software may have and thus provides some better insight. Like tracking finances you never really know where or how much money is being spent until you keep track of it and the same is true for technical debt... you never really know what software components might need attention or how bad they actually are until you measure them directly.

There are a number of metrics that can be used to measure specific aspects of technical debt across software source code. Unfortunately none of these metrics are able to provide a complete picture of the technical debt for a piece of software but taken together they can provide some key insights into its overall quality. Note that care should be taken to fully understand what a metric is in fact measuring and then to not over-state its importance. I have seen many development teams become religious over certain metrics and subsequently implement policies to stay within some arbitrary bounds. This can often lead to development practises that are detrimental to the actual software being developed. Measuring technical debt is more of an art than a science but quantitative values can still be helpful in the arts.

Here are some of the metrics I have used to measure source code:

• SLOC (Source Lines of Code): The size of a software application in SLOC isn't a great metric in an of itself, but when used as a comparison mechanism (e.g. the ratio of production code to testing code) it becomes a good indication of relative code quality.
• Cyclomatic Complexity: A great indication of how complicated the code paths are in a portion of code (usually a method). For methods, it is also a very good indication of how many unit tests are required to effectively traverse and thus test each code path. This is also one of the few metrics statistically correlated with buggy software.
• Coupling: A number of metrics are defined in this category but of particular interest are fan-in and fan-out metrics which measure the number of calling or called routines for a portion of code.
• Branch Coverage (Also known as code coverage): Measures the amount of production code that is covered by unit tests. If you think you have tested a method or class sufficiently this is a good metric to observe to verify that thought.
• Duplicate code: Duplicate code is a particularly interesting metric because the code that is duplicated can in fact be of a high quality. The problems arises when the duplicated code needs to be modified due to a bug. As a result the developer must make sure that each and every occurrence of the bug is fixed otherwise one occurrence of the bug will be fixed but one or possibly many others will remain. A false sense of security that a bug is fixed when it is not can be a disaster in certain contexts.
• Nested depth level: Some algorithms require nested 'if', 'for' or 'while' statements and an large number of nested statements (also called the depth level) increases the complexity of a portion of code and makes it harder for any one developer to understand it without considerable effort (leading to possibly buggier code).
• Violations of coding standards: If a coding standard is defined there a usually a number of tools available that either enforce compliance with the standards or simply count the number of infractions.

Apart from simply using metrics to quantify the technical debt of a software application, I have also found the need to capture various software "cleanup" tasks. Anything from dead code to code refactoring to build system improvements can be added to a ticket in your bug tracking system, labelled as "technical debt" and estimated accordingly. By doing this the technical debt within an application can be is tracked over time and observed as it either increases or decreases. It can therefore be managed in a pragmatic way instead of avoided or simply kept unknown.

:: Tooling

There are a whole range of open-source and commercial static analysis software tools available to capture metrics and enforce their compliance:

• Java Specific:
• Google CodePro Analytix
• Checkstyle
• PMD
• FindBugs
• Crap4j (C.R.A.P stands for Change Risk Anti-Patterns)
• .NET Specific:
• NCover
• FxCop (Comes integrated with Visual Studio 2010 Premium and above or can be run as an external application for lesser versions)
• StyleCop
• Code Contracts (Includes a static analyzer for pre and post conditions within methods)
• MSDN also has an article on ways to calculate code metrics for .NET projects in Visual Studio.
• Multi-language:
• SLOCCount (Linux/Mac)
• CLOC (Windows)
• McCabe IQ

And here is an even bigger list of static analysis tools.

Continuous Integration: Tooling, Processes & Implicit Communication

Continuous integration (CI) is a process and set of tools that development teams can use to continually manage the software they develop, test and deploy. In this post I will highlight the various tooling I have used on various .NET projects that harnesses CI, I will discuss some key processes around using CI that make it most effective and I will present some of the more subtle ways CI allows for implicit communication between team members that isn't so obvious when you're starting out with the practice.

Hudson is a really good open-source, Java, web-based application that can be used as a CI build system on Windows or Unix/Linux. What I find particularly compelling about Hudson is how easy it is to install and get started with a new project. First download the Hudson WAR from the homepage. Once you have started it from the command-line using java -jar hudson.war almost everything else is done from its GUI based menus; there is no need to configure any XML or to have an intimate knowledge of its API (CruiseControl is like this).

:: Tooling

Redsolo has a pretty complete post on setting up Hudson for a .NET project and getting the unit-tests and some static analysis (FxCop) to run. I initially configure my own Hudson instances very similarly to his post but over time I have altered it in the following ways:

• The use of a source control management (SCM) system is crucial when building a CI infrastructure. I use Subversion (SVN) but you could just as easily use Git if you're more the distributed version control type. It is critical for CI to have a central place where all developers can regularly commit their code and configuration changes for a software project. Applications may work in isolation (when worked on independently) but time and time again I have found that the integration of a team's code changes and configuration can be very very challenging. CI promotes integrating early and often and therefore tries, and I believe succeeds, at avoiding the headaches that are associated with integrating at the end of a project (which by the way is very difficult to estimate against as you have no idea what kinds of bugs are lurking when it comes time to fully integrate).
• I use NAnt to easily specify and execute a range of tasks from project compilation, database management, unit and acceptance test execution, static analysis, documentation generation and deployment. Hudson has a NAnt plugin which allows it to use NAnt as a build step. Thus, when Hudson notices that a developer has checked-in code, it does a full checkout of the subversion tree and executes a specific NAnt target to begin compilation or any other task required for that particular project.
• I generally use MSTest and not NUnit for unit-testing. Both integrate nicely with Hudson but Hudson only knows how to interpret JUnit and NUnit test results. Because I still want a graph of the test results to show up on the project's main page, I install the MSTest plugin that converts the test report (a TRX file) into a format that Hudson understands natively (which happens to be JUnit XML).
• I use FxCop, StyleCop and NCover for static analysis of my projects. FxCop and StyleCop are used more for helping the development team adhere to some design and implementation best practises as well as coding standings throughout development whereas NCover is used more for "after the fact" analysis of the code base (e.g. it highlights potential production code our team has not tested adequately). I use the Violations plugin to visually capture the number of FxCop and StyleCop violations within the code base after each new build. I also use a combination of a NAnt target (to generate the NCover report) and the HTML publisher plugin to publish the NCover report to Hudson so it is accessible via a link on the main page.
• I automatically generate Doxygen documentation from the XML comments associated with the classes and methods within my code base. I use a NAnt target along with the HTML publisher plugin to publish the HTML to Hudson so it is accessible via a link on the main page. This way the development team always has a reference to the most up-to-date API documentation for the code base.
• I find the Task Scanner plugin particularly helpful to keep track of various keywords in the code base that developers may leave in to remind themselves or others of what to fix or attend to later. Any custom tags such as FIXME, TODO, @deprecated, ??? can all be tracked and are shown visually on the main project page.
• I use the Claim plugin which allows a specific developer to claim a build break and comment on what the problem/solution might be. The "claim" mechanism is sticky which means that if a developer continually breaks the build they will be continually responsible for fixing it. This plugin is great for helping a distributed team communicate implicitly about who is responsible for build breaks (instead of email or instant message communication between the whole team - less communication spam is always a good thing).
• I also archive specific build artifacts from each CI build that I may want to analyze in the future. That way if I ever want to go back in time and discover where a particular bug entered the code base (and by whom) I can do so. It is much easier to simply download the executables from the CI build server and run them than to checkout a specific revision from source control, compile it and then run it to verify it does or does not have the bug I'm looking for.

Hudson's website also has some best practices to follow for Build Management, Release Management, Deployment Automation, and Test Orchestration

:: Key Processes

Here are some key processes that I use on teams to harness the benefits of CI:

• Only one Commit per Build - Finding and solving software bugs that cause a build to break can be like searching for a needle in a haystack. Finding that needle can be made much easier if your software uses logging (such as log4net), a team member has deep insight into certain parts of the application that they know cause certain problems, specific sets of code and configuration changes can be isolated and labelled as the changes that may have caused the bug, or your software application has a testing suite that can pin-point failure areas. Each of these can be harnessed more effectively and precisely if a CI build is executed after each and every developer's commit (i.e. a single CI build does not encompass multiple commits). The reason for this is, if a build failure does occur, it is far easier to analyze a single code change set from one developer and begin the process of finding the bug than to begin analyzing multiple change sets from multiple developers. Whomever's code change actually broke the build at this point is very hard to determine without all developer's working together instead of just the single developer who's code change broke the build. Caveat: The only additional note on this is that it becomes difficult to keep to this requirement as a team grows in size and the build takes longer and longer to execute. For example: If a single build takes 1hr to execute, in a given working day a development team will only be able to run 8-10 CI builds. The more developers that are on a team, the more commits your team will perform each day. At some point your team won't be able to keep to this "one commit per build" policy but it is definitely something to strive for if you can.
• Start a Build from Scratch - When setting up your CI build make sure to re-checkout the FULL source controlled tree (do not update the local copy of the tree) and fully rebuild (compile) all executables and libraries from their raw source code. Simply doing an update of your local copy of  the tree or rebuilding only the executable/libraries that are needed, allows for the possibility of artifacts, from the previous build, to be used in the current build. The most effective way to eliminate unintended or corrupted build breaks and, more importantly, to truly test how everything integrates from your raw source code, the CI build should start from scratch. Delete everything, start from a freshly created directory and then checkout your source code from source control. Compilation, testing and deployments can then be performed on the raw source code and not any other artifacts still lying around in the directory from a previous build.
• Multiple Builds - On my projects I usually need multiple builds. I need one build, called CI Build, that automatically builds after a single source control commit by a developer. I then have another build called Nightly Build, which is run each and every night and therefore builds against all the source control commits from the previous day. Then I may have a QA Build that has a different deployment procedure to some QA server for the testers. The QA Build is usually setup to build after a manual step that one of the testers will execute. Lastly, I have a Production Build that has a deployment procedure specifically geared towards deploying the application into a production environment and doing all the appropriate configuration and tests required to validate that it is stable and functional in production. Of course the Production Build is very important to get right and should be protected against unauthorized execution (i.e. password protect its usage or have only a select few individuals with the permissions to execute it).
• Build Both the Release and Debug Configurations - When compiling .NET projects in Visual Studio you have the ability to compile in Release or Debug (or others if you have defined your own). I would highly recommended having both a CI Release Build and a CI Debug Build. I have found very subtle differences between the two when it comes to compilation, timing for unit-tests, configuration & static analysis results. I've had situations where the debug build passes but the release build does not and it was usually the result of some mis-configuration but occasionally I catch a real nasty timing issue in a unit or acceptance test. Note that I care about both build configurations because developer's routinely develop and test an application which was compiled in debug mode whereas the software actually being released to production is compiled in release mode.
• Keep the Main Source Control Tree Pristine - Each and every developer works from and modifies the main source control tree. Each modification to the tree affects all other developers so it is imperative that it be kept clean and at the highest quality. The software that a development team produces is what is in the tree, therefore whatever quality the tree has the software has also. This is a slightly more concrete way of thinking about the intangible aspects of software. It is hard to visualize what software really is and what quality it has but it is much easier to visualize the files checked-in to a source control tree and the quality of each of those files. Therefore, keep the files that are checked-in to the source control tree pristine at at the highest quality.
• Priority Number 1: Fix the Broken Build - When a build breaks during CI it effects the whole team. Developer's should be immediately informed that the source control tree is unstable and they then know NOT to commit any changes or to update their local code versions to the broken/unstable one. Developer's can of course continue to develop their feature(s) or resolve their bug(s) but eventually they're going to want to commit code changes or update their local versions with everyone else's  code changes. With a broken build, developer's are forced to work in isolation which is the very thing CI is trying to avoid. A broken build delays integration and should therefore be avoided at all costs.

:: Implicit Communication

Some projects can be routinely bogged down by communication overhead between team members which makes it difficult to be productive (less communication overhead means more time building and improving software). Question such as: why does this code not compile, who's responsible for this piece of code, what effect will this code change have, what exactly are we testing, what are the exact files I need in order to test this on another machine, are all valid questions developers can ask but if there is an easier way to communicate the answers other than direct developer-to-developer interaction, productivity can be greatly improved. This is particularly useful for development teams that are distributed by both location and time but even small teams located in the same office can see marked improvements by reducing communication overhead by using CI practices.

Having a single, repeatable process which is what CI fosters, allows entire development teams to be on the same page. At any point in time you can verify the state of the code base: is it stable, is it broken or is it in transition (i.e. currently building, testing and deploying). If things are stable then I can concentrate on developing features for my software application. If it's broken then I know something is wrong and the CI build system will most likely help me out by informing me why it's broken or at least try to point me in the right direction (i.e. what commit by what developer broke the build, did a test fail, did deployment fail, is it a configuration issue or a code change... etc). If it's in transition I know that it's potentially unstable and should wait until I know more. Following these basic principles, development teams keep in-sync by knowing at any point in time whether the life blood of their software application (it's source code and configuration) are stable or whether there is a problem that needs fixing.

Extending this CI process with the tools mentioned above: static analysis reports, the ability for specific developers to claim broken builds and comment on them, auto-generated API documentation and code base task scanners, a development team has relevant information at its finger tips that helps them communicate without direct face-to-face or email communication. I call this implicit communication. If, as a developer, I can quickly determine who broke the build, if they have taken responsibility for it and what they have decided to do to fix it, I can quickly go back to the feature I was previously building or I can get in touch with the person responsible for breaking the build and help them by providing a deeper insight about the reason for the failure. Apart from build breaks, I can access the latest API documentation and verify that I understand a method's contract that I am about to use. I can also quickly verify the cyclomatic complexity of a method that I am refactoring and decide whether it has enough test coverage. All of this can be done from my own desk with no direct communication with another developer. I have found that on some projects without this CI infrastructure there is always a "guru" developer who seems to know the most about a project (usually because he's been there the longest) and everyone needs their insight about each and every software related issue. You can imagine what kind of bottle neck that causes and what might happen if the "guru" leaves the project... chaos.

At the end of the day this implicit communication helps me build better software as I have the information I need to design and implement software features and fix bugs. By reducing the communication overhead that CI tools and processes allow, I am more productive at developing, improving and delivering the software customers pay for.

-------------------------------

:: UPDATE

Recently the team that was building Hudson left the project and subsequently forked it into what is now called Jenkins (This was due to some political issues with Oracle over the name Hudson and who had control over it). If you already have a Hudson instance and wish to switch to Jenkins (as the team is mostly over their now) this Jenkins wiki article will help you make the switch.