Microsoft’s branch integration process
In September 2005, the Wall Street Journal reported how Microsoft changed its build process after it rebaselined Windows Longhorn.
The Windows division apparently had little or no integration criteria, to block unstable or poorly-integrated code from merge to the integration branch.
Mr. Allchin's reforms address a problem dating to Microsoft's beginnings. Old-school computer science called for methodical coding practices to ensure that the large computers used by banks, governments and scientists wouldn't break. But as personal computers took off in the 1980s, companies like Microsoft didn't have time for that. PC users wanted cool and useful features quickly. They tolerated — or didn't notice — the bugs riddling the software. Problems could always be patched over. With each patch and enhancement, it became harder to strap new features onto the software since new code could affect everything else in unpredictable ways.
The unstable codebase created by uncontrolled changes was fragile, and prone to unanticipated errors from subtle interactions. Without automated testing, dedicated developers (firefighters, probably not those that wrote the errors) spent all their time correcting builds.
Mr. Allchin says he soon saw his fears realized. In making large software programs engineers regularly bring together all the new unfinished features into a single "build," a sort of prototype used to test how the features work together. Ideally, engineers make a fresh build every night, fix any bugs and go back to refining their features the next day. But with 4,000 engineers writing code each day, testing the build became a Sisyphean task. When a bug popped up, trouble-shooters would often have to manually search through thousands of lines of code to find the problem.
This approach, and the architecture of Windows, differed from competitors', and those of other parts of Microsoft.
That was just the opposite of how Microsoft's new rivals worked. Google and others developed test versions of software and shipped them over the Internet. The best of the programs from rivals were like Lego blocks — they had a single function and were designed to be connected onto a larger whole. Google and even Microsoft's own MSN online unit could quickly respond to changes in the way people used their PCs and the Web by adding incremental improvements.
In response, the Windows division automated testing, improved and automated integration criteria, and focussed developers' attention on correcting errors.
By late October, Mr. Srivastava's team was beginning to automate the testing that had historically been done by hand. If a feature had too many bugs, software "gates" rejected it from being used in Longhorn. If engineers had too many outstanding bugs they were tossed in "bug jail" and banned from writing new code. The goal, he says, was to get engineers to "do it right the first time."
All to good effect: fewer defects and shorter cycle time.
On July 27, Microsoft shipped the beta of Longhorn — now named Windows Vista — to 500,000 customers for testing. Experience had told the Windows team to expect tens of thousands of reported problems from customers. Instead, there were a couple thousand problem reports, says Mr. Rana, the team member.
And last month, Microsoft delivered a test version of Mr. Gates's WinFS idea — not as a part of Longhorn but as a planned add-on feature. Microsoft this month said it would issue monthly test versions of Windows Vista, a first for the company and a sign of the group's improved agility.
The last word is significant, since automated testing is important to Agile Methods. The other major improvements are either decent SCM (eg, a merge plan) or management (if you write bugs, don't write new code).