In-Depth

Microsoft Math

As Longhorn looms, we examine Microsoft's track record in hitting product ship dates and offer advice for coping with the inevitable.

• By Paul Desmond
• 03/01/2005

A slip in the ship date for Microsoft's Systems Management Server (SMS) 2003 rendered Chris Ferrari unable to hold up his end of a company-wide upgrade of some 55,000 desktops. A delay in the rollout of Windows XP Service Pack 2 (SP2) put Laura Hunter a year behind in rolling out SP2 to students at the University of Pennsylvania. And a change in the ship date for Microsoft CRM means Wesley Bielinski's Exchange environment doesn't have the redundancy he prefers.

Three dedicated Microsoft customers with three stories following a similar theme: They got burned by Microsoft missing an announced product ship date.

It shouldn't be news to anyone who has dealt with or followed Microsoft for any length of time that the company is notorious for missing ship dates. But with another major operating system upgrade on the horizon in Longhorn, it's a good time to take a hard look at the company's track record to try to glean tendencies that can help customers plan accordingly.

To do that, Redmond magazine researched announcement dates and delays for Microsoft desktop and server operating systems, along with Microsoft Office and Exchange products going back as far as 1983. We took the resulting data and handed it off for analysis to Barry Bayus, a professor of marketing at the Kenan-Flagler Business School at the University of North Carolina. Bayus co-authored a 1999 report titled, "Truth or Consequences: An Analysis of Vaporware and New Product Announcements," and thus has expertise in missed product ship dates.

The result picturing isn't a pretty one. After tallying up all the numbers, we found that, on average, Microsoft ships its desktop OSes 10 months late while its server OSes are just over a year late. The track record for major, mold-breaking OSes, however, is much worse. Windows 95, for example, was 14 months late, while NT Server 4.0 was 21 months late—nearly two years.

It's the mold-breaking products you should likely keep in mind when trying to assess when Longhorn might ship, because it's Microsoft's most ambitious operating system in years—or at least, it was before the company began stripping out key components like WinFS, a step the company took solely so it could hit a reasonable ship date.

UNC's Bayus says that across the Microsoft products we examined, the average time between a product announcement and ship date is a little less than two years. That means some products take far longer. "Although on the long side of the historical distribution, Microsoft's promise of Longhorn [client] in 2006 is within the historical range of other Microsoft products," he says (see "Microsoft Time"). "This would represent an elapsed time between announcement and ship of three to four years."

(Click image to view larger version.)

However, if the Longhorn client is more consistent with Windows NT 4.0, he notes, then it may be delayed until 2007. And, if it's more consistent with Windows NT 5.0 (Cairo), which was abandoned and effectively replaced by Windows 2000, then it may be delayed until 2009.

Here Come the Caveats
We can't deliver these figures, however, without a heavy dose of caveats. For starters, it's rarely easy to nail down exactly when Microsoft announced a given product. Often, company executives simply start talking about a product when it's little more than a development effort. After enough mentions at trade shows and to members of the press, the news eventually takes on a life of its own. Cairo, a.k.a. Windows NT Server 5.0, and Longhorn are both good examples. In neither case was there ever a grand event, with Bill Gates, Steve Ballmer or any other executive making a formal announcement. Instead, various executives simply started talking about the initiatives, laying out grand concepts and key features that each would include—features that tend to morph greatly over time.

In that environment, it's likewise difficult to determine the target ship date for a given product. Here, again, the Microsoft routine is to give vague references to seasons, or even half-years. Rarely will you hear Microsoft say, "We'll ship this product by June 30. Bank on it." A far more likely scenario is something like, "We expect we'll be able to ship by the second half of 2004." If said product doesn't ship until 2005, is it a missed ship date or just a lousy guess?

So, to compile the data used in our research, we pored over hundreds of old trade press stories written by well-respected colleagues. In some instances, we were able to derive a definitive date that a product was announced, complete with expected delivery date. Further research made it relatively easy to determine whether Microsoft hit the delivery date, or how much it missed by.

In other instances, we made judgment calls. When a critical mass of stories began to appear for an effort like Cairo, we declared the product "announced." We then pieced together a target ship date from the various published stories, often relying on "industry sources" as opposed to formal company announcements.

In compiling our data, we used the worst-case scenario for the ship date, meaning the date that was furthest out. Had we not given Microsoft every benefit of the doubt, the results published here would look worse for the company.

Accounting for Cairo
Then there's the special case of Cairo, which never shipped at all. (Yes, bits and pieces of it shipped in other OSes, but it was, effectively, killed.) Accounting for Cairo in our equations would've hopelessly skewed the results, if indeed we could even figure out a way to include a value that amounts to infinity. It's somewhat akin to a pitcher in baseball who gives up six runs in one inning while getting nobody out—his earned run average skyrockets. Rather than take that route, we just skipped Cairo entirely, once more tilting the results in Redmond's favor.

Microsoft is unapologetic about its track record. "As we've always said, the software development process is fluid, and our focus is on the quality of the overall customer experience and providing customers with insights to the goals of the product release, rather than meeting a specific date," says Samm DiStasio, product manager for the Windows Server group, in an e-mail response to Redmond queries. While Microsoft does set internal target dates for development projects, "they are only target dates and product quality always takes precedence over timing."

Many Microsoft customers are all too familiar with this ship date roulette. "I never make plans based on a Microsoft announced ship date," says Randy Davis, IT systems administrator with Lubbers Auto Group in Cheney, Kan. "It's uncommon for most businesses to upgrade to a new version of almost anything from Microsoft until the first service pack has been released."

That's a tried-and-true strategy. But in too many instances, companies simply don't have the luxury to wait and must plan toward Microsoft ship dates.

(Click image to view larger version.)

Waiting for SMS
Ferrari was one such case. A systems engineer for Aventis Pharmaceuticals in Bridgewater, N.J., in early 2003 he was involved in a company-wide, global upgrade of 55,000 desktops to Win2K along with Windows Server 2003 servers. A platform services group was in charge of creating the new desktop image while Ferrari belonged to the systems management group that was responsible for coming up with a way to deliver software, patches and other updates on an ongoing basis.

The plan was to use SMS for that role. Since late 2002, Ferrari's team had been working with SMS 2003 as contributors to Microsoft's Joint Development Program (JDP) and Early Adopter Program (EAP).

SMS 2003 was scheduled to ship in April 2003, he says, but in February he was informed it wouldn't ship until September. It actually shipped on Oct. 3, 2003, he says.

"They missed what I understood to be the deadline for RTM [release to manufacture] by six months," Ferrari says. "That may not sound horrible, but it's half a year. If you're part of an EAP program and you're already involved with deployments and laying out project milestones on a global level, it really throws a wrench into the works."

Part of the reason Ferrari was involved in the EAP program was to be privy to early versions of the software, which he used to do comparisons with other, similar systems management products and to make sure he fully understood all the scheduled delivery dates. Using that information, he was able to schedule dates for testing, deployment and other milestones.

"All those dates might not have been critical on their own, but they tied in with other projects," he explains. "Those [platform upgrades] were moving forward fast. We had our reputations at stake."

His team moved forward as best it could, building pre-RTM infrastructure and pilots. "But you can't roll out any deployment of stature with a beta product," Ferrari says. In the end, another group picked up the ball and developed its own tool for delivering software to the upgraded desktops.

"We should've been with them, rolling with their deployments, but we couldn't," he says. "I'm not blaming Microsoft, nor am I blaming [the platform services group] for moving ahead and not waiting for SMS. But things could've been different."

(Click image to view larger version.)

An SP2 Ripple Effect
Laura Hunter knows Ferrari's pain. At UPenn, where she's a senior IT specialist, timelines are built around the academic year and preparing for the onslaught of new students.

"Every fall, we get a massive truckload of stuff from Dell. They'll pre-install an image for us because it's a bulk purchase—the operating system and all university applications," Hunter says. "One thing we couldn't do [in 2004] was pre-load Windows XP SP2. It showed up too late for us to get it in, test it and make sure it works with all of our apps."

The SP2 ship date was a moving target, she remembers. "First I heard May [2004], then May became June and June became August," she says, noting it's tough to know what to believe, between Microsoft and the rumor mill. "There's what Microsoft says and there's what Slashdot says."

Like Ferrari, she's not knocking Microsoft or SP2—far from it. "I love SP2, but it did break some stuff," she says. While she can't roll it out campus-wide until this coming fall, she says administrators in individual departments are installing it. "And some students have upgraded, some haven't," she says. "Of those who have, some were successful, some not."

"I don't expect Microsoft's timelines to necessarily jibe with ours, but [the delay] did set us back a year, literally."

Waiting for CRM
Like Hunter, Bielinski is also stuck in wait mode as a result of a missed Microsoft ship date. The network administrator for the American Board of Medical Specialties in Evanston, Ill., Bielinski is also learning that the integration among Microsoft products can be a double-edged sword.

A big believer in redundancy, Bielinski likes to run his Exchange servers in a clustered environment. His problems began when his business development group decided the organization needed a customer relationship management system. A consulting firm recommended Microsoft's CRM product, but there was a problem: The e-mail routing tool included in v1.2 of Microsoft CRM wouldn't work with the clustered Exchange 2000 environment Bielinski employed.

Microsoft CRM v2 would solve the problem, he says. It was due out last August, but is now being integrated with the Office Solution Accelerator program and being renamed CRM 2005; it's scheduled for delivery in the second quarter.

Rather than create a new, standalone Exchange 2000 box, the company decided to fast-track its planned upgrade to Exchange 2003. Bielinski attended a course on Exchange 2003 in early July of last year and began the migration at the end of the month. "We now have a single server running Exchange 2003, installed the CRM (v1.2) and finally got it working." He still has the two servers that previously ran Exchange 2000 and will eventually press them into service as Exchange 2003 clusters, but not until he gets the new version of the CRM program. "I'm left sitting here twiddling my thumbs till it comes out, then I can find out whether it's actually going to work," he says. In the meantime, he does have another Exchange server configured as a backup, but it's not in clustered mode. In case of a failure, he'd have to restore from a backup, and would likely lose some e-mail.

"Integration is nice when it works," he says, noting he's frustrated that clustered servers aren't supported in Microsoft's own CRM product. "It's hard to believe, unless Microsoft doesn't use its own product."

Lessons Learned
As these experiences show, even with proper precautions it's difficult at best to plan around Microsoft product ship dates.

Many corporate buyers simply don't plan on Microsoft delivering any product on time, says Stewart Alsop, a venture capitalist who in 1985 launched the PC Letter newsletter, which included the Official Vapor List, a monthly tally of software products that were late to ship (see "Historical Perspective"). "If Microsoft says they are going to deliver something—whether it is Windows Server, Exchange or any big piece of software where they have a lot of users and revenues—whatever they say is going to happen, is not [going to happen]," Alsop says. "So the problem is to figure out how far off they will be." Most customers simply give a discount to whatever date Microsoft gives, he says, and many wait for months after the product is delivered, for the inevitable bugs to be worked out.

Hunter fits the Alsop mold. "Once I see something in release candidate 1 or 2 and it's fairly stable in that format, I can say to myself, ‘I'll see this released to market in the next six months,'" she says. "But I'd like to have a better inkling than that."

Historical Perspective A conversation with Stewart Aslop, author of the original "Vapor List."

In 1985, Stewart Alsop launched a newsletter called PC Letter, which included a popular regular feature called the Official Vapor List. Each month, Alsop would publish the name of any vendor that had not yet delivered a promised product. Alsop also served as editor in chief of the IT newsweekly InfoWorld before moving on to become a columnist for Fortune magazine and a venture capitalist. He is now general partner with New Enterprise Associates (NEA) in Menlo Park, Calif. Ed Scannell, an editor-at-large with InfoWorld, tracked down his old boss to get his thoughts on Microsoft's track record. Scannell: Is there a formula you could suggest that people could use to translate what Microsoft says into even a general time frame for product delivery? Alsop: The first time Microsoft talks about a product, it is wildly, overly enthusiastic. So the best thing to do is just not believe anything the company says; give a 100 percent discount. Then Microsoft will be embarrassed and go into the mode of trying not to say anything. But then the second date it gives you, that's the one it will probably miss by six to 12 months. And when it misses by six to 12 months, it will try not to give another date, but the third date the company gives is the one where you start to think about it actually happening. So the third date is the charm but the software typically won't work. Part of the problem here is that after 1995, Microsoft started talking about future versions of Windows that not only never came out on time, but it would wrap in pieces of the promised new version into point releases of Windows that the company never originally talked about. So it's hard to determine exactly when a complete technology for a new version is delivered. A good example is Cairo, where all the pieces are still not delivered eight to 10 years later. It tends to deliver its vision in pieces now. Yes, true, plus the company is constantly changing the pieces. A key part of both Cairo and Longhorn is the idea of a unified database underneath the file system. It's Bill Gates' fundamental vision of unifying the file system. It was supposed to be the same thing in Exchange and in Windows and everything else, this single unified database. But Microsoft has already changed it out as a component of Exchange. And it was going to be a key component in Longhorn, but Microsoft is no longer promising that. That was the basis for doing Longhorn in the first place, so I don't know how the company justifies doing Longhorn now. Microsoft seems to have the most trouble delivering products that make a huge jump in the technology: Windows 1.0, Windows 3.0, Cairo, Windows 95 and now Longhorn and Yukon. It could set a record with Longhorn of about seven years between when Jim Allchin first laid out the vision and when the company delivers the complete bag of code. Yes, it's vaporware, but it's a funny thing. I have a hard time tagging Microsoft on this whole issue, for this reason: We all want the software to be better. We all really want it to be better. You don't want to beat Microsoft up too much for being late, as long as it's doing the right thing. But the company doesn't seem to solve the problem. Windows was [supposed to be] this beautiful operating system that was promised back when it was competing with the Macintosh and it was highly reliable and open blah, blah, blah. Well, Microsoft still really hasn't delivered that yet. So you could say it has been 20 years late. — Ed Scannell

She also takes advantage of her Microsoft Most Valuable Professional (MVP) lead, who she says tends to provide quick answers, as well as a Microsoft higher education sales rep who serves the Greater Pennsylvania area. In the end, though, she understands that delays happen and is more concerned that Microsoft ships a quality product than hit its own RTM date. "The biggest thing is, and Microsoft has been really good at this, if you're not going to make your announced date, let me know as soon as you know," she explains. "Don't let me know the day before or the week before."

Ferrari is still a believer in participating in Microsoft JDP and EAP efforts, saying they keep customers involved in the process, giving them an inside track. "As consumers and project managers, we should remember that the manufacturer has a responsibility to deliver a solid and sound product as a first responsibility," he says. "Sometimes we forget about the positives."

Bielinski is less forgiving, and notes that if he had to do it over again, he may look at vendors other than Microsoft to supply CRM and other such components. "If they keep pushing dates out, they're going to lose customers," he says.

Microsoft may also be wise to stop announcing products so far in advance, he says. "Once you're in alpha or beta testing, then announce you're coming out with a product in the next six months or a year," Bielinski says. "A year would be good—most people plan a year in advance for budgeting purposes."

Microsoft's DiStasio defends the long lead times between product announcements and ship dates. "Maintaining open lines of communication with customers about their needs is the key to a successful development process. We strive to give customers as much information as possible to help in planning their deployments," he says.

Long Wait for Longhorn
OK, but do customers really need four years or more to plan? That's what they're getting with Longhorn—at least. Microsoft has been talking up Longhorn since at least March 2002. The client is now scheduled for delivery in 2006, albeit without major features such as WinFS, which will ship later as an add-on. A server version is scheduled for 2007. As UNC's Bayus notes, if history is any indicator, both versions may well suffer significant delays.

Users seem all too aware that Longhorn is a moving target and, in general, aren't exactly holding their breath for it.

"We are anticipating Longhorn and are rather disappointed that they're stripping components and delaying on top of that," Bielinski says. "I'm not losing any sleep over it, though."

Hunter is even less concerned. "As far as my 9-to-5 production network is concerned, I really couldn't care less about Longhorn right now," she says. "In terms of business priorities, a product that may or may not be ready to ship in 15 months might as well still be 100 years away."

Having just rolled out a new client and server infrastructure, Ferrari is likewise in no hurry for Longhorn. "Most companies are not ready for many Microsoft products as they are released to manufacturing," he says, noting that Redmond moves "pretty quick for such a large company."

Bielinski agrees that major OS upgrades take time for users to digest, but also says Microsoft would do well to build more cushion into its product delivery schedules. "Worst-case scenario, they release the product early," he says. "That'd be nice."