Ian Murdock’s Weblog

More Knowledge@Wharton: “Following the model that made Microsoft’s Windows so successful, Microsoft licensed PlaysForSure to multiple hardware vendors of digital music players. ‘We thought that was a brilliant strategy — [develop] an open ecosystem, get a lot of people [to support it].’ What happened? As Ballmer puts it, ‘In this particular case, the whole was not bigger than the sum of the parts.’”

As I said the other day, an ecosystem is never the sum of its parts—it’s either a whole lot more or a whole lot less. Microsoft was decidedly on the wrong side of the equation this time. Yes, Microsoft did what it had to do to compete effectively. Question is, will this make it harder for Microsoft to build the next ecosystem? And did the tactics Microsoft used to benefit from the last one play a role in the outcome this time?

(Via Mary Jo Foley.)

There were a lot of good comments on my post about the importance of backward compatibility the other day (both here and in the blogosphere), and a lot more of them were positive than I was expecting, which I find encouraging.

A fair number of people called me out for using such a bad example—come on, changing the OS to fix a buggy application? Fair enough. Perhaps I diminished the point I was trying to make by referencing that extreme example, but it’s a worthwhile example for one reason: At Microsoft, the user experience comes first, notbefore developer sensibilities.

Fortunately, as others (rightly) point out, it’s hard to imagine a situation where such extremes would be necessary in the Linux world—for one thing, we don’t have the sheer number of legacy binary applications to deal with, nor do we have the same volume or average user profile as Windows. But the point stands. User experience should always win.

If you want another example, one from a company who any developer would agree is an outstanding engineering organization, here’s one: “Sun has maintained binary compatibility between operating system releases for nearly a decade, enabling existing Solaris applications to run unmodified on Solaris 10. This means that Solaris applications developed ten years ago will run on Solaris 10 unchanged, taking full advantage of new and advanced Solaris features.”

The New York Times: Group Formed to Support Linux as Rival to Windows

Me: “The hard part of building a standard is getting the right people at the table — those who are in a position to make that standard actionable. You can sit around all day and agree that there should be a standard, you can even agree what the standard will look like, but if you don’t have buy-in and participation from the people who can actually implement it and get the standard to be widely used, then you’re wasting your time.”

Robert Scoble: “Wait a second. You mean there’s a billion phones with Java on them and Cingular’s network hasn’t gone down yet? Damn, how did that happen?”

Matt Asay: “Open source is all about free markets.”

Gordon Haff: “Imagine, if you will, that it’s the late Nineties. A certain software company based in Redmond, Washington has recently released Visual Studio 97—thereby bundling together many of its development tools for the first time. Now imagine that the company decided to release those tools for free. What do you think the general reaction would have been? Applause for Microsoft’s generosity? Or widespread condemnation for using its market power to make such a transparently anti-competitive attack on other makers of development tools?”

Thought provoking observations. Note, however, that “open source as competitive weapon” isn’t limited to large vendors—it works equally well (and, in many ways, better) for the upstarts (see Red Hat, MySQL, JBoss, etc.). In fact, it works so well for the upstarts that even the upstarts have upstarts (see, e.g., Canonical). Can something that levels the playing field so dramatically be called predatory? Predatory has one-sided, unfair advantage connotations. In open source, it goes both ways. With Eclipse, it was IBM doing the disrupting; but with MySQL and JBoss, IBM is on the other end of it. At the end of the day, the real winner is all of us—after all, who can argue that the state of IDEs and middleware isn’t better today than it’s ever been?

I’m often asked why I’m so obsessed with backward compatibility and, as a result, why I’ve made the issue such a central part of the LSB over the past year. Yes, it’s hard, particularly in the Linux world, because there are thousands of developers building the components that make up the platform, and it just takes one to break compatibility and make our lives difficult. Even worse, the idea of keeping extraneous stuff around for the long term “just” for the sake of compatibility is anathema to most engineers. Elegance of design is a much higher calling than the pedestrian task of making sure things don’t break.

Why is backward compatibility important? Here’s a great example, via Joel Spolsky (note: from 2004):

Raymond Chen is a developer on the Windows team at Microsoft. He’s been there since 1992, and his weblog The Old New Thing is chock-full of detailed technical stories about why certain things are the way they are in Windows, even silly things, which turn out to have very good reasons.

The most impressive things to read on Raymond’s weblog are the stories of the incredible efforts the Windows team has made over the years to support backwards compatibility: “Look at the scenario from the customer’s standpoint. You bought programs X, Y and Z. You then upgraded to Windows XP. Your computer now crashes randomly, and program Z doesn’t work at all. You’re going to tell your friends, ‘Don’t upgrade to Windows XP. It crashes randomly, and it’s not compatible with program Z.’ Are you going to debug your system to determine that program X is causing the crashes, and that program Z doesn’t work because it is using undocumented window messages? Of course not. You’re going to return the Windows XP box for a refund. (You bought programs X, Y, and Z some months ago. The 30-day return policy no longer applies to them. The only thing you can return is Windows XP.)”

I first heard about this from one of the developers of the hit game SimCity, who told me that there was a critical bug in his application: it used memory right after freeing it, a major no-no that happened to work OK on DOS but would not work under Windows where memory that is freed is likely to be snatched up by another running application right away. The testers on the Windows team were going through various popular applications, testing them to make sure they worked OK, but SimCity kept crashing. They reported this to the Windows developers, who disassembled SimCity, stepped through it in a debugger, found the bug, and added special code that checked if SimCity was running, and if it did, ran the memory allocator in a special mode in which you could still use memory after freeing it.

This was not an unusual case. The Windows testing team is huge and one of their most important responsibilities is guaranteeing that everyone can safely upgrade their operating system, no matter what applications they have installed, and those applications will continue to run, even if those applications do bad things or use undocumented functions or rely on buggy behavior that happens to be buggy in Windows n but is no longer buggy in Windows n+1…

A lot of developers and engineers don’t agree with this way of working. If the application did something bad, or relied on some undocumented behavior, they think, it should just break when the OS gets upgraded. The developers of the Macintosh OS at Apple have always been in this camp. It’s why so few applications from the early days of the Macintosh still work…

To contrast, I’ve got DOS applications that I wrote in 1983 for the very original IBM PC that still run flawlessly, thanks to the Raymond Chen Camp at Microsoft.

I can almost feel the revulsion among my readership right about now. However, next time you’re in Best Buy or CompUSA, look at the shelf of Windows applications, then compare it to the shelf of Mac applications, and perhaps you’ll better understand why it’s important.

Beyond the results speaking for themselves, I’ll argue that it takes a better engineer to move a platform forward while at the same time making sure things don’t break. It’s pretty easy to wash your hands of something and declare it to be someone else’s problem.

Steve Jobs: “You don’t want your phone to be an open platform.”

More Steve Jobs: “These are devices that need to work, and you can’t do that if you load any software on them. That doesn’t mean there’s not going to be software to buy that you can load on them coming from us. It doesn’t mean we have to write it all, but it means it has to be more of a controlled environment.”

It’s widely understood that Microsoft beat Apple in the PC game because Apple was a tightly integrated, closed platform, whereas Microsoft was an open ecosystem where anyone could play (albeit at their own peril).

Clearly, the strategy has not adapted well to digital media players. Why? Because people are tired of products that don’t work. Anyone who has ever had to glue a PC back together for a family member knows what I mean.

Does “open” have to mean “poorly integrated” though? Clearly, I don’t think so, given how much I’ve invested in Linux over the years and how much I’m investing in the LSB now. And the job the distributions do in the Linux world is nothing short of amazing, if you think about it—taking thousands of moving parts written by thousands of different people and crafting them into what appears, for the most part, to be an orchestrated whole.

Personal experience bears this out too—it’s not that the entire Windows Media ecosystem produces bad products (iTunes pales in comparison to Rhapsody), it’s that the integration points are extremely fragile (e.g., I tried Rhapsody To Go with a Plays For Sure device, which was an awful experience). In other words, just like a chain, an open ecosystem is only as strong as its weakest link, because it’s the links where things break.

Sure, there are bad products. I’ve owned no less than two MP3 players in the last few years, and they were pretty terrible.

It’s the very, very simple things too. For example, I listen to a lot of podcasts. There’s (gasp!) a Podcasts menu on the iPod, whereas on every other device I’ve owned, podcasts are interspersed with the music, making them difficult to find and, adding insult to injury, mixing gobbledygook into my otherwise well organized collection. Furthermore (gasp!), the iPod keeps track of which podcasts I listen to, and in concert with iTunes, automatically deletes the ones I’m done with. Them’s lasers!

Ah, iTunes. Despite the fact that anyone in the world can write a podcatcher (it’s just RSS with enclosures, fer godsakes), the only one I’ve ever used that works worth a damn is iTunes. Everything else is just “integrated” with bailing wire—create a Windows Media playlist, drop it in this directory, etc.

The situation on Linux is a bit better, but guess what—Rhythmbox, Banshee, etc. didn’t work with either of the open media players I owned, even though the integration point for one of them was the file system. No, the players on Linux support the iPod best because that’s what everyone has, and if I’m going to pay extra for the iPod, aren’t I primarily buying the integrated experience? (On a related note, I’ve never understood why people pay top dollar for Macs and install Linux on them. But I digress..)

Once you taste the fruit of the integrated experience, it’s easy to get sucked in all the way. I eventually tired of all the wasted time and money trying to use an open media player and broke down and bought an iPod. Not too long after, I canceled my Rhapsody subscription. Why? Because I couldn’t use it with my iPod. My overall music experience is diminished (I’m a big believer in the celestial jukebox), but I want one music platform, not two. And the iPod is the one with critical mass. Anything else is swimming upstream.

Are open platforms doomed then? I think that’s, as they say, throwing the baby out with the bath water. The key is to have open interfaces, and the key to having open interfaces that work is cooperation between the vendors that implement them. Perhaps surprisingly, the iPod is a good example of how it can be done. The biggest reason for my iPod purchase: I can hook it directly into my car stereo instead of using an FM transmitter. Apple isn’t getting into the car business—it’s decided to cooperate with other companies to make this work. And you don’t hear too much about people calling their technophile relatives to come glue their cars back together. (And talk about the importance of not crashing..)

The lesson here, it seems to me, is that an ecosystem is never the sum of its parts—it’s either a whole lot more or a whole lot less. Furthermore, it’s the responsibility of the vendors (and community) that make up that ecosystem to ensure the result comes up on the right side of the equation in the final analysis.

Richard MacManus: “Instead of using the entire product suite of a Google or an MSN or a Yahoo, you [should be able to] use the particular apps you like most from not only big players - but small startups too.”

I agree wholeheartedly. I’m a big believer in the componentization of the web, as I’ve written about before. Without a way to write independent apps that integrate with each other regardless of where they’re hosted, the future starts to look more and more like the mainframe era (albeit with better graphics). The world needs only five computers (shaping up to be Amazon, Google, Salesforce.com, Windows Live and Yahoo!). However, we need to ensure that these five computers operate more as computing utilities where a thousand flowers can bloom than as centralized monoliths where the only way to integrate is to be assimilated.