Software comments: quality and metaphor

My paternal grandfather was a railway man. In those days lots of people were. He must have had opinions about how to run a train, but he didn't have a blog.

Today software is the latest railway, but we have blogs. So people like Joel Spolsky, who has about a million or so readers, tell us some very interesting things:

Hitting the High Notes - Joel on Software

.... the conventional wisdom in the world of copycat business journalists and large companies who rely on overpaid management consultants to think for them, chew their food, etc., seems to be that the most important thing is reducing the cost of programmers.

In some other industries, cheap is more important than good. Wal*Mart grew to be the biggest corporation on Earth by selling cheap products, not good products. If Wal*Mart tried to sell high quality goods, their costs would go up and their whole cheap advantage would be lost. For example if they tried to sell a tube sock that can withstand the unusual rigors of, say, being washed in a washing machine, they'd have to use all kinds of expensive components, like, say, cotton, and the cost for every single sock would go up.

So, why isn't there room in the software industry for a low cost provider, someone who uses the cheapest programmers available? (Remind me to ask Quark how that whole fire-everybody-and-hire-low-cost-replacements plan is working.)

Here's why: duplication of software is free. That means that the cost of programmers is spread out over all the copies of the software you sell. With software, you can improve quality without adding to the incremental cost of each unit sold.

Essentially, design adds value faster than it adds cost...

I've read similar arguments in discussions about artists and entertainers -- such as professional baseball players). A hundred journeyman players can't deliver anywhere near the value of one superstar. The incremental cost of sending signal to a few more antennae is very, very small.

I hadn't, however, though about how well this applies to software as well. The incremental cost of creating another CD is very, very small.

In most industries quality doesn't scale all that well. In software, it can. It makes sense to pay enough to hire the very best designers and coders, and to give them working conditions that make them happy and productive. That's what Spolsky has done (See the "in house" software description.) I hope the meme is contagious.

So let's accept that producing software is still very hard and it takes really good and expensive people to create good software.

By why is it so hard to create good software, and why do so many products never fulfill their early promise?

I think I can add something from my own experience here.

I think we have trouble creating and nurturing great software because it's a very long process with a thousand different paths to an premature end. You can get a lot of things right, have a few good years of growth, then fumble a critical transition and see your work die a slow and ugly death.

We commonly use metaphors from architecture, literature, or music to describe the software creation process. I think they all add value, but I think they miss the organic aspect of how software grows and evolves. Maybe a better metaphor is building a company, creating a formal garden, or, perhaps, raising a particularly challenging child.

There's no perfect metaphor, but the common ones have a trap. They imply that there's a point that a product is "done" and the "A team" can move on. I'm sure that's sometimes true, but even then process may take 10 years. In many instances complex software remains a very demanding beast, requiring a lot of skill and ingenuity to keep it health and adaptive. Often that skill has moved on and the software can enter a spiral of every increasing entropy.

I think the long life cycle of software, and the easily underestimated costs of keeping a complex software beast "healthy", help explain why truly excellent software is unusual.