Saturday, March 21, 2009

The senescent web - lessons from eHow's cedar shake advice

In the middle age of the web there are an astounding number of garbage web sites. I fear the ratio of garbage to genuine sites is now approaching ten to one on Google search pages for some topics.

Consider home repair. As near as I can tell several businesses have created junk-bases full of stolen or ridiculously lousy advice. They then publish this under hundreds of domains, and sell ads across all of them. Many of these sites show up in Google searches, so today they're winning the algorithm wars.

I recently read a science fiction writer's description of a future "web" that was 99.99999% garbage, some of it insanely subtle garbage. None of this was apparent to users, because near-sentient AIs filtered 99.99999% of the garbage out.

A completely unrealistic scenario of course, since any such AI would be fully sentient and quite uninterested in such a menial task.

The web, inevitably, is going the way of usenet. It's being overwhelmed by fraudulent junk. It's barely twenty years old, and it's already senile.

There's hope, however. We carry a huge amount of noise and parasitic junk in our DNA but complex life forms exist. From chaos can arise a metastable system.

Maybe, on the way to Skynet, the web will do the same. In the meantime, on the web, as in the real world, reputation and brand identity matter more than ever.

When I was looking for advice on repairing dog-broken cedar siding shakes, eHow, a relatively old and established site, had the best advice. It has ads too, but there's a reasonable balance between ads and content.

There's a lesson here for Google, Yahoo, and especially for Microsoft. Google's algorithmic approach to search is quite vulnerable to fraud. In some domains it's failing. On the other hand a simple-minded Yahoo-1996 style approach is vulnerable to bias and other kinds of deception. If Microsoft can find a way to solve this particular problem, by joining algorithmic and human judgment, they may yet challenge Google.

On the other hand, if Google solves it ...

No comments: