Left head bites right head -- Gmail filters blogger comments to spam box

Gmail is mostly impressive -- except for the spam filtering. Creakly old Yahoo mail, or even my local ISP, does a much better job at separating the wheat from the chaff.

Gmail errs in both directions. It puts spam in my inbox, and not-spam in my spambox. It invariably filters comment submissions from blogger into the spambox -- even though both Blogger and Gmail are Google properties. Google's a multi-headed monster, and the heads aren't necessarily on good terms.

Credit to the young scientists of South Korea

In a sad but not unfamiliar story of fame and failure, a once famed and now infamous South Korean scientist is suspected of faking a series of breakthrough articles. His scientific career is finished, though one hopes he will use his talents to good aims in other areas.

South Korea is said to take his downfall very much to heart, but there's another angle to the story:

Scientist Faked Stem Cell Study, Associate Says - New York Times

... Although the new disclosures are being presented as a blow for South Korean science, they can also be seen as a triumph for a cadre of well-trained young Koreans for whom it became almost a pastime to turn up one flaw after another in his work. All or almost all the criticisms that eventually brought him down were first posted on Web sites used by young Korean scientists.

Now there's a story that deserves to be told. The difference between science and the Alternative, is a system for challenge and disproof. (South) Korean scientists showed real skill and leadership in exposing a scientific fraud. They deserve to be honored.

Winner of the Homeland Security Incompetence Award

Bruce Schneier, security guru, rails against the extraordinary stupidity of a "watch list" for airline passengers:

Wired News: Airline Security a Waste of Cash

... Consider CAPPS and its replacement, Secure Flight. These are programs to check travelers against the 30,000 to 40,000 names on the government's No-Fly list, and another 30,000 to 40,000 on its Selectee list.

They're bizarre lists: people -- names and aliases -- who are too dangerous to be allowed to fly under any circumstance, yet so innocent that they cannot be arrested, even under the draconian provisions of the Patriot Act. The Selectee list contains an equal number of travelers who must be searched extensively before they're allowed to fly. Who are these people, anyway?

The truth is, nobody knows. The lists come from the Terrorist Screening Database, a hodgepodge compiled in haste from a variety of sources, with no clear rules about who should be on it or how to get off it. The government is trying to clean up the lists, but -- garbage in, garbage out -- it's not having much success.

The program has been a complete failure, resulting in exactly zero terrorists caught. And even worse, thousands (or more) have been denied the ability to fly, even though they've done nothing wrong. These denials fall into two categories: the "Ted Kennedy" problem (people who aren't on the list but share a name with someone who is) and the "Cat Stevens" problem (people on the list who shouldn't be). Even now, four years after 9/11, both these problems remain.

Can I weep now?

This is similar to the same problem of deciding that two health records belong the the same person. That's a hard problem, but if you use a combination of attributes (various identifiers, SSN, age, address, name) from reasonably robust sources you can make some trade-off between false matches and false non-matches. Having a national identifier (passport number) or even a state identifier (driver's license) makes the problem a bit simpler.

The reason using this in airport screening is completely stupid is:

1. Intelligent terrorists don't want to be matched, so they'd obfuscate data they provided. Duhhhh.
2. If name and age are the only identifiers, and the goal is to avoid misses at all costs, the error rate (false accusation) will be incredible. I'd imagine well over 10,000 to 1 (10,000 mistakes for every success, probably it's more like 1,000,000 to 1).
3. There's no mechanism to deal with mistakes, and the outsourced vendors don't pay a price for their errors.
   

The matching part of this could be made to work -- through a draconian system of biometric authentication. Even then, as Schneier points out, this would only identify known terrorists, and it would still leave the option of using non-terrorists as unwitting accomplices.

Read Schneier's essay. This is a stupid program proposed by idiots and implemented by dolts. It wins the prize.

nthposition online magazine

"Follow Me Now" lead me to an article in nthposition online magazine. It's quite a fascinating production, but I'd never heard of it. There's so much out there in the shadows of the Net. Alas, their RSS feed appears to be quite broken. I sent them a note.

If you're interested in literature, fine arts, and the world - take a look!

Update 12/21/05: Alas, they have lost the tech person who built the site -- so they can't fix the feed problem. If you know anyone who'd like to volunteer, they're looking!

The internet and "white flight"

"White flight" was an old term for the departure of euros from Birmingham, Detroit and many other American cities during the 1960s and 1970s. I suspect it wasn't merely a matter of melanin deficiency, I think many middle-class folk with adequate melanin also decided to decamp for less troubled spots.

My sense is the same thing is happening with the internet, and with email in particular. Over the past few years my neighbors have increasingly given up on email -- particularly outside of the workplace. They usually can't explain why, but if you drill down it's spam, viruses, worms, etc. It just wears people down. They stop checking their email regularly, and then the spam pile-up is even worse. Eventually they give up on email. They may even give up on the net entirely.

The Internet is simply becoming a "bad neighborhood". I don't think that reality is getting factored into enough business plans. I would not be surprised to see a decrease in internet use over the next few years, even as the net is increasingly used for media distribution ....

Eating the egg -- earth and humanity

Monbiot writes for the Guardian. He's sometimes interesting, but more emotional than analytical. So it's noteworthy that he's starting to confront some ugly realities about CO2 emissions and energy alternatives:

George Monbiot: Worse Than Fossil Fuel

In 2003, the biologist Jeffrey Dukes calculated that the fossil fuels we burn in one year were made from organic matter “containing 44 *10 to the 18 grams of carbon, which is more than 400 times the net primary productivity of the planet’s current biota.” In plain English, this means that every year we use four centuries’ worth of plants and animals.

The idea that we can simply replace this fossil legacy – and the extraordinary power densities it gives us – with ambient energy is the stuff of science fiction. There is simply no substitute for cutting back. But substitutes are being sought everywhere. They are being promoted today at the climate talks in Montreal, by states – such as ours – which seek to avoid the hard decisions climate change demands. And at least one of them is worse than the fossil fuel burning it replaces.

The last time I drew attention to the hazards of making diesel fuel from vegetable oils, I received as much abuse as I have ever been sent by the supporters of the Iraq war. The biodiesel missionaries, I discovered, are as vociferous in their denial as the executives of Exxon. I am now prepared to admit that my previous column was wrong. But they’re not going to like it. I was wrong because I underestimated the fuel’s destructive impact...

Minnesota is big on biodiesel, but it's really a side-show. Sigh. Not much has changed since the energy crisis of the early 1980s, when OPEC gave us a preview of "peak oil". One of the best books of that era was an analysis of the scope for conservation without impacting lifestyle. I wish I had a copy of that book, I suspect that, by and large, we've implemented many of the recommendations.

Even so, with some lifestyle changes (smaller homes, denser communities) that aren't all negative, we still have a lot of room to conserve. I suspect we could reduce our consumption 20% or more without new technology and without making our lives miserable; maybe we'd even lose some weight. It's sad that Bush has foreclosed this option for the US ...

Roy Moore, Alabama, and Christian Reconstruction

Orcinus writes about Roy (10 Commandments) Moore, Alabama, and the Christian Reconstruction movement. My one Alabama contact assures me Roy won't win the governorship, but the article is still well worth reading. Christian Reconstruction (think "Left Behind" and "The Crusades") is a fringe movement, but like many fringe movements it influences much mainstream thinking. I am so tired of theocracy.

Gwynne Dyer has 3 new essays up

Gwynne Dyer, the last man to enter the internet age, will be the last man to get an RSS feed. In the meantime, he has 3 new essays on his pleasantly spartan web site:

Dyer 2005

28 November Kyoto and the Blair Switch
2 December Japan, China "Congagement"
5 December Rice and Count Metternich
8 December The Last Anti-American

As usual, each essay provides unique historical and geopolitical insights. For example:

The flights were presumably carrying Muslim detainees between the US-run prison camps in Cuba, Iraq and Afghanistan, other secret CIA camps that allegedly existed in Poland, Romania and the Indian Ocean island of Diego Garcia, and places like Egypt and Syria in the case of those destined for major torture or death. Thousands of detainees may have been carried on these "ghost flights" over the past four years, and Lawrence Wilkerson, a former US army colonel who served as chief of staff to former secretary of state Colin Powell from 2002 until early this year, told the BBC last week that between 70 and 90 prisoners have died in "questionable circumstances."

Incidentally (somewhat relevant to the 12/8 essay), the USA Today had an article buried in the paper about how some pompous wretch in the Bush administration was suggesting Canada's Prime Minister tone down his anti-American rhetoric lest he endanger US-Canadian relationships. Either this was an attempt to help Martin by suggesting Bush dislikes him, or the Bushie wants to see just how seething anti-American rhetoric can get. Hmm. Devious or stupid. That's always the question with the Bush administration ... (I personally suspect both are sometimes true.)

Blind spots, tech commentary and complexity

"Digital music" (odd term, CDs are digital - really should be "lossy compressed music") is "big". So it wasn't surprising that NPR spent an hour or so this morning with a tech columnist talking mostly about "MP3" (meaning AAC, MP3, MP4, etc etc) players. What was a bit surpising, and annoying, was that the expert seemed to have never of heard of something called "iTunes". He compared Dell's MP3 player to the iPod as though the devices existed in isolation and felt they were of roughly equal value.

His advice was thus fairly worthless. An "MP3 player" today is only as good as the desktop software it works with; for the moment the billions of CDs in circulation keep iPods and their competitors bound to XP or OS X. (There are lots of ways this could change dramatically, but that's another story.) iTunes is a brilliant piece of software, and much of the success of the iPod is due to iTunes. He was comparing jet engines instead of jets.

I see this kind of glaring omission reasonably often. Is the complexity of our world ovewhelming the "experts", or is this simply an old story -- the popular 'expert' is selected more often for entertainment value than expertise ...

Samsung - hiring 5000 engineers for DRAM work?

This is astounding:

Among Makers of Memory Chips for Gadgets, Fierce Scrum Takes Shape - New York
Times:

...Samsung also said it would hire 5,000 more engineers to increase research and development of new chips...

Five thousand engineers? That's not a typo, though the hiring is over 7 years. Even so, if they mean real engineering school graduates that's a heck of a lot of brainpower. That's 50 years output from a typical engineering school. They'd have to pillage the engineering powerhouses of the world: India, China, Taiwan, South Korea ...

I can't see how they could effectively use that many engineers ...

Illogical design - the Narwhal's tusk

Only evolution could produce such as klduge, such a hack, as the Narwhal's tusk:

It's Sensitive. Really. - New York Times

The find came when the team turned an electron microscope on the tusk's material and found new subtleties of dental anatomy. The close-ups showed that 10 million nerve endings tunnel from the tusk's core toward its outer surface, communicating with the outside world. The scientists say the nerves can detect subtle changes of temperature, pressure, particle gradients and probably much else, giving the animal unique insights.

The tusk started out as a tooth; the research was led by an adventurous dentist. It makes a bizarre sort of sense; anyone with an exposed tooth root or broken tooth knows how sensitive teeth can be to temperature. Invert the tooth, put the soft tissue on the outside, and you have a weather sensor.

Evolution works with the tools at hand. The Narwhal is now Darwin's poster child.

A plausible summary of the state of Iraq

This feels plausible.

BBC NEWS | Middle East | Danger cannot dent Iraqi optimism

... So here, courtesy of the BBC poll, is a snapshot of Iraq today: a country whose people often seem close to civil war, yet feel overwhelmingly safe in their own neighbourhoods.

Iraqis are scathing about the performance of the American, British and other troops, yet believe it's too soon for them to leave.

People are so worried the country is falling apart that they want a strong government to take control, yet believe that in a few years' time things will be really good here.

These findings are only useful if you bear in mind how complex and varied Iraqi society is.

Often, when I write my online column about Iraq and try to explain about life in this country, someone will write in and say something like: 'My son is a soldier in Iraq and people there keep telling him what a wonderful job the US is doing', or alternatively: 'Iraqis just want to get rid of all foreign troops, period.'

Both things are true; it's just that different Iraqis are saying them. Unless you understand who they are - and why - it's impossible to make out what is happening in the country.

And in the meantime the one thing that everyone can agree with is that life is much more dangerous in Iraq than it was when our last opinion poll came out.

Wozniak agrees: Apple's software quality is truly abysmal

It pains me to acknowledge that Microsoft's software quality in the past year has been substantially superior to Apple's. It's not just their relentlessly buggy software releases, it's the vast weight of half-implemented OS features. OS X services? AppleScript? Automator? Those absurd widgets that don't run on the desktop as they should? Quartz Extreme? No reliable and useable way to address a user session via AppleScript? Most of all, the churn in the OS that breaks applications with each update (that is supposed to improve post 10.4).

It turns out that someone with far more credibility than I feels the same way. Wozniak is an OS X user, but he's also a legendary engineer (emphases mine, broader implications below):

The Cardinal Inquirer - Talking with The Woz

Do you think that is at all similar to the computer industry, where engineers develop a product and someone else sets the price?

It's very much like that, but sometimes the engineers are - yeah, no, I think it's very similar, very similar. Sometimes the engineers are true artists and really care what they're doing, doing a really great job. Although, I don't know how much I can even say that because the big companies, Microsoft, Apple and AOL, they tend to turn out the crappiest products, you know, software-wise. The ones that have the most bugs, the most items that are supposedly in there but don't work. The most things that are left out because they aren't finished. The most things that are inconsistent with the way they did their last program. I get the worst, worst software almost always from Apple.

You think so?

Oh yeah. I get third-party stuff and it's almost always just better, cleaner and more understandable. It works better and does what you'd expect instead of, you know, buggy things or not what you expect.

Is OS X is problematic in that way?

I don't even call it a problem; it's just something you learn to work around. It's like, there was such a cleaner, good approach to it and they did this stupid thing. But remember, the people who wrote the OS X weren't the people who developed the Lisa and Macintosh. Those guys are gone.

Do you see this problem getting worse or better over time?

Worse. And part of it is because the software gets huge and complex and we're always moving to the new things rather than fix old things. I think a lot of it is because people just get complacent with what they have.

The Woz and I are about the same age (except he's a gazillionaire), maybe some of our reaction is generational. I think the young-uns have grown up in a world where things don't work reliably -- they're accustomed to routing around defects. Aged want things to work the first time and every time.

The one ray of hope is the small software companies. Maybe it's because there's often one person who's architect, analyst, engineer and QA. Maybe it's because these firms live or die by their reputations. I agree with Woz; it's the small shops that produce products that work.

Aside from the complacency of those who've grown up with unreliable and unstable systems, the complexity factor is hard to overstate. Despite enormous efforts, we don't have a handle on building reliable complex systems; the cost of reliability still seems to rise exponentially with complexity. An airplane is reliable, but the cost of that reliability is very high.

Beyond complexity, we have crummy software for the same reason we have a singularly crummy American government. In both cases, the aspects that sell aren't the things the customer really needs on an ongoing basis. We won't get better software until we get smarter consumers. I think that may eventually happen.

It's probably also true that we need to move from buying software to renting it, but that step requires very reliable data interchange. The combination of software rental and proprietary data formats is far worse than unreliable software.

More speculations on the evolutionary biology of acne

A comment on a blog posting

It's a humorous post, but the evolutionary biology of acne is, to me, quite fascinating.

The best theory I've heard of (or maybe I invented it, I'm not sure) is that the primary value is to render fertile young women less attractive, and to avoid male assault and early pregnancy. This theory would say acne is males is a side-effect of males and females sharing the same genes; it would have no adaptive advantage for males. We know from other gene studies that there are many genes that have such gender-specific value.

What is the relationship then with 'nerdiness' and pimples in males? I would wonder about a correlation between maternal androgen levels, maternal mate selection, and intrauterine androgen exposure affecting adolescent male aggressiveness. So the connection would be subtle, but both the 'nerdiness' (lack of aggression) and the acne would be indirectly a result of maternal adolescent acne."

Incompetent Design: a scientist points out the flaws in humanity

Seed: The Other I.D. points out that humans are very badly "designed".

Fun and interesting, no surprise to a physician of course. I enjoyed the comment on facial plumbing (aka sinuses). They are a disaster. I remember in medical school a lecturer earnestly explaining the role of the sinus as 'changing timbre of the voice' or 'warming inhaled air'. Bah!!

Now I'd like to see a physicist comment on the flaws in the universe. It seems far too hostile to life for my tastes (and yes, I do acutely follow discussions on finely balanced fundamental constants.)