Undoing the Enlightenment: Divide First, Conquer Second

In an earlier post I digressed from some Senator Rick Santorum language associated with the No Child Left Behind legislation to a broader discussion of the Orwellian language used by creationists (now repacked as 'intelligent designists").

Upon discussion with my Muse (Emily), the diabolical strategy of the counter-Enlightenment agenda was revealed to me (how's that for theological language?). It is a strategy that Machiavelli, a scion of the Englightenment, would have most appreciated. For that matter, Sun Tzu would also approve. Consider this paragraph in a National Center for Science Education essay:

Santorum language appeared in the 2003 Kansas legislative session in the form of Senate Bill 168. SB 168 encouraged curricula that helped students understand "the full range of scientific views that exist," and exempted educational staff from any penalties for deviating from state curriculum requirements. SB 168 further betrayed its creationist leanings when it explicitly singled out "origins science" for special treatment, requiring that "origins science" -- but not other science -- be taught "inclusively, objectively, and without religious, naturalistic or philosophic bias or assumption." The artificial distinction between "origins science" (sciences dealing with the past) and "operations science" (sciences dealing with the present) has been a running theme in creationist publications for decades.

If one wished to undo the Enlightenment, how would one begin? One would not strike at strength, but at weakness. First divide, then conquer. "Operations Science" includes physiology, biochemistry, bacteriology, molecular biology, meteorology, chemistry, applied physics and far more. This is strong science. (Note, however, that medicine, a profession related to some of the strong sciences, has been shown to be vulnerable to attack.

On the other hand "origins science" is less immediate to the everyday life of the average person. Origins science is concerned with history, with fossils, with cosmology, and with evolution. The average citizen would feel that their life is not affected one bit by the study of dinosaurs, of quasars, of Neandertals, of the Aztecs, or of natural selection. This is vulnerable science. This is where one would start undoing the Englightenment.

Children of the Enlightenment -- arouse yourselves!

On spam, marketing, noise, search, reputation management and the evolution of the blog

Need an Outlook reference?

I was looking for a book on Outlook 2003. I don't need a novice reference, I need a power-user reference. I looked first at O'Reilly (the home of the power user), but they don't have anything current. Google and Amazon weren't helping, though Amazon was somewhat useful.

So I turned to an authority, a blog called Marc's Outlook on Productivity. Marc is not a super-power user, but he concentrates on this domain.

This is new. I could ask a friend, but I already know more about Outlook than anyone else I know. Five to ten years ago I'd have tried Usenet, but it's been done in by spam and blogs. Five years ago Google would have given me a good answer, now Google is overwhelmed by spam and marketing. Amazon usually works, but there aren't enough people interested in this topic to provide me with useful information. This isn't surprising, Amazon is often weak in dealing with multiple editions of computer books with a small readership -- there's too much dilution of a limited number of reviews. Once upon a time I'd have gone to a 'community site', but they have too much noise and are too hard to track.

So I took a new tack - the next best thing to asking an expert colleague for a recommendation. I went to the blog of an expert I trusted. I trusted this expert because I've followed his writing for a year or so. (Note: blog, expert, domain specific, reputation->trust)

The Post Categories in this blog (Note: ontology, terminology, classification, search, metadata) helped me find a relevant post very quickly. I ordered the book from Amazon. Unfortunately the publisher doesn't allow one to view the contents in Amazon (most annoying).

The pace of evolution of the web and of search in general is breathtaking. In ten years we've marched through an astonishing array of solutions, most of which have been destroyed by the twin demons of marketing and spam (arguably spam is a form of marketing!).

I wonder if the idea of personal authority and reputation will be the persistent solution. In all these years it's been a common thread. Attempt to automate (most nobly by Google) have fallen to the corrupting forces of marketing. Sure, people can be corrupted, but I've found that there's a goodly number of people that are surprisingly resistant to such influences. If they demonstrate this resistance over time, they become influential. (Then the corrupting forces can become irresistible, but there are always replacements .. and so it goes ...). Blogs, especially when they are authored by a single person, are the only way I know of in today's net to create an identify, a reputation, and finally, trust in domain recommendations.
am (noise). Interesting to consider what the biological equivalence is.

It's interesting to consider what the biological equivalence is.

PS. Note to marketing folk -- much of my professional life falls in the broad domain of marketing. Marketing is a powerful domain that rules much of modern life -- like all powers it has two faces.

Update 4/25/05: So, what was the book like? Not bad, but of the 1000 or so pages probably 30 are useful for me. People like me are not common enough to base a book on, so I guess I have to make do with a chapter her or there.

No Child Left Behind advocates teaching creationism?

Battle on Teaching Evolution Sharpens (washingtonpost.com)

The No Child Left Behind Law contains a language that espouses the teaching of creationism in science classes [see update below for a partial correction]:

Some evolution opponents are trying to use Bush's No Child Left Behind law, saying it creates an opening for states to set new teaching standards. Sen. Rick Santorum (R-Pa.), a Christian who draws on Discovery Institute material, drafted language accompanying the law that said students should be exposed to 'the full range of scientific views that exist.'

'Anyone who expresses anything other than the dominant worldview is shunned and booted from the academy,' Santorum said in an interview. 'My reading of the science is there's a legitimate debate. My feeling is let the debate be had.'

In today's Orwellian world "full range of scientific views" is a codephrase for "intelligent design" which is a code phrase for "creationism".

If Bush and his ilk mandated a mandatory Yahweh- only religion class in public schools they'd at least win points for honesty. Alas, they're too cowardly for that.

I think I'll start advocating the teach of chiropractic theories as an alternative to traditional human physiology. There's at least as much "legitimate scientific debate" around chiropracty as there is around the fundamental value of natural selection. (ie. zero in both cases).

Update 3/15: Some more on the Santorum Amendment from an authoritative source discussing the assault on reason taking place in Kansas -- ground zero for the counter-enlightenment. The Santorum amendment was stripped from the NCLB final legislation but remains in the conference report. It is this tactic that's being repeated at every level of government. (Emphases mine, note the full range of Orwellian strategies and neologisms -- George is spinning now ...)

According to the Lawrence Journal-World, an antievolution resolution was introduced in the Kansas House of Representatives on February 15, 2005. The sponsor is Representative Mary Pilcher-Cook (R-Shawnee), who said that the proposed resolution, which is nonbinding, was meant to promote "objectivity in science education."

Although the full text of the bill is not yet available, the story reports that the resolution includes language recommending the teaching of "the full range of scientific views that exist." This language is derived directly from the "Santorum Amendment," which U.S. Senator Rick Santorum (R-PA) attempted to insert into the federal No Child Left Behind Act in 2002. Phillip Johnson, a leading promoter of "intelligent design," wrote the amendment for Santorum. The Santorum Amendment was passed by the U.S. Senate but stripped from the bill by the House-Senate conference committee, and now only appears in modified form in the NCLB conference report. See NCSE's compilation on the Santorum Amendment for details.

Santorum language appeared in the 2003 Kansas legislative session in the form of Senate Bill 168. SB 168 encouraged curricula that helped students understand "the full range of scientific views that exist," and exempted educational staff from any penalties for deviating from state curriculum requirements. SB 168 further betrayed its creationist leanings when it explicitly singled out "origins science" for special treatment, requiring that "origins science" -- but not other science -- be taught "inclusively, objectively, and without religious, naturalistic or philosophic bias or assumption." The artificial distinction between "origins science" (sciences dealing with the past) and "operations science" (sciences dealing with the present) has been a running theme in creationist publications for decades.

On February 6, 2005, nine days before Pilcher-Cook introduced her antievolution resolution, an op-ed promoting the Santorum language appeared in the Kansas City Star. The opinion piece, written by Kansas City resident Vicki Palatas, was entitled "'Full range of scientific views' includes theory of a creator." Palatas wrote, "The report interpreting this legislation explains that on controversial issues like evolution, 'the curriculum should help students to understand the full range of scientific views that exist.'" Palatas cited several talking points popular among intelligent design proponents, and concluded, "Intelligent design teaches the theory of a creator based on scientific observation and analysis, not the worship of one." In another opinion piece on January 2, 2005, Palatas explicitly grouped intelligent design with a laundry list of other conservative religious causes, decrying the failure of public schools to "allow teaching intelligent design as a theory of the creation of the universe." She concluded that governmental restraint in this and other matters amounted to "intolerance and maligning of our faith."

Passed by the Senate, but stripped in a committee?? Wow, American politics is interesting - especially considering that the House is famously conservative.

The National Center for Science Education essay is horrifying and fascinating. Note the contradictory themes of the creationists who can speak both "intolerance of faith" and a "theory of a creator". Or perhaps they are not so inconsistent -- for it is their faith that assures them that the "analysis of the creator" will give them the answer they expect. Historians of science might tell them that such analyses often give quite surprising answers. If we do develop methods to test for analyze "deities", Palatas and her ilk may yet regret their enthusiasms.

More practically, the lessons of the past few decades is that extremists are very, very persistent. They love the struggle itself. Rationalists fatigue, they move on, they have a life (yes, even I have a life -- actually, rather a lot of one). In our era the dominant extremists in America are right wing conservatives -- I think they'll win in Kansas. Science is going to take a serious beating for years to come.

Hmm. The enlightenment is beginnning to feel pretty anomalous. I need to consult a historian.

Race as a collection of genes that travel together

The New York Times > Opinion > Op-Ed Contributor: A Family Tree in Every Gene

The hypothesis is that genes tend to travel together, and that it's possible to assign a human being living today to a geographically isolated population in which a characteristic collection of genes was very common. That assignment is a "race".

This is a statistical model of race. Imagine a 'gene-space' consisting of (say) 100 or so marker gene values. If we treat this as a 100-dimension space then an individual human should appear as a point in this space. If we then add a dimension for frequency, we may "see" hills and valleys on this "surface". Those are "races". Most of us are somewhere on the flank of a mountain, but there ought to be (how can one resist the word?) "pure" folk at the peaks. Conversely we ought to be able to find folks living in "valleys" who are truly "unique". (Some say I'm "different", but I think they have something else in mind.)

Here's how Leroi puts it:

The New York Times March 14, 2005
A Family Tree in Every Gene
By ARMAND MARIE LEROI

... If modern anthropologists mention the concept of race, it is invariably only to warn against and dismiss it. Likewise many geneticists. "Race is social concept, not a scientific one," according to Dr. Craig Venter - and he should know, since he was first to sequence the human genome. The idea that human races are only social constructs has been the consensus for at least 30 years.

The dominance of the social construct theory can be traced to a 1972 article by Dr. Richard Lewontin, a Harvard geneticist, who wrote that most human genetic variation can be found within any given "race." If one looked at genes rather than faces, he claimed, the difference between an African and a European would be scarcely greater than the difference between any two Europeans...

Three decades later, it seems that Dr. Lewontin's facts were correct, and have been abundantly confirmed by ever better techniques of detecting genetic variety. His reasoning, however, was wrong...

.... The shapes of our eyes, noses and skulls; the color of our eyes and our hair; the heaviness, height and hairiness of our bodies are all, individually, poor guides to ancestry.

But this is not true when the features are taken together. Certain skin colors tend to go with certain kinds of eyes, noses, skulls and bodies... To put it more abstractly, human physical variation is correlated; and correlations contain information.

Genetic variants that aren't written on our faces, but that can be detected only in the genome, show similar correlations. It is these correlations that Dr. Lewontin seems to have ignored. In essence, he looked at one gene at a time and failed to see races. But if many - a few hundred - variable genes are considered simultaneously, then it is very easy to do so...

... Study enough genes in enough people and one could sort the world's population into 10, 100, perhaps 1,000 groups, each located somewhere on the map. This has not yet been done with any precision, but it will be. Soon it may be possible to identify your ancestors not merely as African or European, but Ibo or Yoruba, perhaps even Celt or Castilian, or all of the above.

... Hispanics, for example, are composed of a recent and evolving blend of European, American Indian and African genes, then the Uighurs of Central Asia can be seen as a 3,000-year-old mix of West European and East Asian genes.

... When the Times of India article referred to the Andaman Islanders as being of ancient Negrito racial stock, the terminology was correct. Negrito is the name given by anthropologists to a people who once lived throughout Southeast Asia. They are very small, very dark, and have peppercorn hair. They look like African pygmies who have wandered away from Congo's jungles to take up life on a tropical isle. But they are not.

The latest genetic data suggest that the Negritos are descended from the first modern humans to have invaded Asia, some 100,000 years ago. In time they were overrun or absorbed by waves of Neolithic agriculturalists, and later nearly wiped out by British, Spanish and Indian colonialists. Now they are confined to the Malay Peninsula, a few islands in the Philippines and the Andamans...

The full article tries to justify race identification as a way to improve healthcare. I'm skeptical. Maybe as an interim approach, but we'll do better with pharmacogenomics than race as a proxy for individual gene values. I'd call this an interesting hypothesis rather than something that's immediately useful.

China will pay - the bennies of empire

Recently I predicted (hey, I'm not alone!) that our party-days are numbered, that foreigners won't continue to fund our profligate ways. Nonsense, says Niall Ferguson. China will keep the taps open much longer for the US than for, say, Argentina. It's all the wages of empire, the ROI for our massive military and young warriors ...

The New York Times > Magazine > Niall Ferguson> The Way We Live Now: Our Currency, Your Problem

Every congressman knows that the United States currently runs large ''twin deficits'' on its budget and current accounts. Deficit 1, as we well know, is just the difference between federal tax revenues and expenditures. Deficit 2 is generally less well understood: it's the difference between all that Americans earn from foreigners (mainly from exports, services and investments abroad) and all that they pay out to foreigners (for imports, services and loans). When [only] a government runs a deficit, it can tap public savings by selling bonds. But when the economy as a whole is running a deficit -- when American households are saving next to nothing of their disposable income -- there is no option but to borrow abroad.

There was a time when foreign investors were ready and willing to finance the U.S. current account deficit by buying large pieces of corporate America. But that's not the case today. Perhaps the most amazing economic fact of our time is that between 70 and 80 percent of the American economy's vast and continuing borrowing requirement is being met by foreign (mainly Asian) central banks.

Let's translate that into political terms. In effect, the Bush administration's combination of tax cuts for the Republican ''base'' and a Global War on Terror is being financed with a multibillion dollar overdraft facility at the People's Bank of China. Without East Asia, your mortgage might well be costing you more. The toys you buy for your kids certainly would.


Why are the Chinese monetary authorities so willing to underwrite American profligacy? Not out of altruism. The principal reason is that if they don't keep on buying dollars and dollar-based securities as fast as the Federal Reserve and the U.S. Treasury can print them, the dollar could slide substantially against the Chinese renminbi, much as it has declined against the euro over the past three years. Knowing the importance of the U.S. market to their export industries, the Chinese authorities dread such a dollar slide...

... there are three good reasons that the administration is tacitly delighted by the Asian central banks' support. Not only is it keeping the lid on the price of American imports from Asia (a potential source of inflationary pressure). It is also propping up the price of U.S. Treasury bonds; this in turns depresses the yield on those bonds, allowing the federal government to borrow at historically very low rates of interest. Reason No. 3 is that low long-term interest rates keep the Bush recovery jogging along.

Sadly, according to a growing number of eminent economists, this arrangement simply cannot last. The dollar pessimists argue that the Asian central banks are already dangerously overexposed both to the dollar and the U.S. bond market..

... Though neither side wants to admit it, today's Sino-American economic relationship has an imperial character. Empires, remember, traditionally collect ''tributes'' from subject peoples. That is how their costs -- in terms of blood and treasure -- can best be justified to the populace back in the imperial capital. Today's ''tribute'' is effectively paid to the American empire by China and other East Asian economies in the form of underpriced exports and low-interest, high-risk loans.

How long can the Chinese go on financing America's twin deficits? The answer may be a lot longer than the dollar pessimists expect. After all, this form of tribute is much less humiliating than those exacted by the last Anglophone empire, which occupied China's best ports and took over the country's customs system (partly in order to flood the country with Indian opium). There was no obvious upside to that arrangement for the Chinese; the growth rate of per capita G.D.P. was probably negative in that era, compared with 8 or 9 percent a year since 1990.

Meanwhile, the United States may be discovering what the British found in their imperial heyday. If you are a truly powerful empire, you can borrow a lot of money at surprisingly reasonable rates. Today's deficits are in fact dwarfed in relative terms by the amounts the British borrowed to finance their Global War on (French) Terror between 1793 and 1815. Yet British long-term rates in that era averaged just 4.77 percent, and the pound's exchange rate was restored to its prewar level within a few years of peace.

An intelligent discussion of the Italian hostage (Sgrena) shooting

The New York Times > Opinion > Op-Ed Contributor: Manning the Barricades

The author of this Op-Ed piece is a former Marine captain. He discusses the Sgrena shooting. He rapidly dispenses of the absurd claim that Sgrena's car was specifically targeted (if so, why is she alive?). He also points out that arguments about prior notification are irrelevant. Most of all, he had informed ideas on what to do differently. Of course I also like the recommendations because they're what I thought of myself after a the young children of accidentally killed Iraqis were famously photographed covered with blood.

... Unfortunately, instead of helping to answer that question, the uproar after the shooting has focused on two distractions. From her hospital bed, Ms. Sgrena hinted that the Americans had tried to kill her to protest Italy's policy of negotiating with hostage-takers. Her assertion begs the questions of what the United States could possibly gain from such an act and, why, after approaching her car, the soldiers apologized and called for medical help rather than finishing the job.

More dangerous, because it sounds more plausible, is the claim that proper coordination between Italian and American authorities could have prevented the shooting. Gen. George W. Casey Jr. , the top American commander in Iraq, said Italian officials gave no advance notice of the car's intended route. Prime Minister Silvio Berlusconi disagrees. This dispute is a red herring. No high-level government coordination, short of an American military escort for Ms. Sgrena's car, would have changed the outcome on that highway. The pivotal players were the men on the ground.

A hallmark of modern warfare is what the Marine Corps calls the "strategic corporal." The immense firepower of our troops, the haphazard nature of the Iraqi insurgency and the ever-watching eyes of the global news media combine to place decisions of strategic consequence on the shoulders of the junior-most troops. Consider the videotaped shooting of a wounded insurgent by a marine during the fight over Falluja in November, or the atrocities committed by soldiers at Abu Ghraib referred to by some in the military as "the seven idiots that lost the war." The training provided to young marines and soldiers must be commensurate with the extraordinary demands we now make of them.

The fact is, checkpoint techniques can be taught. My platoon had to learn them on the fly, but that was two years ago. The lessons we and other troops learned should have been institutionalized long ago.

For example, we tried and discarded the three tactics that were used to warn the Italians as they approached the checkpoint: hand and arm signals, warning shots and shooting into the vehicle's engine block. We found that hand and arm signals were tough to decipher, and subject to different cultural interpretations. Warning shots are hard to hear or see, and frequently only panic the driver they're intended to warn. Shooting into engine blocks to avoid injuring passengers is Hollywood fantasy. Even my Marine snipers - some of the best marksmen in the world - couldn't do it consistently.

So we adapted. For example, once while driving through a town, we cut down a traffic sign - a bright, red octagon with the word "stop" written in Arabic - and used it at checkpoints. Who knows how many lives this simple act of theft may have saved? We also learned to shoot off highly visible smoke grenades and brightly colored flares when possible threats approached. We started putting our concertina wire at least two football fields away to give us more reaction time.

Every combat unit learns its own lessons from hard experience. The important thing is that they be passed on so they are not continually relearned at the cost of innocent lives. Americans must understand that tragic mistakes in war are unavoidable, but that every legal, moral and strategic imperative demands that they be kept to a minimum. This is our obligation to Ms. Sgrena and to Mr. Calipari's family, to the thousands of Iraqi civilians who pass through military checkpoints each day, and to the Americans who must man them and live with their decisions.

David Brin has a blog

Contrary Brin

David is one of the smartest and most interesting thinkers I know of. I was surprised to see he has a blog. I'll have to read through the archives, he started about 5 months ago. Poor guy is using Blogger (as am I). Ouch.

PLATO Notes, Microsoft Groove, and the curious history of software

This is a bit more software-centric than my usual 'Notes' postings, but it's really not about a particular technical issues, rather it's an interesting and topical ancectdote about how software evolves. Once upon a time I thought software largely came from the imagination of a few people. Sometimes it does (for better or worse), but most complex software projects have a long and often unrecognized legacy.

Groove is in the news today, it's a software solution recently acquired by Microsoft. Ray Ozzie, the CEO of Groove, will become Microsoft's Chief Technologist. Microsoft's involvement has created the recent interest in this "new" software.

Groove seems new, but it's been in development for at least six years. It's not six years old, however, because it's an offspring of Lotus Notes, which was developed in the 1980s. But it's not twenty years old, because it's really a descendant of PLATO Notes, which was developed at the University of Illinois in the 1970s atop the 1960s (1950s?) PLATO platform. So it's thirty years old. Heck, one could argue that it's really a child of the Memex (1945), so it's about sixty years old.

This is what Kapor of Lotus/spreadsheet fame wrote about the connection of Groove to PLATO:

Mitch Kapor's Weblog: Microsoft Acquires Groove
Ray has been a colleague and friend for over 20 years. He came to Lotus is 1982 with the vision of Notes already in mind, having been inspired by the PLATO system he used as an undergraduate at the University of Illinois...

Kapor's posting led me to a Google search, and thus quickly to a history of PLATO Notes, a pre-PC system for communication and collaboration. The history is well worth reading for anyone who develops or works with complex software systems, or who is just interested in the history of ideas. There are lessons there about electronic community (10 million hours!), about open source development, about the software development process, about software evolution, about software-as-platform -- and more besides.

There are also some minor personal serendipities here. I am writing this on a blog, a modern version of the kind of collaborative community that PLATO pioneered. I live in Saint Paul, and PLATO Notes was commercialized by a Minneapolis company -- Control Data. I have worked with many Control Data veterans who no doubt have connections to the CDC PLATO team, but, in addition, I have a longstanding interest in collaborative software systems (warning: old web pages). About 8 years ago my interest led me to review several alternatives and to comment on the work of David Woolley and his web conferencing guide.

David Woolley, as a young man, created PLATO Notes in 1973; he wrote the article I mention above. David is also a leader at Minnesota e-Democracy, which I've long appreciated. I shall have to send him a note of appreciation.

Update 3/15: David Woolley corrected some errors I made in dates. Thanks David!

What's worse than no privacy? Lies.

A while back I posted on the (gross) errors that my shadow medical profile is likely accumulating thanks to a persistent billing error. I thought I was making a prediction, but tomorrow is today. The data stolen from ChoicePoint (and Lexis/Nexis and everywhere else) is full of errors:

MSNBC - ChoicePoint files found riddled with errors
By Bob Sullivan MSNBC

... Pierce, a privacy advocate, obtained her report nearly two years ago, long before the current controversy. Thanks to the unknown source -- perhaps a company employee, Pierce said, but she has no way of knowing -- she got a rare privilege most consumers don't: a chance to see what ChoicePoint knows about her.

... What first caught Pierce's eye, she said, was a heading titled "possible Texas criminal history." A short paragraph suggested additional, "manual" research, because three Texas court records had been found that might be connected to her. "A manual search on PIERCE D.S." is recommended, it said.

Pierce says she's only visited Texas twice briefly, and never had any trouble with the law there.

"But if I was applying for a job, and there were other candidates, and this was on my record, the company would obviously go for another person," she said. "It raises a question in your mind."

... On ChoicePoint's Web site, the National Comprehensive Report is described as a collection of searches that glean data from "national and state databases for a summary of assets, driver licenses, professional licenses, real property, vehicles, and more. Each report offers the ability to add associates to the report, which include relatives, others linked to the same addresses as the subject and neighbors."

... Under former addresses, an ex-boyfriend's address was listed. Pierce said she never lived there, and in fact, he moved into that house after they broke up. The report also listed three automobiles she never owned and three companies listed that she never owned or worked for.

Under the relatives section, her sister's ex-husband was listed. And there are seven other people listed as relatives who Pierce doesn't know...

...Most alarming to Pierce is the fact that, with all this information, the ChoicePoint report she received had glaring omissions, too. Many of her former addresses aren't listed; and despite the host of other people listed on her report, many relatives and nearby neighbors were missing.

... Pierce's experience neatly parallels that of Richard Smith, another privacy advocate, who paid a $20 fee and received a similar report from ChoicePoint several years ago. The company offers a wide variety of reports on individuals; Smith purchased a commercial version that's sold to curious consumers.

Smith's dossier had the same kind of errors that Pierce reported. His file also suggested a manual search of Texas court records was required, and listed him as connected to 30 businesses which he knew nothing about.

Some of the mistakes on Smith's report were comical: That his wife had a child three years before they were married, that he had been married previously to another woman, and most absurd, that he had died in 1976...

It's a longish article, and pretty depressing. The quality of the data is awful, it can control your destiny, you can't see it and you can't fix it. I would have been surprised if anything else were true. Most of my blogly bloviating is pure opinion; in this particular domain I have actual expertise (shock! It's even exotic expertise). Even if there weren't inevitable and severe matching errors associated with gathering data from multiple sources, the data would only be as good as its sources. Then there's the risk of the inevitable inferences that must be performed to process the data. Lastly, there's ChoicePoint's motivations. They don't get in trouble if they label someone a "child molester" who isn't -- how would anyone find out? They get in serious trouble if they mislabel a "child molester" as "clear".

Think about it. What are they more likely to do? Err on the side of labeling a good person as bad, or a bad person as good? Which error costs them money?

Add these four things together:

1. Fundamental problems related to "matching" identities managed in different systems.
2. Mismatch between the quality needs of the acquiring systems vs. the use to which the data is put by ChoicePoint.
3. Semantic issues too complex to mention here.
4. The intense motivation to err on the dark side of life.

and it would be astounding of ChoicePoint records were not full of severe errors and prone to cause harm to the (relatively) innocent.

Alas, do Americans care? Not now they don't. They will one day.

PS. David Brin covered this topic in great depth many years ago. He wrote a book about it (The Transparent Society), but you get the main ideas here and (more recently) here. The Amazon reviews make interesting reading -- the best are pained admissions that Brin might be right.

Why going open source is not a trivial thing

Mitch Kapor's Weblog: Should Groove Have Gone Open Source?

If you work with software this is a quite important statement. Read it carefully and think about it. I believe these issues are also true if one wishes to deliver a "solution platform" rather than an application. I also have a prejudice that the same designs that enable an open source or platfoorm approach may also allow long term software growth and evolution -- but I have no proof points for that prejudice. In contrast, Kapor has lots of proof points:

... There are advantages to going open source as well as challenges. In some cases it may even be necessary to forestall a competitive threat, i.e., do it before it is done to you. When I see businesses whose strategies involve defending a class of business model which is simply going to be obsolete going forward, my heart sinks about all the wasted effort.

Caveat altert: In a transitional era like the one we are in now, it is notable that it's harder to convert a code base developed in a proprietary context to be open source than it is to start from scratch for the same reason renovating a house completely is harder than new construction. Trust me if you haven't been through this. I have. This is one of the reasons it took seven years from the day Netscape announced it was going to open source the Mozilla browser to get to Firefox 1.0.

It typically requires a complete overhaul of the code and the development process, which is much harder than starting from scratch. Typically, the existing code base is not one which is amenable to community development. There is major code re-factoring and rewriting to be done, rethinking and reworking of API's, switching to open standards, and changing of the tool set to use transparent, community-oriented tools for source code management, issue and bug tracking, build status, knowledge base, and synchronous messaging.

The Lefkow murders, mental illness, and jumping to conclusions

ABC News: Man Claims to Have Slain Judge's Family

Judge Lefkow's husband and mother were murdered by a mentally ill man unrelated to the neo-Nazi groups that had threatened her in the past.

I give credit to the police and commentators for not jumping to conclusions about the neo-Nazi connection. The articles I read, in newspapers and blogs, were generally fairly cautious. Hale and his kin were obvious suspects, but both the police and the commentators were clear that there were many possible alternatives.

I haven't seen, however, much discussion of the mental illness aspects of this case. Bart Ross sued his care providers because of what sounds like some fixed beliefs (delusions) about his care. He lived alone and had few social contacts. His suicide note suggests a very troubled and guilt-racked person, not the typical sociopath.

I hope the mainstream newspapers will delve into the deeper story, the important story. Was Bart Ross paranoid schizophrenic, perhaps with some superimposed dementia? What kind of care did he receive for his illness? What is the best way to care for mentally ill adults, particularly paranoid schizophrenics?

Paranoid schizophrenia is a terrible disease that inflicts great suffering on both ill persons and their families. It's also associated with violent and irrational acts. We need to understand it much better, and we need to treat it far, far better than we do. The mainstream newspapers feel neglected and unwanted these days, but this is the kind of story they can and should do well. Perhaps Judge Lefkow, once she has time to grieve, will use her story to avert future tragedies.

GOP theology: a return to basics

Salon.com | The gospel of the rich and powerful

Joe Conason complains about the Bush agenda. I do too, but there's not much new to say. We fought hard because we knew the stakes. We lost.

His summary of GOP theology, however, is interesting. I've been saying the same thing for a while, but Conason is a writer and I'm a hack (emphases mine):

... Appalling as these policies may be, however, they are in no sense inconsistent with the cosmology of the religious right, which melds laissez-faire economics with fundamentalist orthodoxy. Underlying these conservative attacks on the poor by professing Christians is a worldview that dates back to earlier centuries, when the church defended privilege and declared that the wealthy and powerful were God's elect. From that perspective, minimum wages, subsidized health care, and other such laws and regulations only corrupt the poor, who must earn charity by their temporal and spiritual submission.

If these ideas sound a bit old-fashioned -- or even primitive -- be assured that they represent the latest thinking on the evangelical far right, which is where "compassionate conservatism" originated. Guided by the most literal interpretation of Old Testament law, the preachers who have influenced the President are determined to undermine every modern protection enjoyed by poor and working-class Americans. Let's hope they draw the line at bringing back public whippings and debt slavery.

Conason expresses these core ideas far more succinctly that I have. The premise that wealth and power is a sign of God's blessing predates medeival christianity, it predates monontheism, heck, it probably predates language. It is the oldest and the most powerful theology. It is resurgent in America and it is the theology of George Bush.

Given human nature, it's not surprising that the old ways are back. What's odd is that there ever were a few powerful people who actually tried to follow the inhuman, illogical and unattainable teachings of Christ. What's peculiar is that the Book of Job hasn't been deleted from the Old Testament.

By comparision the theology that wealth is virtue, and that poverty and disease are a sign of God's virtue, makes perfect sense. It's good to return to a sane world where the powerful behave as they ought to.

Faughnan's Notes needs a new home - Blogger doom

Faughnan's Notes

I've given up on Google's Blogger/Blogspot. They've had periods of reliability over the past four months, but the rule has been unreliable and mediocre service. The past week has been quite dismal.

I'm looking for a new host. I expect to pay; I've never been a fan of "free" services and Blogger has reinforced my prejudices. (BTW, I think it's fascinating that even while Microsoft's desire to sell leased software has been frustrated web apps have been able to deliver the same result from a different direction.)

If you have recommendations for a host or software solution please email me at jfaughnan@spamcop.net. I'm leaning towards installing a package on my current web hosting solution (lunarpages - faughnan.com).

Patrick Leahy and privacy

Senator predicts 'overdue' changes to privacy | CNET News.com

WASHINGTON--Recent data mishaps at ChoicePoint and Reed Elsevier Group's LexisNexis service could usher in a dramatic reshaping of privacy laws, a U.S. senator predicted.

Vermont's Patrick Leahy, the top Democrat on the Judiciary committee, said Wednesday evening that a recent slew of data thefts and other leaks requires a 'comprehensive rethinking' of the laws regulating companies that compile electronic dossiers on Americans that are typically purchased by creditors, employers or police.

Howard Dean. Patrick Leahy. I want to live in Vermont.

If Paul Wellstone had not died in a plane crash prior the 2002 senate race, then Norm Coleman wouldn't be Minnesota's (wretched, awful) senator and Patrick Leahy might chair the judiciary committee. Then we'd be a happier and healthier nation.

The one bright spot is that there's always been a fair interest in privacy in conservative America. Of course this interest was stronger when Clinton was president, but it's not entirely gone. I'm not optimistic however. There just hasn't been much real outrage about the Lexis/Nexis and ChoicePoint security breaches, or the flaws in the ChoicePoint profiles. As far as I can tell, the American people don't really care.

Remember Argentina: The US trade gap and the charity of strangers

BBC NEWS | Business | US trade gap expands to $58.4bn

...The trade deficit makes up much of the red ink in the US's current account [deficit], the measure of the divergence between the US's incomings and outgoings.

Both the current account and budget deficits - the latter currently close to $500bn - are funded largely by the purchase of dollar-denominated debt by foreign central banks.

The inflow of about $2bn a day, estimated by some economists to be about 80% of the world's excess savings, also keeps the dollar from sliding further and faster.

We Americans keep spending, you foreigners keep sending us money so we can spend. What a deal! I think I'll buy another Mac. Or maybe a Mercedes.

Meanwhile Manhattan, previously sold to the Japanese and the Saudis, is now being sold to the Italians. Somehow I suspect New Yorkers came ahead on each sale.

We Yanks are partying like the boom and bust never happened. We don't pay taxes, and foreigners pay for our government's services and invasions. We buy Armani and foreigners keep our credit card rates low. Those bogeyman bond traders who are supposed to threaten irresponsible nations just wink at us. Greenspan keeps the taps runnin' and foreign money flows out. Yeah, our homes cost bazillions now, but that's not inflation, that's wealth. Right?

Ok, I'm no economist. Modern economics reminds me of modern physics -- both have far outstripped my dwindling intellectual resources. The universe is an infinite pile of infinitely expanding grapefruits, and the flow of capital between the US, China and India means we can party forever.

All the same, I'd be interested in an investment strategy that allowed me to make a goodly bet that the house of cards collapses before 2010. I'm willing to lose the bet (but not all my assets!) if the game goes on. Anyone know a legitimate place that offers such wagers to non-billionaires? (There's a name for this kind of complex financial instrument but I need more coffee to remember it. [1])

[1] Google came to my rescue. The search on "complex financial instrument" provided a nice list from DeloitteLearning (course costs money and is IE only!). The term is "Derivative and it takes one deep into the bowels of finance -- a field where mathematicians have gone to play. So all I need is a packaged derivative that won't wipe me out if it goes south, but lets me bet that the wheels come off the train.