Digital Rights Management: letting the snake in the door

via MacInTouch: Andrew Orlowski (The Register) writes about Apple tightening the iTunes DRM noose:

While Apple has been in the news again this week for its war against the people who promote its products, another of its wars has received much less attention. It may as well be a covert war.

Bit by bit, Apple is tightening the DRM noose, reducing the amount of freedom its own customers enjoy. Last year, the company cut the number of times users could burn a playlist from ten to seven. This time, Apple has chosen to cripple one of its coolest and most socially beneficial technologies - Rendezvous.

Apple actually applied the restriction two months ago, but the passage of time hasn't made it any sweeter. In iTunes, Rendezvous allows users on the same subnet to share their music - although this is limited to streaming only. But the most recent version of iTunes, 4.7.1, restricts that streaming capability even further, and users aren't happy, as this support discussion shows. It used to support five simultaneous listeners, but now iTunes only permits five listeners a day.

I don't think these are new lessons, but they are not easy to remember or apply. Apple's Fair Play DRM technology was introduced with a relatively large amount of freedom. As Apple's distribution scheme has gained market share, Apple has consistently revised its software to reduce the freedom of Apple's customers to use their software. Since Apple controls the software (iTunes), hardware (iPod), DRM standard (FairPlay), and distribution channel (iTunes store) they have unlimited power to control what their customers can do with their music. (Don't want to upgrade your copy of iTunes? Hah. It will be trivially easy to move you along.)

This is why I don't buy anything from the Apple music store - despite my affection for my iPod and for iTunes. I buy standard, non-DRM protected CDs. I use AAC encoding -- an open standard (MPEG-4) that is supported by non-Apple devices.

We'll see how this all plays out. The more we see how the market is deploying DRM the more we may come to love the pirates of the PRC.

If anyone still thinks Apple has some peculiar virtues I hope they are fully disillusioned. I'd absolutely choose Bill Gates to rule the computing world over Steve Jobs; if Apple can stay at a 5% market share I'm likely to remain an Apple customer. My great fear is that somehow Apple will hit a 10% market share -- at that point they'll be quite intolerable.

Is Bill Gates a creationist?

Science & Technology at Scientific American.com: Questions That Plague Physics: A Conversation with Lawrence M. Krauss -- [ COSMOLOGY ] -- Lawrence M. Krauss speaks about unfinished business

Lawrence Krauss, a theoretical physicist, is a champion of the Enlightenment. In an interview with Scientific American he mentions something both disturbing and interesting.

...I'm not against teaching faith-based ideas in religion classes; I'm just against teaching them as if they were science. And it disturbs me when someone like Bill Gates, whose philanthropy I otherwise admire, helps finance one of the major promoters of intelligent design by giving money to a largely conservative think tank called the Discovery Institute. Yes, they got a recent grant from the Gates Foundation. It's true that the almost $10-million grant, which is the second they received from Gates, doesn't support intelligent design, but it does add credibility to a group whose goals and activities are, based on my experiences with them, intellectually suspect. During the science standards debate in Ohio, institute operatives constantly tried to suggest that there was controversy about evolution where there wasn't and framed the debate in terms of a fairness issue, which it isn't. [Editors' note: Amy Low, a media relations officer representing the Bill and Melinda Gates Foundation, says that the foundation 'has decided not to respond to Dr. Krauss's comments.']

One grant could be a mistake. Two grants is design. I'd gotten rumblings that among some middle-aged west coast geeks there's been a resurgent interest in 'intelligent design'. George Gilder might be the tip of the iceberg.

These are very smart and wealthy people, but they are not often scientists. If they have a technical background, it's not in biology. They are middle-aged now and feeling definitively mortal. Perhaps this is not surprising.

This will be interesting.

Yoo-hoo? Mainstream journalists? All you folks fretting about your role and relevance? How about looking into the Gates donations to the Discovery Institute?

Tsunami and the Nicobar islanders

The Tsunamis is "ancient" history now, but in researching the alleged near- Homo sapiens sapiens extinction event of the Tabo eruption I came across a web site describing the peoples of the Nicobar islands -- a group that colonized that area shortly after the Tabo eruption. The Negrito in particular represent a very ancient "race". The web site has news of how these people survived the Tsunami, but in at least one case rescue efforts have caused more pain than the Tsunami itself:

05-mar: ...While the Tsunami was in itself a traumatic experience for most, what has followed is even more traumatic. This is true especially for the inhabitants of Chowra, Bompoka and Trinket, the three islands in the Central Nicobars that were badly damaged and are now seen as unfit for habitation.

Chowra, a small flat island with a high population density and limited natural resources, had a pre-Tsunami population of 1,464, now reduced to 1,408. The relatively low number of casualties was completely unexpected. In view of the Tsunami's behaviour in Car Nicobar, the complete destruction of Chowra was initially feared. However, when rescue operations ended on Chowra on 4th January, it was found that 'only' 56 had died. No, the waves had not spared the rest of the population: most of them had been washed out to sea. But the Chowrites, sea people that they are, managed to fight the gigantic waves and swim back to land. The Chowrites are no ordinary people. Up to now, they are the only group in the Nicobars that have maintained a strong cultural identity despite outside interventions. It is known throughout the archipelago that the Chowrites have remained opposed to we1fare and development programs promoted by the Administration until recently. To the Chowrites, their identity and culture, inextricably linked to their land, are endowed with magic and has been foremost in all contact with the outside world.

For the first time, the people of Chowra have now been separated from their land. Following the rescue operations, all inhabitants of Chowra and Bompoka were moved to relief camps on Teressa island. Having spent two unhappy months there, they now wish to return to their islands and start a new life. But, unfortunately, they are not allowed to do so. In a meeting that was organized by the Administration on Teressa in early February, discussions were held with the Chief Captain of Chowra, Jonathan. The Administration urged Jonathan to stay in their new home on Teressa and tend to their plantations on Chowra, at least during the temporary rehabilitation phase. The main argument put forth by the Administration was the lack of water on Chowra. Jonathan failed to understand why this should be an issue after the Tsunami. Chowra had always faced water scarcity, no solution to which had ever been found or sought by the Administration, so why should this suddenly be an issue now? Jonathan expects nothing from the Administration. Why should he? He said that the desalinization plant that had been set up at one point worked only for a short while. It never got repaired or serviced when it broke down. His people had managed well in the past and they could do so now. Two weeks after that meeting, Jonathan, in a letter, still begged for boats 'to return to Chowra for at least 10 days to collect our left belongings ... before the [southwest] winds, because then there will be many problems once this wind starts. We have to reach Chowra before that'. When an official assessment team visited Chowra for the first time on 28th February, it was surprised to find that conditions on Chowra were much better than in most other villages/islands. Unfortunately, it had taken 2 months to realize this...

On human extinctions

BBC NEWS | Science/Nature | Experts weigh supervolcano risks

The BBC is doing a show on super volcanoes. These are supposed to happen every 100,000 years or so. They're as devastating as a really major meteor impact but five times more common. On the other hand, we think we might be able to avert a meteor strike. Volanoes ... well, probably not much to do there.

This quote, however, really suprised me:

One past super-eruption struck at Toba in Sumatra 74,000 years ago and is thought by some to have driven the human race to the edge of extinction. Signs from DNA suggest human numbers could have dropped to about 10,000, probably as a result of the effects of climate change."

Huh? I recall the 10K number from estimates of the origins of Homo sapiens sapiens, but that's 250K years ago. This is the first I've heard that we just about went down the tubes a mere 74K years ago. I think Home Floriensis and Homo sapiens neanderthalis both survived that period. Neaderthal man might have been better adopted to cold than other humans; I wonder if they would have flourished after the eruption.

Anyway, I want to learn more.

The ten commandments case brings Scalia out of the closet

Brad DeLong's Website: Nino Scalia, by Grace of God Justice and Lord

DeLong on Scalia:

Nino Scalia is allowed to break with those like Jefferson, Madison, and Lincoln who think that legitimate power ascends from the consent of the people. It's a free country. He can take his stand with those like James I Stuart, Innocent III, and Khomeini who think that legitimate power descends from God.

But does such a guy have any business being a Justice of the Supreme Court of a free country? No.

The Ten Commandments case is forcing a lot of things out into the light. However it turns out, we'll learn a lot about America. The principle of separation of Church and State may be about to take a hard fall.

Good.

We need to concentrate some minds.

Undoing the Enlightenment: Divide First, Conquer Second

In an earlier post I digressed from some Senator Rick Santorum language associated with the No Child Left Behind legislation to a broader discussion of the Orwellian language used by creationists (now repacked as 'intelligent designists").

Upon discussion with my Muse (Emily), the diabolical strategy of the counter-Enlightenment agenda was revealed to me (how's that for theological language?). It is a strategy that Machiavelli, a scion of the Englightenment, would have most appreciated. For that matter, Sun Tzu would also approve. Consider this paragraph in a National Center for Science Education essay:

Santorum language appeared in the 2003 Kansas legislative session in the form of Senate Bill 168. SB 168 encouraged curricula that helped students understand "the full range of scientific views that exist," and exempted educational staff from any penalties for deviating from state curriculum requirements. SB 168 further betrayed its creationist leanings when it explicitly singled out "origins science" for special treatment, requiring that "origins science" -- but not other science -- be taught "inclusively, objectively, and without religious, naturalistic or philosophic bias or assumption." The artificial distinction between "origins science" (sciences dealing with the past) and "operations science" (sciences dealing with the present) has been a running theme in creationist publications for decades.

If one wished to undo the Enlightenment, how would one begin? One would not strike at strength, but at weakness. First divide, then conquer. "Operations Science" includes physiology, biochemistry, bacteriology, molecular biology, meteorology, chemistry, applied physics and far more. This is strong science. (Note, however, that medicine, a profession related to some of the strong sciences, has been shown to be vulnerable to attack.

On the other hand "origins science" is less immediate to the everyday life of the average person. Origins science is concerned with history, with fossils, with cosmology, and with evolution. The average citizen would feel that their life is not affected one bit by the study of dinosaurs, of quasars, of Neandertals, of the Aztecs, or of natural selection. This is vulnerable science. This is where one would start undoing the Englightenment.

Children of the Enlightenment -- arouse yourselves!

On spam, marketing, noise, search, reputation management and the evolution of the blog

Need an Outlook reference?

I was looking for a book on Outlook 2003. I don't need a novice reference, I need a power-user reference. I looked first at O'Reilly (the home of the power user), but they don't have anything current. Google and Amazon weren't helping, though Amazon was somewhat useful.

So I turned to an authority, a blog called Marc's Outlook on Productivity. Marc is not a super-power user, but he concentrates on this domain.

This is new. I could ask a friend, but I already know more about Outlook than anyone else I know. Five to ten years ago I'd have tried Usenet, but it's been done in by spam and blogs. Five years ago Google would have given me a good answer, now Google is overwhelmed by spam and marketing. Amazon usually works, but there aren't enough people interested in this topic to provide me with useful information. This isn't surprising, Amazon is often weak in dealing with multiple editions of computer books with a small readership -- there's too much dilution of a limited number of reviews. Once upon a time I'd have gone to a 'community site', but they have too much noise and are too hard to track.

So I took a new tack - the next best thing to asking an expert colleague for a recommendation. I went to the blog of an expert I trusted. I trusted this expert because I've followed his writing for a year or so. (Note: blog, expert, domain specific, reputation->trust)

The Post Categories in this blog (Note: ontology, terminology, classification, search, metadata) helped me find a relevant post very quickly. I ordered the book from Amazon. Unfortunately the publisher doesn't allow one to view the contents in Amazon (most annoying).

The pace of evolution of the web and of search in general is breathtaking. In ten years we've marched through an astonishing array of solutions, most of which have been destroyed by the twin demons of marketing and spam (arguably spam is a form of marketing!).

I wonder if the idea of personal authority and reputation will be the persistent solution. In all these years it's been a common thread. Attempt to automate (most nobly by Google) have fallen to the corrupting forces of marketing. Sure, people can be corrupted, but I've found that there's a goodly number of people that are surprisingly resistant to such influences. If they demonstrate this resistance over time, they become influential. (Then the corrupting forces can become irresistible, but there are always replacements .. and so it goes ...). Blogs, especially when they are authored by a single person, are the only way I know of in today's net to create an identify, a reputation, and finally, trust in domain recommendations.
am (noise). Interesting to consider what the biological equivalence is.

It's interesting to consider what the biological equivalence is.

PS. Note to marketing folk -- much of my professional life falls in the broad domain of marketing. Marketing is a powerful domain that rules much of modern life -- like all powers it has two faces.

Update 4/25/05: So, what was the book like? Not bad, but of the 1000 or so pages probably 30 are useful for me. People like me are not common enough to base a book on, so I guess I have to make do with a chapter her or there.

No Child Left Behind advocates teaching creationism?

Battle on Teaching Evolution Sharpens (washingtonpost.com)

The No Child Left Behind Law contains a language that espouses the teaching of creationism in science classes [see update below for a partial correction]:

Some evolution opponents are trying to use Bush's No Child Left Behind law, saying it creates an opening for states to set new teaching standards. Sen. Rick Santorum (R-Pa.), a Christian who draws on Discovery Institute material, drafted language accompanying the law that said students should be exposed to 'the full range of scientific views that exist.'

'Anyone who expresses anything other than the dominant worldview is shunned and booted from the academy,' Santorum said in an interview. 'My reading of the science is there's a legitimate debate. My feeling is let the debate be had.'

In today's Orwellian world "full range of scientific views" is a codephrase for "intelligent design" which is a code phrase for "creationism".

If Bush and his ilk mandated a mandatory Yahweh- only religion class in public schools they'd at least win points for honesty. Alas, they're too cowardly for that.

I think I'll start advocating the teach of chiropractic theories as an alternative to traditional human physiology. There's at least as much "legitimate scientific debate" around chiropracty as there is around the fundamental value of natural selection. (ie. zero in both cases).

Update 3/15: Some more on the Santorum Amendment from an authoritative source discussing the assault on reason taking place in Kansas -- ground zero for the counter-enlightenment. The Santorum amendment was stripped from the NCLB final legislation but remains in the conference report. It is this tactic that's being repeated at every level of government. (Emphases mine, note the full range of Orwellian strategies and neologisms -- George is spinning now ...)

According to the Lawrence Journal-World, an antievolution resolution was introduced in the Kansas House of Representatives on February 15, 2005. The sponsor is Representative Mary Pilcher-Cook (R-Shawnee), who said that the proposed resolution, which is nonbinding, was meant to promote "objectivity in science education."

Although the full text of the bill is not yet available, the story reports that the resolution includes language recommending the teaching of "the full range of scientific views that exist." This language is derived directly from the "Santorum Amendment," which U.S. Senator Rick Santorum (R-PA) attempted to insert into the federal No Child Left Behind Act in 2002. Phillip Johnson, a leading promoter of "intelligent design," wrote the amendment for Santorum. The Santorum Amendment was passed by the U.S. Senate but stripped from the bill by the House-Senate conference committee, and now only appears in modified form in the NCLB conference report. See NCSE's compilation on the Santorum Amendment for details.

Santorum language appeared in the 2003 Kansas legislative session in the form of Senate Bill 168. SB 168 encouraged curricula that helped students understand "the full range of scientific views that exist," and exempted educational staff from any penalties for deviating from state curriculum requirements. SB 168 further betrayed its creationist leanings when it explicitly singled out "origins science" for special treatment, requiring that "origins science" -- but not other science -- be taught "inclusively, objectively, and without religious, naturalistic or philosophic bias or assumption." The artificial distinction between "origins science" (sciences dealing with the past) and "operations science" (sciences dealing with the present) has been a running theme in creationist publications for decades.

On February 6, 2005, nine days before Pilcher-Cook introduced her antievolution resolution, an op-ed promoting the Santorum language appeared in the Kansas City Star. The opinion piece, written by Kansas City resident Vicki Palatas, was entitled "'Full range of scientific views' includes theory of a creator." Palatas wrote, "The report interpreting this legislation explains that on controversial issues like evolution, 'the curriculum should help students to understand the full range of scientific views that exist.'" Palatas cited several talking points popular among intelligent design proponents, and concluded, "Intelligent design teaches the theory of a creator based on scientific observation and analysis, not the worship of one." In another opinion piece on January 2, 2005, Palatas explicitly grouped intelligent design with a laundry list of other conservative religious causes, decrying the failure of public schools to "allow teaching intelligent design as a theory of the creation of the universe." She concluded that governmental restraint in this and other matters amounted to "intolerance and maligning of our faith."

Passed by the Senate, but stripped in a committee?? Wow, American politics is interesting - especially considering that the House is famously conservative.

The National Center for Science Education essay is horrifying and fascinating. Note the contradictory themes of the creationists who can speak both "intolerance of faith" and a "theory of a creator". Or perhaps they are not so inconsistent -- for it is their faith that assures them that the "analysis of the creator" will give them the answer they expect. Historians of science might tell them that such analyses often give quite surprising answers. If we do develop methods to test for analyze "deities", Palatas and her ilk may yet regret their enthusiasms.

More practically, the lessons of the past few decades is that extremists are very, very persistent. They love the struggle itself. Rationalists fatigue, they move on, they have a life (yes, even I have a life -- actually, rather a lot of one). In our era the dominant extremists in America are right wing conservatives -- I think they'll win in Kansas. Science is going to take a serious beating for years to come.

Hmm. The enlightenment is beginnning to feel pretty anomalous. I need to consult a historian.

Race as a collection of genes that travel together

The New York Times > Opinion > Op-Ed Contributor: A Family Tree in Every Gene

The hypothesis is that genes tend to travel together, and that it's possible to assign a human being living today to a geographically isolated population in which a characteristic collection of genes was very common. That assignment is a "race".

This is a statistical model of race. Imagine a 'gene-space' consisting of (say) 100 or so marker gene values. If we treat this as a 100-dimension space then an individual human should appear as a point in this space. If we then add a dimension for frequency, we may "see" hills and valleys on this "surface". Those are "races". Most of us are somewhere on the flank of a mountain, but there ought to be (how can one resist the word?) "pure" folk at the peaks. Conversely we ought to be able to find folks living in "valleys" who are truly "unique". (Some say I'm "different", but I think they have something else in mind.)

Here's how Leroi puts it:

The New York Times March 14, 2005
A Family Tree in Every Gene
By ARMAND MARIE LEROI

... If modern anthropologists mention the concept of race, it is invariably only to warn against and dismiss it. Likewise many geneticists. "Race is social concept, not a scientific one," according to Dr. Craig Venter - and he should know, since he was first to sequence the human genome. The idea that human races are only social constructs has been the consensus for at least 30 years.

The dominance of the social construct theory can be traced to a 1972 article by Dr. Richard Lewontin, a Harvard geneticist, who wrote that most human genetic variation can be found within any given "race." If one looked at genes rather than faces, he claimed, the difference between an African and a European would be scarcely greater than the difference between any two Europeans...

Three decades later, it seems that Dr. Lewontin's facts were correct, and have been abundantly confirmed by ever better techniques of detecting genetic variety. His reasoning, however, was wrong...

.... The shapes of our eyes, noses and skulls; the color of our eyes and our hair; the heaviness, height and hairiness of our bodies are all, individually, poor guides to ancestry.

But this is not true when the features are taken together. Certain skin colors tend to go with certain kinds of eyes, noses, skulls and bodies... To put it more abstractly, human physical variation is correlated; and correlations contain information.

Genetic variants that aren't written on our faces, but that can be detected only in the genome, show similar correlations. It is these correlations that Dr. Lewontin seems to have ignored. In essence, he looked at one gene at a time and failed to see races. But if many - a few hundred - variable genes are considered simultaneously, then it is very easy to do so...

... Study enough genes in enough people and one could sort the world's population into 10, 100, perhaps 1,000 groups, each located somewhere on the map. This has not yet been done with any precision, but it will be. Soon it may be possible to identify your ancestors not merely as African or European, but Ibo or Yoruba, perhaps even Celt or Castilian, or all of the above.

... Hispanics, for example, are composed of a recent and evolving blend of European, American Indian and African genes, then the Uighurs of Central Asia can be seen as a 3,000-year-old mix of West European and East Asian genes.

... When the Times of India article referred to the Andaman Islanders as being of ancient Negrito racial stock, the terminology was correct. Negrito is the name given by anthropologists to a people who once lived throughout Southeast Asia. They are very small, very dark, and have peppercorn hair. They look like African pygmies who have wandered away from Congo's jungles to take up life on a tropical isle. But they are not.

The latest genetic data suggest that the Negritos are descended from the first modern humans to have invaded Asia, some 100,000 years ago. In time they were overrun or absorbed by waves of Neolithic agriculturalists, and later nearly wiped out by British, Spanish and Indian colonialists. Now they are confined to the Malay Peninsula, a few islands in the Philippines and the Andamans...

The full article tries to justify race identification as a way to improve healthcare. I'm skeptical. Maybe as an interim approach, but we'll do better with pharmacogenomics than race as a proxy for individual gene values. I'd call this an interesting hypothesis rather than something that's immediately useful.

China will pay - the bennies of empire

Recently I predicted (hey, I'm not alone!) that our party-days are numbered, that foreigners won't continue to fund our profligate ways. Nonsense, says Niall Ferguson. China will keep the taps open much longer for the US than for, say, Argentina. It's all the wages of empire, the ROI for our massive military and young warriors ...

The New York Times > Magazine > Niall Ferguson> The Way We Live Now: Our Currency, Your Problem

Every congressman knows that the United States currently runs large ''twin deficits'' on its budget and current accounts. Deficit 1, as we well know, is just the difference between federal tax revenues and expenditures. Deficit 2 is generally less well understood: it's the difference between all that Americans earn from foreigners (mainly from exports, services and investments abroad) and all that they pay out to foreigners (for imports, services and loans). When [only] a government runs a deficit, it can tap public savings by selling bonds. But when the economy as a whole is running a deficit -- when American households are saving next to nothing of their disposable income -- there is no option but to borrow abroad.

There was a time when foreign investors were ready and willing to finance the U.S. current account deficit by buying large pieces of corporate America. But that's not the case today. Perhaps the most amazing economic fact of our time is that between 70 and 80 percent of the American economy's vast and continuing borrowing requirement is being met by foreign (mainly Asian) central banks.

Let's translate that into political terms. In effect, the Bush administration's combination of tax cuts for the Republican ''base'' and a Global War on Terror is being financed with a multibillion dollar overdraft facility at the People's Bank of China. Without East Asia, your mortgage might well be costing you more. The toys you buy for your kids certainly would.


Why are the Chinese monetary authorities so willing to underwrite American profligacy? Not out of altruism. The principal reason is that if they don't keep on buying dollars and dollar-based securities as fast as the Federal Reserve and the U.S. Treasury can print them, the dollar could slide substantially against the Chinese renminbi, much as it has declined against the euro over the past three years. Knowing the importance of the U.S. market to their export industries, the Chinese authorities dread such a dollar slide...

... there are three good reasons that the administration is tacitly delighted by the Asian central banks' support. Not only is it keeping the lid on the price of American imports from Asia (a potential source of inflationary pressure). It is also propping up the price of U.S. Treasury bonds; this in turns depresses the yield on those bonds, allowing the federal government to borrow at historically very low rates of interest. Reason No. 3 is that low long-term interest rates keep the Bush recovery jogging along.

Sadly, according to a growing number of eminent economists, this arrangement simply cannot last. The dollar pessimists argue that the Asian central banks are already dangerously overexposed both to the dollar and the U.S. bond market..

... Though neither side wants to admit it, today's Sino-American economic relationship has an imperial character. Empires, remember, traditionally collect ''tributes'' from subject peoples. That is how their costs -- in terms of blood and treasure -- can best be justified to the populace back in the imperial capital. Today's ''tribute'' is effectively paid to the American empire by China and other East Asian economies in the form of underpriced exports and low-interest, high-risk loans.

How long can the Chinese go on financing America's twin deficits? The answer may be a lot longer than the dollar pessimists expect. After all, this form of tribute is much less humiliating than those exacted by the last Anglophone empire, which occupied China's best ports and took over the country's customs system (partly in order to flood the country with Indian opium). There was no obvious upside to that arrangement for the Chinese; the growth rate of per capita G.D.P. was probably negative in that era, compared with 8 or 9 percent a year since 1990.

Meanwhile, the United States may be discovering what the British found in their imperial heyday. If you are a truly powerful empire, you can borrow a lot of money at surprisingly reasonable rates. Today's deficits are in fact dwarfed in relative terms by the amounts the British borrowed to finance their Global War on (French) Terror between 1793 and 1815. Yet British long-term rates in that era averaged just 4.77 percent, and the pound's exchange rate was restored to its prewar level within a few years of peace.

An intelligent discussion of the Italian hostage (Sgrena) shooting

The New York Times > Opinion > Op-Ed Contributor: Manning the Barricades

The author of this Op-Ed piece is a former Marine captain. He discusses the Sgrena shooting. He rapidly dispenses of the absurd claim that Sgrena's car was specifically targeted (if so, why is she alive?). He also points out that arguments about prior notification are irrelevant. Most of all, he had informed ideas on what to do differently. Of course I also like the recommendations because they're what I thought of myself after a the young children of accidentally killed Iraqis were famously photographed covered with blood.

... Unfortunately, instead of helping to answer that question, the uproar after the shooting has focused on two distractions. From her hospital bed, Ms. Sgrena hinted that the Americans had tried to kill her to protest Italy's policy of negotiating with hostage-takers. Her assertion begs the questions of what the United States could possibly gain from such an act and, why, after approaching her car, the soldiers apologized and called for medical help rather than finishing the job.

More dangerous, because it sounds more plausible, is the claim that proper coordination between Italian and American authorities could have prevented the shooting. Gen. George W. Casey Jr. , the top American commander in Iraq, said Italian officials gave no advance notice of the car's intended route. Prime Minister Silvio Berlusconi disagrees. This dispute is a red herring. No high-level government coordination, short of an American military escort for Ms. Sgrena's car, would have changed the outcome on that highway. The pivotal players were the men on the ground.

A hallmark of modern warfare is what the Marine Corps calls the "strategic corporal." The immense firepower of our troops, the haphazard nature of the Iraqi insurgency and the ever-watching eyes of the global news media combine to place decisions of strategic consequence on the shoulders of the junior-most troops. Consider the videotaped shooting of a wounded insurgent by a marine during the fight over Falluja in November, or the atrocities committed by soldiers at Abu Ghraib referred to by some in the military as "the seven idiots that lost the war." The training provided to young marines and soldiers must be commensurate with the extraordinary demands we now make of them.

The fact is, checkpoint techniques can be taught. My platoon had to learn them on the fly, but that was two years ago. The lessons we and other troops learned should have been institutionalized long ago.

For example, we tried and discarded the three tactics that were used to warn the Italians as they approached the checkpoint: hand and arm signals, warning shots and shooting into the vehicle's engine block. We found that hand and arm signals were tough to decipher, and subject to different cultural interpretations. Warning shots are hard to hear or see, and frequently only panic the driver they're intended to warn. Shooting into engine blocks to avoid injuring passengers is Hollywood fantasy. Even my Marine snipers - some of the best marksmen in the world - couldn't do it consistently.

So we adapted. For example, once while driving through a town, we cut down a traffic sign - a bright, red octagon with the word "stop" written in Arabic - and used it at checkpoints. Who knows how many lives this simple act of theft may have saved? We also learned to shoot off highly visible smoke grenades and brightly colored flares when possible threats approached. We started putting our concertina wire at least two football fields away to give us more reaction time.

Every combat unit learns its own lessons from hard experience. The important thing is that they be passed on so they are not continually relearned at the cost of innocent lives. Americans must understand that tragic mistakes in war are unavoidable, but that every legal, moral and strategic imperative demands that they be kept to a minimum. This is our obligation to Ms. Sgrena and to Mr. Calipari's family, to the thousands of Iraqi civilians who pass through military checkpoints each day, and to the Americans who must man them and live with their decisions.

David Brin has a blog

Contrary Brin

David is one of the smartest and most interesting thinkers I know of. I was surprised to see he has a blog. I'll have to read through the archives, he started about 5 months ago. Poor guy is using Blogger (as am I). Ouch.

PLATO Notes, Microsoft Groove, and the curious history of software

This is a bit more software-centric than my usual 'Notes' postings, but it's really not about a particular technical issues, rather it's an interesting and topical ancectdote about how software evolves. Once upon a time I thought software largely came from the imagination of a few people. Sometimes it does (for better or worse), but most complex software projects have a long and often unrecognized legacy.

Groove is in the news today, it's a software solution recently acquired by Microsoft. Ray Ozzie, the CEO of Groove, will become Microsoft's Chief Technologist. Microsoft's involvement has created the recent interest in this "new" software.

Groove seems new, but it's been in development for at least six years. It's not six years old, however, because it's an offspring of Lotus Notes, which was developed in the 1980s. But it's not twenty years old, because it's really a descendant of PLATO Notes, which was developed at the University of Illinois in the 1970s atop the 1960s (1950s?) PLATO platform. So it's thirty years old. Heck, one could argue that it's really a child of the Memex (1945), so it's about sixty years old.

This is what Kapor of Lotus/spreadsheet fame wrote about the connection of Groove to PLATO:

Mitch Kapor's Weblog: Microsoft Acquires Groove
Ray has been a colleague and friend for over 20 years. He came to Lotus is 1982 with the vision of Notes already in mind, having been inspired by the PLATO system he used as an undergraduate at the University of Illinois...

Kapor's posting led me to a Google search, and thus quickly to a history of PLATO Notes, a pre-PC system for communication and collaboration. The history is well worth reading for anyone who develops or works with complex software systems, or who is just interested in the history of ideas. There are lessons there about electronic community (10 million hours!), about open source development, about the software development process, about software evolution, about software-as-platform -- and more besides.

There are also some minor personal serendipities here. I am writing this on a blog, a modern version of the kind of collaborative community that PLATO pioneered. I live in Saint Paul, and PLATO Notes was commercialized by a Minneapolis company -- Control Data. I have worked with many Control Data veterans who no doubt have connections to the CDC PLATO team, but, in addition, I have a longstanding interest in collaborative software systems (warning: old web pages). About 8 years ago my interest led me to review several alternatives and to comment on the work of David Woolley and his web conferencing guide.

David Woolley, as a young man, created PLATO Notes in 1973; he wrote the article I mention above. David is also a leader at Minnesota e-Democracy, which I've long appreciated. I shall have to send him a note of appreciation.

Update 3/15: David Woolley corrected some errors I made in dates. Thanks David!

What's worse than no privacy? Lies.

A while back I posted on the (gross) errors that my shadow medical profile is likely accumulating thanks to a persistent billing error. I thought I was making a prediction, but tomorrow is today. The data stolen from ChoicePoint (and Lexis/Nexis and everywhere else) is full of errors:

MSNBC - ChoicePoint files found riddled with errors
By Bob Sullivan MSNBC

... Pierce, a privacy advocate, obtained her report nearly two years ago, long before the current controversy. Thanks to the unknown source -- perhaps a company employee, Pierce said, but she has no way of knowing -- she got a rare privilege most consumers don't: a chance to see what ChoicePoint knows about her.

... What first caught Pierce's eye, she said, was a heading titled "possible Texas criminal history." A short paragraph suggested additional, "manual" research, because three Texas court records had been found that might be connected to her. "A manual search on PIERCE D.S." is recommended, it said.

Pierce says she's only visited Texas twice briefly, and never had any trouble with the law there.

"But if I was applying for a job, and there were other candidates, and this was on my record, the company would obviously go for another person," she said. "It raises a question in your mind."

... On ChoicePoint's Web site, the National Comprehensive Report is described as a collection of searches that glean data from "national and state databases for a summary of assets, driver licenses, professional licenses, real property, vehicles, and more. Each report offers the ability to add associates to the report, which include relatives, others linked to the same addresses as the subject and neighbors."

... Under former addresses, an ex-boyfriend's address was listed. Pierce said she never lived there, and in fact, he moved into that house after they broke up. The report also listed three automobiles she never owned and three companies listed that she never owned or worked for.

Under the relatives section, her sister's ex-husband was listed. And there are seven other people listed as relatives who Pierce doesn't know...

...Most alarming to Pierce is the fact that, with all this information, the ChoicePoint report she received had glaring omissions, too. Many of her former addresses aren't listed; and despite the host of other people listed on her report, many relatives and nearby neighbors were missing.

... Pierce's experience neatly parallels that of Richard Smith, another privacy advocate, who paid a $20 fee and received a similar report from ChoicePoint several years ago. The company offers a wide variety of reports on individuals; Smith purchased a commercial version that's sold to curious consumers.

Smith's dossier had the same kind of errors that Pierce reported. His file also suggested a manual search of Texas court records was required, and listed him as connected to 30 businesses which he knew nothing about.

Some of the mistakes on Smith's report were comical: That his wife had a child three years before they were married, that he had been married previously to another woman, and most absurd, that he had died in 1976...

It's a longish article, and pretty depressing. The quality of the data is awful, it can control your destiny, you can't see it and you can't fix it. I would have been surprised if anything else were true. Most of my blogly bloviating is pure opinion; in this particular domain I have actual expertise (shock! It's even exotic expertise). Even if there weren't inevitable and severe matching errors associated with gathering data from multiple sources, the data would only be as good as its sources. Then there's the risk of the inevitable inferences that must be performed to process the data. Lastly, there's ChoicePoint's motivations. They don't get in trouble if they label someone a "child molester" who isn't -- how would anyone find out? They get in serious trouble if they mislabel a "child molester" as "clear".

Think about it. What are they more likely to do? Err on the side of labeling a good person as bad, or a bad person as good? Which error costs them money?

Add these four things together:

1. Fundamental problems related to "matching" identities managed in different systems.
2. Mismatch between the quality needs of the acquiring systems vs. the use to which the data is put by ChoicePoint.
3. Semantic issues too complex to mention here.
4. The intense motivation to err on the dark side of life.

and it would be astounding of ChoicePoint records were not full of severe errors and prone to cause harm to the (relatively) innocent.

Alas, do Americans care? Not now they don't. They will one day.

PS. David Brin covered this topic in great depth many years ago. He wrote a book about it (The Transparent Society), but you get the main ideas here and (more recently) here. The Amazon reviews make interesting reading -- the best are pained admissions that Brin might be right.

Why going open source is not a trivial thing

Mitch Kapor's Weblog: Should Groove Have Gone Open Source?

If you work with software this is a quite important statement. Read it carefully and think about it. I believe these issues are also true if one wishes to deliver a "solution platform" rather than an application. I also have a prejudice that the same designs that enable an open source or platfoorm approach may also allow long term software growth and evolution -- but I have no proof points for that prejudice. In contrast, Kapor has lots of proof points:

... There are advantages to going open source as well as challenges. In some cases it may even be necessary to forestall a competitive threat, i.e., do it before it is done to you. When I see businesses whose strategies involve defending a class of business model which is simply going to be obsolete going forward, my heart sinks about all the wasted effort.

Caveat altert: In a transitional era like the one we are in now, it is notable that it's harder to convert a code base developed in a proprietary context to be open source than it is to start from scratch for the same reason renovating a house completely is harder than new construction. Trust me if you haven't been through this. I have. This is one of the reasons it took seven years from the day Netscape announced it was going to open source the Mozilla browser to get to Firefox 1.0.

It typically requires a complete overhaul of the code and the development process, which is much harder than starting from scratch. Typically, the existing code base is not one which is amenable to community development. There is major code re-factoring and rewriting to be done, rethinking and reworking of API's, switching to open standards, and changing of the tool set to use transparent, community-oriented tools for source code management, issue and bug tracking, build status, knowledge base, and synchronous messaging.