Scary moment in the digital life: a corrupted image

I recently used iPhoto Library Manager to merge two iPhoto 5 libraries of several thousand images apiece. I did this in preparation for migration to iPhoto 6; IPLM has a longer track record with iP 5 than 6, so I chose to merge before upgrading.

Thus far it seems to have gone well, which is pretty impressive since #$!$! Apple doesn't support Library imports or merges in $!@#$!@ iPhoto. I'm still looking for problems and I'll report on my experience soon. I did get some malrotated images, but I expected that. OS X had some bad bugs with EXIF headers that Apple never admitted to but fixed; I expected older images to malrotate on import. I reverted to original and fixed them.

One image among thousands failed to import. IPLM produced a handy error message. I tracked down the image -- it was corrupt. Preview and GraphicConverter wouldn't render it and if I tried to open it in iPhoto the app locked up. (How stupid is that?)

Unnerving. I expect this image went bad years ago. Could have been a disk crash, could have been a rude copy of iPhoto, could have been a network copy glitch -- heck, it could have been cosmic rays. My regular backups only go back a few months, so they wouldn't help. I do have some CDs and DVDs that would probably cover this problem, but it would take a while to track those down. Fortunately I happened to have older versions of some iPhoto Libraries on my external drive, and I found a good copy in minutes.

This is exceedingly annoying. We are so far from having really good backup solutions. I depend on redundancy -- a primary automated network backup that runs daily and several different approaches to backup that are done irregularly. Odd are that if one method fails, the very different "backup backup" methods will work. It's the same principle applied to designing the space shuttle's control system.

Backups have saved me at least a dozen times over the past 20 years or so. They're very demanding to manage however.

The NSA affair: it's about how they selected their intercepts

Cringely's sources say pretty much exactly what I guessed last week (in part from his prior column, but also from connecting other dots) -- the technical aspects of the NSA affair are about social network analysis of phone metadata.

So Bush is right when he claims only a few calls are tapped, but what he doesn't talk about, and what reporters don't ask him, is how the NSA decides which phones to tap. It's the process of figuring out who to go after that's technically interesting, but the real story is about the ability of President to override Congress. That's a constitutional question that will go to the supreme court, and that's why Kerry and Kennedy are trying to filibuster Alito. I don't think they'll succeed, but I'm glad they're trying.

PBS | I, Cringely . January 26, 2006 - The Falafel Connection

... After last week's column, a number of readers wrote to explain that the National Security Agency's problem with complying with the Foreign Intelligence Surveillance Act (FISA) had to do with the sheer volume of wiretaps involved, which they guessed numbers in the millions or billions. Evidently, these worried readers think, the NSA has been long listening-in to ALL of our calls, and thought that might not go down well with the secret court that issues FISA warrants.

I don't think so.

The NSA has a very advanced program called Echelon for monitoring radio communication around the world, and probably intercepts a lot of phone calls that way, but for FISA-type wiretaps they tend to use the same outsourcing firms the phone companies use, and these generally tiny outfits can only handle a few thousand taps per year each.

By the way, if you are wondering whether YOUR phone could be easily tapped, just check to see if your phone company offers three-way dialing, because that's the feature we're talking about. If you can get it, they can get you. And if you are wondering whether VoIP service can't be tapped, the answer is both yes and no. For the moment, SIP services like Skype can't be tapped but that will change soon. And if you are a Vonage or Packet8 user, well they already have your number.

Here's what is most likely going on with the NSA and FISA from a guy who used to work for the NSA:

"What I think is going on here is that they're using social network analysis. They get some numbers or endpoints of interest, and start out with classical traffic analysis, which can all be done (as I think you pointed out) with pen registers or their moral equivalent. They look for other numbers, and follow the graph of connections by transitivity.

"It's well known that any graph of associations in the real world tends to generate cliques, and that the clique size for a social group of any sort tends to actually be fairly small. This is the 'six degrees of Kevin Bacon' effect. But in a social network, there will also be people with many edges coming to them, and many paths in the transitive closure of the graph of their relationships, and those people are often 'centers.'

"In fact, just that sort of analysis was done -- after the fact -- of the 9/11 hijackers (in this week's links).

"I would guess that the SNA is used to identify people of interest -- although there would be some false positives, like if they all rented apartments from the same rental management firm, or all ordered from the same we-deliver falafel place. But someone who shows up in the transitive network of a lot of calls from overseas, and is also a high edge-count in the SNA graph, is definitely someone to be interested in. I wouldn't be at all surprised if that's when they apply for a FISA warrant and start actually intercepting."

So what we have the NSA doing is probably data mining, calling records in order to identify the people they want to order intercepts on. They are doing it without warrants because they like being sneaky, don't think they could get past the FISA court a warrant for 100 million calling records, and because the FISA law from 1978 probably doesn't distinguish between a pen-trap and an intercept.

If that's really the case, this doesn't sound quite as bad as we've feared. I feel better thinking that they are culling calling records rather than listening-in to my conversations. And it makes a lot more sense, from a pure technical capability standpoint.

So why couldn't they just tell us? Why couldn't they have simply amended the FISA law to take such activities into account? Because they like to be sneaky, tend to distrust even the people who pay them (that's us), and because they for some reason think that the bad guys won't figure this out for themselves.

Duh.

But is it really Duh? Either al Qaeda is a lethal threat, and thus smart enough to figure this out on their own, or do they have got a thin bench full of dolts who couldn't think their way out of a paper bag?

Either way, Bush is wrong. If al Qaeda is a lethal threat, then they've already figured this out and we could pass laws making this legal without tipping them off. On the other hand, if they have a thin bench, then we don't need the Long War and we shouldn't be shredding the Constitution.

Challenger: 20 years

Twenty years ago seven amazing people died when the space shuttle Challenger "exploded" (it didn't). James Oberg was a senior NASA insider at that time. He critiques some myths that have grown up around the tragedy.

I was not surprised by his critiques, in each case they matched what I remember reading during and after the investigation of the disaster. It's true that most commentators didn't dwell on the fact that the cabin was intact when it hit the water, but even that wasn't a secret.

I was surpised by the myths, especially the one that "environmental regulations" had led to a sealant problem. I guess 20 years is long enough for legends to start.

I was in the Mojave desert when the first shuttle flight landed. I remember worrying about the damned tiles -- they'd broken loose in orbit (shades of later tragedy). In those days I dreamt about becoming a mission specialist and flying, but really I knew the competition was too stiff. Everyone who has flown, and many who just waited for a slot, are among the most exceptional human beings one can know.

7 myths about the Challenger shuttle disaster - Space News - MSNBC.com

Twenty years ago, millions of television viewers were horrified to witness the live broadcast of the space shuttle Challenger exploding 73 seconds into flight, ending the lives of the seven astronauts on board...

...spaceflight historians believe that each element of the opening paragraph is factually untrue or at best extremely dubious. They are myths, undeserving of popular belief and unworthy of being repeated at every anniversary of the disaster.

The Long War and the Quadrennial Defense Review: 20 years of the imperial president?

Bush's justification of his assertion of new presidential powers is that we're at War. What he doesn't emphasize is that he's thinking about a rather Long War ...

Early Warning by William M. Arkin - washingtonpost.com: "Goodbye War on Terrorism, Hello Long War

One phrase contained in the draft Quadrennial Defense Review document circulating amongst defense experts is sure to be a part of your life for years to come: The long war.

Defense experts want the long war to be the new name for the war on terror, a kind of societal short hand that will stand shoulder to shoulder with the Cold War, promoted to capital letters, an indisputable and universally accepted state of the world.

'This generation of servicemembers will be in what we're calling the Long War,' Army Lt. Gen. Ray Odierno, assistant to the chairman of the Joint Chiefs of Staff, said earlier this week.

'Our estimate is that for at least the next 20 years … our focus will be … the extremist networks that will continue to threaten the United States and its allies.'

A War against Evil could go on a long time, or until the End Times anyway. Given what Bush has done so far, is it inconceivable that he and his supporters will decide he should be able to run again? Yes, that's nuts. Of course. Forgive me...

Building a business without VCs

Kawaki's blog is still superb. He's gotta run out of steam sometime.

In some recent postings he confirmed my uninformed prejudice that VCs are a bit ridiculous, and are really only suited to a small minority of entrepreneurs.

Here he considers the alternative: The Art of Bootstrapping.

Fascinating.

Adopting a mutt - new age

We're adopting a dog. Things have changed since we picked up Molly more than 15 years ago at the Delta County pound. Back then there was no web, now we have petfinder.com and computer generated breed recommendations.

Impressive. The breed recommendations were pretty much what I'd have guessed. On the other hand, the Delta pound had some beautiful mongrels, and their mutts weren't $250 each.

They should, however, include a longevity question in the breed selector. I'm going to see if I can get them to add that in.

Defining the Drake equation: how common are rocky planets?

A new imaging technique allows visualization of a very large rocky planet orbiting a red dwarf:

Scientists spot a new Earthlike planet - Space.com - MSNBC.com:

The planet and star are separated by about 2.5 astronomical units.

The finding means planet hunters are one step closer to detecting their holy grail: a habitable Earthlike planet that can sustain liquid water and support life.

The distance sounds comparable to our asteroid belt. I wonder if that's coincidental.

The significance of the discovery may be the proof of the microlensing technique. We probably have years to go before we can begin to put more meaningful numbers into the Drake Equation, and thus constrain the solutions to Fermi's Paradox.

A retrospective look: the war on terror

I don't read Juan Cole regularly, but I thought his recent retrospective on the 'War on Terror' was well timed. In September of 2001 I was certainly asking myself -- was 9/11 a fluke, or was al Qaeda really something new? Did they "get lucky", or could they repeat?

Early on the safest guess was that they were something new. I still think that the falling costs of havoc have changed the old equations. Richard Reid, though made me wonder. Clearly al Qaeda was scraping the bottom of the barrel when they used him. Since then we've seen more evidence that al Qaeda's bench is weak. They can get lots of fairly ordinary people to commit suicide, but they seem to have trouble holding and recruiting the elite operatives that would make them truly dangerous.

In retrospect, al Qaeda's "success" on 9/11 seems to have had a large element of chance. They were "unlucky" on a prior attack on the WTC (the basement bombing) and they were almost supernaturally "lucky" on 9/11. Having Afghanistan as a base was critical, and being left relatively alone helped them too. They don't loom so large now, in large part due to the US military action in Afghanistan. Since then, however, Bush has seemed to be acting as an agent of bin Laden -- helping rather than hurting him. Cole expands:

Juan Cole - Informed Comment Top Ten Mistakes of the Bush Administration in Reacting to Al-Qaeda
Tuesday, January 24, 2006

... On September 11, 2001, the question was whether we had underestimated al-Qaeda. It appeared to be a Muslim version of the radical seventies groups like the Baader Meinhoff gang or the Japanese Red Army. It was small, only a few hundred really committed members who had sworn fealty to Bin Laden and would actually kill themselves in suicide attacks. There were a few thousand close sympathizers, who had passed through the Afghanistan training camps or otherwise been inducted into the world view. But could a small terrorist group commit mayhem on that scale? Might there be something more to it?...

Over four years later, there is no doubt. Al-Qaeda is a small terrorist network that has spawned a few copy-cats and wannabes. Its breakthrough was to recruit some high-powered engineers in Hamburg, which it immediately used up. Most al-Qaeda recruits are marginal people, people like Zacarias Moussawi and Richard Reid, who would be mere cranks if they hadn't been manipulated into trying something dangerous... They are left mostly with cranks, petty thieves, drug smugglers, bored bank tellers, shopkeepers, and so forth, persons who could pull off a bombing of trains in Madrid or London, but who could not for the life of them do a really big operation.

The Bush administration and the American Right generally has refused to acknowledge what we now know. Al-Qaeda is dangerous. All small terrorist groups can do damage. But it is not an epochal threat to the United States or its allies of the sort the Soviet Union was (and that threat was consistently exaggerated, as well).

In fact, the United States invaded a major Muslim country, occupied it militarily, tortured its citizens, killed tens of thousands, tinkered with the economy-- did all those things that Muslim nationalists had feared and warned against, and there hasn't even been much of a reaction from the Muslim world. Only a few thousand volunteers went to fight. Most people just seem worried that the US will destabilize their region and leave a lot of trouble behind them. People are used to seeing Great Powers do as they will. A Syrian official before the war told a journalist friend of mine that people in the Middle East had been seeing these sorts of invasions since Napoleon took Egypt in 1798. "Well," he shrugged, "usually they leave behind a few good things when they finally leave."

Because they exaggerate the scale of the conflict, and because they use it cynically, Bush and Cheney have grossly mismanaged the struggle against al-Qaeda and Muslim radicalism after September 11. Here are their chief errors:

1. Bush vastly exaggerates al-Qaeda's size, sweep and importance, while failing to invest in genuine counterterrorist measures such as port security or security for US nuclear plants.

2. Bush could have eradicated the core al-Qaeda group by putting resources into the effort in 2002. He did not, leaving al-Zawahiri and Bin Laden to taunt us, inspire our enemies and organize for years after the Taliban were defeated. It would be as though Truman had allowed Hitler to broadcast calls for terrorism against the US from some hiding place as late as 1949.

3. Bush opened a second front against Iraq before he had put Afghanistan on a sound footing.

4. Bush gutted the US constitution, tossing out the Fourth Amendment, by assiduously spying on Americans without warrants. None of those spying efforts has been shown to have resulted in any security benefits for the United States. Bush says that he wants to watch anyone who calls the phone numbers associated with al-Qaeda. But some of those phone numbers were for food delivery or laundry. We want a judge to sign off on a wire tap so that innocent Americans are not spied on by the government.

5. Bush attempted to associate the threat from al-Qaeda with Iran and Syria. Iran is a fundamentalist Shiite country that hates al-Qaeda. Syria is a secular Arab nationalist country that hates al-Qaeda. Indeed, Syria tortured al-Qaeda operatives for Bush, until Bush decided to get Syria itself. Bush and Cheney have cynically used a national tragedy to further their aggressive policies of Great Power domination.

6. Bush by invading Iraq pushed the Iraqi Sunni Arabs to desert secular Arab nationalism. Four fifths of the Sunni Arab vote in the recent election went to hard line Sunni fundamentalist parties. This development is unprecedented in Iraqi history. Iraqi Sunni Arabs are nationalists, whether secular or religious, and there is no real danger of most of them joining al-Qaeda. But Bush has spread political Islam and has strengthened its influence.

7. Bush diverted at least one trillion dollars in US security spending from the counter-terrorism struggle against al-Qaeda to the Iraq debacle, at the same time that he has run up half a trillion dollar annual deficits, contributing to a spike in inflation, harming the US economy, and making the US less effective in counterterrorism.

8. Counterterrorism requires friendly allies and close cooperation. The Bush administration alienated France, Germany and Spain, along with many Middle Eastern nations that had long waged struggles of their own against terrorist groups. Bush is widely despised and has left America isolated in the world. Virtually all the publics of all major nations hate US policy. One poll showed that in secular Turkey where Muslim extremism is widely reviled and Bin Laden is generally disliked, the public preferred Bin Laden to Bush. Bush is widely seen as more dangerous than al-Qaeda. This image is bad for US counterterrorism efforts.

9. Bush transported detainees to torture sites in Eastern Europe. Under European Union laws, both torture and involvement in torture are illegal,and European officials can be tried for these crimes. HOw many European counterterrorism officials will want to work closely with the Americans if, for all they know, this association could end in jail time? Indeed, in Washington it is said that a lot of our best CIA officers are leaving, afraid that they are being ordered to do things that are illegal, and for which they could be tried once another administration comes to power in Washington.

10. Bush's failure to capture Bin Laden and al-Zawahiri allows them to continue to grandstand, to continue to frighten the public, to continue to affect financial markets, and to continue to plot. Al-Zawahiri almost certainly plotted the 7/7 London subway bombings himself, and gloated about it when he issued Muhammad Siddique Khan's suicide statement. Misplaced Bush priorities are getting our allies hit. The CIA is reduced to firing predators at villages because our counterterrorism efforts have been starved for funds by the Iraq quagmire. If al-Qaeda does pull off another American operation, it may well give Bush and Cheney an opportunity to destroy the US constitution altogether, finally giving Bin Laden his long-sought revenge on Americans for the way he believes they have forced Palestinians and other Muslims to live under lawless foreign domination or local tyranny.

If al Qaeda is not an ephocal threat, do we really need to surrender our freedom, and change the balance of American government? Are we throwing away our freedom for no good reason?

My best explanation for Bush is that he's a deep KGB plant -- or an agent of a malign alien civilization. Next that he's incompetent. Lastly that he's the human expression of a rather scary desire of many Americans for the Great White Father. However, America elected him (once), and America elected his party to control the House, Senate and Supreme Court, and now America may choose to support his reinterpretations of the Constitution. If Bush is not challenged, history will judge that Americans took this road voluntarily and consciously. Thus will a great dream die.

Smile for the satellite: Google Maps update

Six inch imagery at some spots.

Official Google Blog: New year, new imagery

... added extensive 6-inch imagery for many parts of the U.K.

... added two more zoom levels in Google Local's Satellite mode!

Take a look at people standing at the gates of Buckingham Palace in London, or jump over the pond and see the Statue of Liberty in New York, and then maybe drop down to the southern hemisphere and check out the boats sailing past the Sydney Opera House.

Our home isn't available at the very highest resolution, but check out this random shot from Boulder, Colorado. The altitude, clear air and high resolution imagery means it's easy to tell cars from trucks.

A use for blacklists: Political spam

I've never had much use for blacklists. Most spammers don't use valid email addresses.

Times change. Political parties and candidates do use valid email addresses, and they are spamming me like mad. Oh what a fool I was to give my email address along with money to various democratic party candidates. (Lash, lash, lash.)

This guy is typical: spam email with no way to get off the list. Of course he won't get my money, but he's a fringe candidate anyway.

So now my blacklists and bounce tools will get a workout. It's nice to know blacklists are good for something.

Cervical ribs and cancer risk

I remember being taught that cervical ribs were a relatively harmless congenital anomaly. True, it's been years since I practiced clinical medicine, but is this widely known? Is it true?

Pharyngula

There isn't much variation in cervical vertebra number, though. There is an exception: sometimes, the 7^th cervical vertebra is found to undergo a partial homeotic transformation and forms a pair of ribs, which are normally found only on thoracic vertebrae. Humans develop cervical ribs with a frequency of about 0.2%; do they also develop cancers? The answer is yes, with a frequency 125 times greater than the general population.

The power of the contest: Booting XP on a MacTel

[Update 1/24: Colin added an Amazon donation link and I kicked in $20.]

A very clever person has created a several thousand dollar prize for getting XP to boot on a MacTel box -- out of thin air:

The Contest

My MacBook is shipping on the 15th of February. I told my boss that this would replace my IBM desktop and I could boot Windows XP on it. I am still confident it can be done. I am pledging $100 of my own money and offering anyone else who would like the instructions on how to Dual boot these two operating systems the ability to donate some of their money into the pot as a reward for the person / group that can make dual-booting Mac OS X and Windows XP happen on an Intel Mac.

He started with $100 in seed money, and is now raising serious funds. There are thousands of geeks who seriously want to do this. Apple, for unknown reasons, is not helping (security?, strategy?, support concerns?, Digital Rights Management?). The primary obstacles are drive format and the MacTel's BIOS replacement.

Unfortunately he only accepts PayPal or I'd kick in $20. Happily he's added an Amazon link to the PayPal option so I kicked in $20 (I despise PayPal.) The contast has now gotten enough attention that it's a matter of both money and glory. I would not be surprised to see some silicon valley millionaires sweeten the pot considerably.

We've seen contests used in aerospace, human powered vehicles, and other settings. In a connected world, where the costs of reaching millions of people is very low, the power of these contests is likely to grow. If this particular effort succeeds, it may, in retrospect, be a truly historic event.

Google news now learns what you like

Google News is "out of beta", meaning Google is committing to it for at least a few years. Google news now "learns" what you might like to read. Another small step for Skynet.

Official Google Blog: And now, News:

You can sign up for Personalized Search to view and manage your history of news searches and the articles you've read. When you're signed in to your Google Account, you'll receive recommended news stories based on the previous stories you've read. These recommendations will be highlighted just below the top news stories on the page, in a clearly marked section. You can also get a full page of recommended stories by clicking on the section. All of this is done automatically using algorithms.

Update. It took a few minutes for Google news to start showing the new section, but now I have an extra news section that looks like this:

Recommended for jfaughnan@gmail.com » Learn more
UK support for rights groups raises suspicion in Kremlin
Guardian Unlimited - 1 hour ago - Russian non-governmental organisations, yesterday expressed concern that spying allegations against British diplomats ...
Los Angeles Times - Telegraph.co.uk - The Moscow Times - all 376 related »

Sudan seeks to end split at African Union
Financial Times - 1 hour ago - By Andrew England in Nairobi and Reuters. Sudan said yesterday it was willing to withdraw its candidacy for the ...
Reuters - News24 - Voice of America - all 425 related »

Saudi Araba in Energy Deal With China
Houston Chronicle - 13 hours ago - By ALEXA OLESEN Associated Press Writer. BEIJING — Saudi Arabia and China inked a deal on energy cooperation on ...
Financial Times - Arab News - Hindu - all 339 related »

The stories listed are moderately interesting to me; it will be neat to see where this goes.

Travelers tales: a list of favored cities and times

Obsidian Wings, a politically moderate security-focused blog, drifted into the topic of fun business travel and places. It's the comments that make the piece. Many of the comments are about places that would require a time machine to visit.

I can vouch for some of these lost places -- Chiang Mai in 1981, Jerusalem in 1983. I regret not having visited Kashmir when I might have @1981 -- the way the war is going there I won't ever see it. I'm told old Laos was a rare gem. Manhattan in the 1940s and 1950s. Shanghai before the boom. Koh Samui when it was hard to reach.

A time traveler would have no end of wonderful cities and places to visit. Many were most wonderful for a short period of time, modernity has come fast and hard to much of the world. I can't deny the urgency of poverty, but I suspect one day many Chinese will deeply regret what has been destroyed. In my home town of Saint Paul/Minneapolis an awful lot of local history and charm was plowed over for roads and trains from 1950 to 1970.

Travel while you can. You cannot cross the same river twice (or even once).

PS. If once read a book titled '1,000 Places to See Before You Die'. It was awful. Really 1,000 hotels to visit before you die. The author apparently never left her room. I have to admit, I remember some of those hotels fondly -- but really ...

iTunes users are Democrats?

iTunes grown (14% of net users) is huge news for Apple. iTunes embedding is fundamental to Apple's success. They may have crossed the "tipping point". Most interesting, however, is that their user base appears to be democrat:

BBC NEWS | Technology | Apple iTunes users growing fast

Curiously, the market research firm also found that iTunes users comprise a readily identifiable audience in terms of their likes and dislikes for certain goods and services.

For instance, Nielsen said, iTunes users were 2.2 times more likely to own a Volkswagen than the average internet user. Audi and Subaru were also popular with regular users of the Apple store.

Marketers usually consider ownership of these vehicles as a very reliable market for someone who votes Democrat first.

Microsoft needs to go for the Republicans.