NSA traffic analysis: the seduction of data and the creation of conspiracy

I heard a part of the senate hearings on the NSA intercepts today. The GOP Senator was lobbing softball questions at the Bush attorney, who answered them well and carefully. Of course none of the questions were about the more interesting issue -- how the intercepts are selected and whether that process is legal. Cringely, who's done some good summaries to date, adds a bit more to the picture:

PBS | I, Cringely . February 2, 2006

... last thought comes from an old friend of mine who is conservative in the very best sense and knows what he is writing about:

"Traffic analysis, at the NSA? I'm tempted to be sarcastic, but I won't be. As you might know, I started a company a few years ago with a former NSA guy -- somebody who was a cryptographer and Russian linguist on those submarines that snuck into Soviet harbors to tap their phone lines -- and we applied traffic analysis to Internet discussion groups to identify opinion leaders, conversation trends and so forth. We used a lot of techniques that were developed or applied to law enforcement. And we didn't use anything that violated anybody's security clearances... really!

"(My company) was acquired by a business intelligence company funded by the CIA venture capital outfit. Apparently the stuff I invented is now in the hands of a couple of intelligence agencies, including Homeland Security.

"I'll tell you what I think the most troubling thing about all this is. It's easy to see whatever pattern you're looking for. It's like curve fitting in the stock market -- looks beautiful historically and maybe even in the short run, but it's a disaster in the making. So we have these guys running the country who saw a non-existent pattern in Iraq that justified a war ... and now we're going to give them software that will make it easy to create the illusion of patterns of conspiracy.

"Your friend from the NSA was right, but it's worse than he suggests. It's not just that social network analysis casts a wide net. It's that without oversight by people who really grasp the mathematics and have some distance from the whole thing, they're going to see patterns where there aren't any.

"They have a history of that."

The history of lie detector testing is informative. Current lie detector technology is very unreliable. It falsely implicates and falsely absolves. It's only somewhat better than a random guess, and some observers do better without the technology. Nonetheless, it is immensely abused by law enforcement. We can expect the NSA, FBI, and CIA to make the same mistakes with traffic monitoring technologies. Geeks don't get promoted in these agencies, and only geeks understand the limitations of this technology.