Sunday, July 29, 2007

How worthless is the no fly list: 20,000:0 false to true positive

We have long history in medicine of worse-than-useless tests. Tests that produced ten false alarms for every genuine alarm, causing more harm from misguided retesting and treatment than the disease being tested for. The PSA test in men may turn out to be one of those misguided tests; at best it's a borderline test.

I can't recall any diagnostic test in modern medical history, however, that produced 20,000 false positive results and no true positives. I think for a test like that you have to go back to the pre-rational era.

The pre-rational era in which Homeland Security lives today:

Schneier on Security: Terrorist Watch List: 20,000 False Alarms

The Justice Department's proposed budget for 2008 reveals for the first time how often names match against the database, reporting that there were 19,967 "positive matches" in 2006. The TSC had expected to match a far fewer number 14,780. The watch list matched people 5,396 and 15,730 times in 2004 and 2005 respectively.

The report defines a positive match as "one in which an encountered individual is positively matched with an identity in the Terrorist Screening Data Base, or TSDB."..

How do I know they're all false alarms? Because this administration makes a press splash with every arrest, no matter how scant the evidence is. Do you really think they would pass up a chance to tout how good the watch list is?

I've written about this before, often reacting to Schneier's prior posts:
There are two classes of problems with stupidity like this. One is that it causes potential harm to all the false positives, from travel delays to targeted data mining to harassment and false arrest. The other class of problem is that it harms our security. We have only limited resources to use against our enemies, spending them on chasing false leads leaves less for the real work.

    No comments: