Haystacks, Needles and the Constitution

If you are not convinced that merely talking about the NSA wiretapping program corrodes and destroys our American intelligence gathering capability, you should take the time to read an important article in the Washington Post this morning. It provides far more detail on what this program actually does than the administration has provided and also suggests that the program is far less effectual and far more intrusive than Mr. Bush would have us believe.

As previously discussed, the program appears to use artificial intelligence systems to sort through intercepted communications, looking for clues. The Post article suggests that the number of actual clues turned up this way is small. Very small.

Here is the image that comes to mind. Imagine a huge field, with thousands of haystacks. Imagine you have a spiffy-cool technology that allows you to sort through a haystack to find objects that might be a needle. You suspect that there may be a needle, because there are bad people out there who want to stick you.

Some might think, 'Well, we could look through this whole field, which might be a huge waste of time as well as potentially illegal. Or we could redouble our efforts to use more effective techniques like conventional human intelligence and detective work to find the needle-stickers, while putting on needle-resistant clothing." As we know, the Bush administration never does what "some" think.

They say 'to a man with a hammer, everything looks like a nail.' To a government with a multi-billion dollar supercomputer analysis facility and access to the fiber-optic cables that carry most of the world's data traffic, everything looks like a processing problem. There is that pesky problem about the haystacks not belonging to you, and the Constitutional prohibition of randomly searching haystacks, but hey! There are needle-stickers out there!

Which is how we get here:

Intelligence officers who eavesdropped on thousands of Americans in overseas calls under authority from President Bush have dismissed nearly all of them as potential suspects after hearing nothing pertinent to a terrorist threat, according to accounts from current and former government officials and private-sector sources with knowledge of the technologies in use.

Bush has recently described the warrantless operation as "terrorist surveillance" and summed it up by declaring that "if you're talking to a member of al Qaeda, we want to know why." But officials conversant with the program said a far more common question for eavesdroppers is whether, not why, a terrorist plotter is on either end of the call. The answer, they said, is usually no.

The article suggests that the program has used both link analysis and pattern matching to look for clues. Link analysis is the "Six Degrees of Kevin Bacon" game: start with a captured terrorist's address book, and check out all those people. Then check out the people they are connected with. Some problems: you quickly have an enormous number of links that mean nothing, like the pizza delivery guy, as you move further from the original terrorist, and, if you capture fewer terrorists, the beginning links dry up. Which leads to pattern analysis:

Analysts build a model of hypothetical terrorist behavior, and computers look for people who fit the model. Among the drawbacks of this method is that nearly all its selection criteria are innocent on their own. There is little precedent, lawyers said, for using such a model as probable cause to get a court-issued warrant for electronic surveillance.

Jeff Jonas, now chief scientist at IBM Entity Analytics, invented a data-mining technology used widely in the private sector and by the government. He sympathizes, he said, with an analyst facing an unknown threat who gathers enormous volumes of data "and says, 'There must be a secret in there.' "

But pattern matching, he argued, will not find it. Techniques that "look at people's behavior to predict terrorist intent," he said, "are so far from reaching the level of accuracy that's necessary that I see them as nothing but civil liberty infringement engines."

One aspect of this that I find still more troubling, and which the Post doesn't really address, is what happens to the "hay" that the artificial systems discard as being "not a needle".

It would be pretty to think that this was all immediately disposed of. But it seems unlikely. Once you've got that data, why waste it? Data storage gets cheaper every day. Why not keep it around, in case your haystack-sorting model is improved later?

Or, perhaps, the definition of "needle" changes from "al Qaeda terrorist" to "political opponent of the President"? The distinction between the two has already been erased rhetorically, so what's keeping it from being erased at the NSA?

Nothing. Not even the tissue-thin defense of the FISA court. So the only thing keeping conversations safe is the honesty of the men who've lied to us, and the mathematical challenge of finding the right callers.

That challenge is probably easier when you know who they are in advance. So easy, in fact, that "some" might even think it was a logical reason for developing this program, and defending it so fiercely.