Bart Gellman DEBUNKS NSA 'data-driven analysis' debunking: says "math isn't the problem"

[nashville_brook]

ZDNet has an article that's caught the attention of some here on DU, titled: "Data-driven analysis debunks claims that NSA is out of control." It's a critique of Bart Gellman's WaPo piece titled: "NSA broke privacy rules thousands of times per year, audit finds."

The critique is penned by David Gerwitz who is a member of FBI InfraGard and the Cyberwarfare Advisor for the International Association for Counterterrorism & Security Professionals -- clearly someone with a personal stake in the success of our domestic spying apparatus via NSA/DoD/FBI/etc contractors.

Here's a pic of the author Gerwitz.




Given that this article is referenced earlier on DU I thought it might be nice to share what the original Washington Post author, Pulitzer-Prize winning investigative journalist Barton Gellman has to say about. Gellman has appeared on Rachel Maddow and Chris Hayes recently to comment on the issue of NSA error-rate and noncompliance.

His response below was posted in the comments section of the Gerwitz column on ZDNet. It's well worth the read.




___________________________

from the author of the Washington Post story (Barton Gellman)

I'm the author of The Washington Post story. There's a newsroom expression. "Danger: reporter doing math." I'm not going to audit David, but in any case the math won't be the problem here. The problem is that he misunderstands what he's counting. I don't blame him for that: This is a very complex set of legal, technical and operational questions. I have been following them closely since 2005, and devoted two chapters of my last book to them, and I still don't find them easy.

No time for a treatise but a few quick points:

* The "compliance incidents" do not all involve collection. As the story and the documents note, they can take place anywhere along the spectrum of electronic surveillance: collection, retention, processing or distribution. Any of them can range from the minor, with little privacy impact, to the very serious.

* David assumes the surveillance is all about metadata. It is not. Much of it -- an unknown quantity, because the report does not break this down -- is content. As the story notes, the NSA does not "target" Americans for content collection but it does collect a great deal of American content "inadvertently," "incidentally" or deliberately when one party is known to be a foreign target overseas. Most of it stays in databases, and a single search can pull up gigabytes.

* A crucial point to understand: the last two categories of collection on Americans -- "incidental" and deliberate, when one party is overseas -- account for the highest volume of American data in NSA hands. They DO NOT COUNT as incidents. NONE of them are among the 2,776 incidents. As the NSA interprets the law, it is not a violation to collect, keep and process it. Until my story that had never been clear, and the White House still works hard to obscure the difference between forbidden and routine collection (including collection of content) from Americans. "Minimization" rules strip out identities by default, but there are many exceptions and requests from "customers" to unmask identities are readily granted.

* It is not possible to calculate or even estimate within several orders of magnitude the quantity of data involved in 2,776 incidents, nor the number of people affected, even if you know whether you're dealing with metadata or content. A small but unknown number of incidents -- those involving unlawful search terms but obtaining no results -- do not collect, process or disseminate any data at all and thus have zero privacy impact. Other incidents may involve only a few surveillance subjects but includ large volumes of data, either because collection takes place over a span of time or because the previously collected data set is very large. One "incident" in the May 2012 report involved over 3,000 database files, and each file contained an unknown (but typically very large) number of records. Another episode -- not counted as an "incident" at all -- collected data on all calls from Washington, DC for an unknown period of time. There is no way to tell from the report alone, but based on the routine procedures and scale of NSA operations it is likely that some of these individual incidents (1 of 2,776) affected hundreds of thousands of people.

* By the way, as again the story notes, the 2,776 cover only Ft. Meade and nearby offices. There would be substantially more incidents in an audit that included the SIGINT Directorate's huge regional operations centers in Texas, Georgia, Colorado and Hawaii -- and the activities of other directorates such as Technology, and such as Information Assurance, that also touch enormous volumes of data.

* It's fair game to take a full data set and challenge a reporter's (or researcher's) analysis of the data. But this was not a full data set and it's a mistake for David to think he can suss out the whole story from the limited number of documents we posted alone. I drew upon other documents and filled the gaps with many hours of old-fashioned interviews. I took some primary material, combined it with other leads, and applied journalism in order to understand what the material says, what it doesn't say, and what inferences can and can't be drawn from it. That's among the reasons we don't just dump documents into the public domain. There are not many stories in the Snowden archive that can be told by documents alone.

* Despite all this, David is surely right to say the error rate is very low in percentage terms. That is important in assessing individual performance, and maybe that's the end of the story for you. That's your choice. For some people, public policy question considers the absolute number as well. We might not accept the more mundane harm of 1 million lost airline bags a year, even if 99.9 percent of 1 billion bags checked annually made it to their destinations. Some systems have to be designed with less fault tolerance than others. That's a political and social decision, but we have been unable to debate it until the Snowden disclosures.

* Part of the importance of this story is that the government worked so hard to obscure it. In public releases of semi-annual reports to Congress, the administration blacked out ALL statistical data. (By the way, note that the tables in the 14-page document I posted are unclassified. In the DOJ/DNI report to Congress, they were marked Top Secret // Special Intelligence, which made public release impossible and restricted the readership in Congress.) Alongside the refusal to release any data, the government left the very strong impression that mistakes were vanishingly rare and abuse non-existent. That may depend on the definition of "abuse."

Marcy Wheeler quotes a tv interview in which I discussed that and makes some additional points here: http://www.emptywheel.net/2013/08/20/if-nsa-commits-database-query-violations-but-nobody-audits-them-do-they-really-happen/.

[Luminous Animal]

1. Kick.

Why are people calling for Gellman to be investigated? He has many of the source materials, if not all, that Greenwald has.

Would they be defending the British security actions if Gellman from the WaPo had been held for 9 hours and his equipment confiscated?

[KittyWampus]

2. "much of it is content". Could you point me to that info? And what does "content" mean?

How is it stored? Is it gotten via warrants?

[nashville_brook]

3. 'content' here refers to the actual content of communications, rather than metadata

and the problem that both Gellman and Wheeler have been sussing out is that it's gathered under a questionable reading of Section 215 -- the questionable-ness is that Section 215 wasn't meant to give blanket approval of collecting content (just the opposite), but that's the justification that's being used.

So, there's no warrant.

[East Coast Pirate]

16. Well that was s short subthread.

[DirkGently]

22. THIS needs to be repeated in about 37 threads, immediately.

That is all.

[KittyWampus]

31. Provide proof of this "content" please. Of course you can't.

[nashville_brook]

41. by definition...this is content...content is content.

[Vinnie From Indy]

7. You STILL have not answered my earlier question!

I figure since you are reading an article by Bart Gellman, this would be an appropriate place to ask you again if you support and would defend the detention of Bart Gellman should he travel through or in Great Britain??????

I can fully understand your reluctance to answer this question as it would make you appear quite silly after your many posts supporting the detainment of David Miranda.

Cheers!

[nashville_brook]

11. that's quite an interesting question

i think there's a line drawn between Gellman and Greenwald...insider/outsider thing, maybe.

i could be wrong, but i sense an amount of slack afforded to him that Greenwald doesn't enjoy. might be b/c he's under the radar...which is why i love your question.

[East Coast Pirate]

20. Which makes it a crucial question.

Should the U.S. government decide who is a journalist? Should they use other governments to go after someone who they decide isn't really a journalist?

[nashville_brook]

24. precisely. totally agree.

i think this points up a major hypocrisy, and a dangerous elitism -- especially when you take into account that wrt local reporting, some of the most important writers are bloggers: unaffiliated with dead-trees publishing.

[DirkGently]

26. Anyone publishing truthful information can be

a journalist. Which of course is precisely the threat that's bringing all these hammers down so frantically.

[sabrina 1]

27. Greenwald became an enemy of the State when he began writing about Wall St. Banks,

specifically BOA a few years ago. It was at that time that Anonymous hacked into HB Gary's website and exposed all of their internal emails. There we found that this Private Security Corp was bidding on a contract and was proposing how 'silence blogger Glenn Greenwald on the issue of BOA corruption.

The proposal suggested a 'smear campaign' against him, doing 'opposition research to find out info on his 'family, kids if he had any, what schools they went to, his wife, if he had one, what were her interests, etc. The idea was that 'these types', meaning bloggers like Greenwald, would 'back off' as they cared more about their 'status' and would likely not want to risk being sidelined.

It was a shock at the time since Greenwald, up to then, was not known nationally, even HB Gary didn't know much about him as evidenced by the 'opposition research' they were proposing. Once exposed, HB Gary had to shut down that section of their operations.

However, the smears against Greenwald, who expressed surprise that he would be the focus of a contract like that from a 'Security Corp', began shortly after as we saw right here on DU. This most likely means that someone got the contract.

They were wrong when they thought he was like the Corporate Media 'journalists' who 'would not risk being sidelined' as he was not intimidated into silence on Wall St banks.

What is clear is that even a blogger who is relatively unknown nationally, will be attacked with smear campaigns and now it has obviously escalated, IF they do not stick the 'rules' of what passes for American Journalism.

Independent media is a real threat to them obviously, since they do now 'own' them and cannot control them the way they have the MSM.

That is why I believe he should be supported because he has become the Independent media's 'experiment, along with Wikileaks, also targeted AFTER announcing they had info on BOA, by those who wish to control the media, to see if they can reign in the Independent Media as they have the MSM.

It is vitally important now to protect the Independent media as it clearly is the only way the people can get information that is censored on the Corporate Media.

Whether you like people like Greenwald or not, that is not the issue. This is a battle for freedom of the press and when we see that Private Security Corps are bidding on contracts to silence even small bloggers, we should be scared enough to stand up and fight for those who are putting themselves on the line.

Just my opinion.

[Luminous Animal]

17. Hey! I've asked that question, too! Keerickets.

[Vinnie From Indy]

19. I guess a cat has her tongue!

Cheers!

[KittyWampus]

33. Find proof of his assertion of "content" being looked at with the definition of that content and

that it was gotten without a warrant.

[DirkGently]

4. A nice sharp whack through the b.s.

A lot of convenient conceits are being invented by people whose first instinct is to defend the authorities here. One is the assumption that we're only talking about metadata.

But it raises a huge red flag when someone tries to reduce something this complex and secretive into something as anonymous as a percentage point. A percentage of what? What's in the percentage that went wrong? If one in a million bags go missing, okay, maybe?

But if 10 million of a hundred zillion data grabs appropriate the private communications of Americans not accused of any crime, when what we theoretically allow is zero, that's a problem.

And of course the NSA's reports select and interpret in favor of the NSA to begin with. I saw someone argue recently that the NSA would never "lie to itself." Really? Like a bank would never kid itself about someone's ability to pay off a loan, or the value of a mortgage - backed security?

There's plenty to look at here, and all indications are unraveling it all will take approximately forever.

But airily declaring a partial statistic from a partial report of carefully selected data based on unsupervised internal interpretations of the law to answer all questions is about as serious and credible as arguing about Snowden's garage boxes or whether he associates with pole dancers.

[nashville_brook]

5. it's indeed complex which makes it easy for lobbyists like Gerwitz to spin from

this sort of thing works wonders in front of clients. But not so much with Pulitzer-Prize winning investigative reporters, or those civvies like us trying to keep up.

that's why i especially love how Gellman points out that there's big qualitative problems in the critique. that it's really not a matter of math b/c the numbers aren't going to do you any good if you don't know what they represent.

back to Gerwitz...that's what he's paid to...obfuscate what those numbers represent.

[DirkGently]

14. No one loves statistics like liars do.

You can attempt to conclude anything you want, as long as you're willing to gloss over where numbers come from and what they truly represent.

[nashville_brook]

42. lies, damn lies and statistics.

[jmowreader]

15. More like a bank never kidding itself about the amount of money in its accounts

[Catherina]

25. +1 n/t

[AzDar]

6. K & R

[MissMarple]

8. +1

[stevenleser]

9. The money shot is the paragraph about the low error rate and I will tell you all why...

A low error rate is a powerful statement on motive. If the error rate is as low as Gerwitz and Gellman agree it is, we don't have a government running amok trying to spy on its citizens and Snowden and Greenwald have alarmed us for nothing.

re:

* Despite all this, David is surely right to say the error rate is very low in percentage terms. That is important in assessing individual performance, and maybe that's the end of the story for you. That's your choice. For some people, public policy question considers the absolute number as well. We might not accept the more mundane harm of 1 million lost airline bags a year, even if 99.9 percent of 1 billion bags checked annually made it to their destinations. Some systems have to be designed with less fault tolerance than others. That's a political and social decision, but we have been unable to debate it until the Snowden disclosures.

[DirkGently]

13. That conclusion doesn't follow at all if you read the piece.

The NSA saying that two offices of the NSA gathered more or less what the NSA says the NSA is entitled to gather is conclusive of nothing.

As the piece noted, they're talking about their own interpretations of how they gather data. Not how it's used, and not "incidental" or deliberate collection of the content of Americans' data -- which is where most of the Americans' data the NSA has comes from.

This is the NSA saying the NSA made several thousand errors that it acknowledges. In two offices. Not counting incidental or deliberate collection of data.

Part of that is because the NSA *interprets the law* to mean that collection, keeping, and processing of information it is not supposed to have doesn't count. But then it's very murky at best as to what oversight or controls are used when they then take that collected material and actually access it.

It doesn't get to the core issues at all.

[nashville_brook]

23. love how this follows directly from Gellman's comments -- "it's not the math, stupid."

[sabrina 1]

28. Well, let's see:

* Despite all this, David is surely right to say the error rate is very low in percentage terms.

But he goes on to say this. Note the words 'individual performance':

That is important in assessing individual performance, and maybe that's the end of the story for you.

And then this, which really is the 'money shot' if you are looking for the facts. Note the words 'absolute number'.

That's your choice. For some people, public policy question considers the absolute number as well.

And then finally this:

We might not accept the more mundane harm of 1 million lost airline bags a year, even if 99.9 percent of 1 billion bags checked annually made it to their destinations. Some systems have to be designed with less fault tolerance than others. That's a political and social decision, but we have been unable to debate it until the Snowden disclosures.

It all depends on what the motive is, how one decides to 'do the math'. But when it is about the very foundation of democracy, you would think that no one whose motivations are sincere and not driven by an agenda, would risk just skimming over the numbers and would be more inclined, no would be insistent on a system with the least amount of 'fault tolerance' considering the seriousness of the issue.

I don't know why you consider his generous and honest noting of the fact that David's skimming over the numbers method, while it may be correct, has a very narrow focus, to be the 'money shot'.

When it comes to our civil liberties, I prefer a lot more care from people, who do not have an 'agenda' 'doing the math'.

His acknowledgement that while focusing only on a small % of the numbers involved, can produce a 'correct result', within that narrow focus, It was in fact a warning that someone needs to do a lot more math to ensure the least possible margin of error when it comes to our Civil Liberties and it looks like nobody has been doing it or even talking about it, until Snowden made it public, and therefore possible to talk about it.

[DirkGently]

29. Right. 4th Amendment violations aren't lost bags.

Amazing how cavalierly destruction of our supposed core principles as Democrats are being tossed on the "close enough" pile here.

[nashville_brook]

43. of the thousands of people I saw today, i only shot 2...what's the big deal?

[stevenleser]

44. Right, because mistakenly reading an email is the same as shooting someone.

[nashville_brook]

39. sabrina...you are such an amazing asset to DU

always taking the time to truly engage on these important matters...it just makes me happy.

what you say here is key:

"It was in fact a warning that someone needs to do a lot more math to ensure the least possible margin of error when it comes to our Civil Liberties and it looks like nobody has been doing it or even talking about it, until Snowden made it public, and therefore possible to talk about it."

they have the power to define what's an error, and to sample accordingly. it's amazing that looking at just ONE collection site produced >2,700 "compliance incidents" that have untold thousands of 1, 2 and 3 "hop" inquiries.

[sabrina 1]

50. Thank you, I return the compliment and appreciate your kind words very much.

And this is just what we know, which Ron Wyden says is 'just the tip of the iceberg'.

[stevenleser]

45. See my #43. This isn't one of those issues...

... where an honest mistake is a big deal. We're not killing someone by mistake.

[sabrina 1]

46. No, we're killing their Civil Rights and for what? n/t

[stevenleser]

47. No, we're not. nt

[sabrina 1]

49. Speak for yourself. I have seen it with my own eyes, the crackdown on OWS with militarized

robo cops and weapons eg. The crushing of Whistle Blowers, no matter how they choose to blow the whistle, Drake eg, who did everything by the book, was persecuted for five years and had his family and career destroyed.

No democracy treats their own people, peacefully protesting, exposing wrongdoing, the way this country does.

I don't intend to close my eyes, but I did fire Verizon so at least I am not paying them to spy on me for the Government anymore.

[Rumold]

48. How many illegal drug raids result in death

how many were wrongly started by the NSA.

Can you say zero.

[Shivering Jemmy]

10. "David is surely right to say the error rate is very low in percentage terms"

At least this addresses my main concern.

[Democracyinkind]

12. Arrest Bart Gellman!!!

How dare he write such terroristic stuff!!!

[nashville_brook]

18. i know, right. clearly he's an enemy of the state.

[DirkGently]

21. Apparently being a competent journalist is the

... first symptom of being dangerous these days.


Good.

[nashville_brook]

30. being a competent thinker is next.

[KittyWampus]

32. This entire thread is pathetic. You STILL haven't provided any proof of "content". Zero. None.

[nashville_brook]

34. what for you would constitute proof?

[Dustlawyer]

35. What I don't understand is why more people are

not upset at this! I would not think that this should be a Republican vs. Democrat issue. By their lies they have already proven that they cannot be trusted with this information!

[nashville_brook]

37. judging from the discourse here, there are those who are upset precisely bc this transcends partisan

boundaries. hence the "paulite!" smears and the accusation that if you support civil rights you do so at the peril of the party.

it's nauseating.

[Dustlawyer]

38. Wholeheartedly agree! They miss what is really important due to the petty bickering.

It is all by design and works too well!

[nashville_brook]

40. Obama promised to build coalitions on things "we can agree upon" as Americans

you'd think that the most ardent partisans for Obama would heed this and welcome coalition-building where it's possible on things that matter to us.

civil liberties is one area of possible coalition building. medical marijuana reform is a classic other. we saw Grayson co-sponser a bill with Ron Paul to audit the fed which stood out as one of the only bipartisan votes of that Congress.

we eschew coalition-building at our own expense.

[myrna minx]

36. K&R n/t

[99th_Monkey]

51. K & R ~ nt