Mail Watch

[Paul in Saudi]

They don't collect "all" internet traffic and the site you linked (in the post you linked to) isn't real.

Seriously, collecting "all" internet traffic means they're storing millions and millions of copies of every episode of Breaking Bad that anyone has ever streamed on Netflix.

Nevermind that the majority of the internet traffic never passes through US servers.

Edit: Also, back of hand math shows that simply the number of hard drives to store 5 zetabytes would be more than the entire US intelligence budget, and to store the actual amount of yearly internet traffic (~30 PB) would take between 25-50% of the world's entire supply of enterprise hard drives.

Consider (again VERONA) in the 1940s, the US and its allied recorded and stored tens of thousands of Soviet messages that were not ever theoretically breakable. (Reel-to-rell tape? Wire recorders? Paper?) They kept it "just in case." (And for traffic, technical and meta-data.)

In the modern era, Google (for example) keeps every search request. (They have to, if only for marketing.) While each copy of Breaking Bad may not be uniquely copied, certainly each request for it, each payment for it, each pirate copy sent is noted and recorded. And even if I am wrong, and each item cannot be saved, we can see that is what they want to do.

The NSA has technical abilities five or even ten years in advance of the marketplace. Look at the neat toys (very small satellite dishes, various public-key coding technologies and so on) we know they have.

The NSA and its counterparts around the world think it is their jobs to record and save everything. Even if they are not exactly doing it now, they will be in the future.

Everyone would be wise to assume everything they do online is recorded.

(Add to the mail covers, records of your spending kept by the banks, records of your location kept by mobile phone companies. Now imagine where that will be in ten years.)

I am not trying to panic-monger about this. It is a natural outgrowth of technology. It will have good and not-so-good aspects. But we ought to realize what is happening.

 

[Formaldehyde]

They don't collect "all" internet traffic and the site you linked (in the post you linked to) isn't real.

Of course it is "real". It is being completed right now. Parts of it may already be online. And it is hardly unique. The NSA had to have been storing the vast amounts of data somewhere else in the interim.

Seriously, collecting "all" internet traffic means they're storing millions and millions of copies of every episode of Breaking Bad that anyone has ever streamed on Netflix.

"Seriously", you thought I meant they don't filter anything?

However, they do quite likely take note of all the people who did rent it, as well as when and where, just as they are storing the details of these posts.

Nevermind that the majority of the internet traffic never passes through US servers.

We have proof the NSA conducts similar operations in other countries. What makes you think they don't do so in yours? Or even worse, that you own government doesn't simply provide it to them? That they don't help monitor the acts of American citizens in exchange for information concerning their own in order to be in compliance with the laws?

Edit: Also, back of hand math shows that simply the number of hard drives to store 5 zetabytes would be more than the entire US intelligence budget, and to store the actual amount of yearly internet traffic (~30 PB) would take between 25-50% of the world's entire supply of enterprise hard drives.

So now you claim the information stated in this article and elsewhere are fiction? What proof do you have they are lying other than your "back of hand math"?

 

[Zelig]

Of course it is "real". It is being completed right now. Parts of it may already be online. And it is hardly unique. The NSA had to have been storing the vast amounts of data somewhere else in the interim.

"Seriously", you thought I meant they don't filter anything?

However, they do quite likely take note of all the people who did rent it, as well as when and where, just as they are storing the details of these posts.

We have proof the NSA conducts similar operations in other countries. What makes you think they don't do so in yours? Or even worse, that you own government doesn't simply provide it to them? That they don't help monitor the acts of American citizens in exchange for information concerning their own in order to be in compliance with the laws?

So now you claim the information stated in this article and elsewhere are fiction? What proof do you have they are lying other than your "back of hand math"?

The website isn't real, it's a parody.

 

[Formaldehyde]

Ah. Good catch. But the data center obviously does exist and much of the information they state has been posted by reputable sources, including the estimated capacities.

Much of this content was derived from news media, privacy groups, and government websites. Links to these sites are posted on the left-sidebars of each page.

http://en.wikipedia.org/wiki/Utah_Data_Center

 

[Zelig]

Nothing remotely close to the numbers the parody site has.

Again, the cost in hard drives alone for 5 zetabytes would be about 20x the total cost cited by wiki.

And for more comparison, that amount of data is on the order of 1000x times more total storage space than Facebook, Microsoft and Amazon have combined.

 

[Formaldehyde]

Right. Because you say so...

The NSA Is Building the Country’s Biggest Spy Center (Watch What You Say)

Given the facility’s scale and the fact that a terabyte of data can now be stored on a flash drive the size of a man’s pinky, the potential amount of information that could be housed in Bluffdale is truly staggering. But so is the exponential growth in the amount of intelligence data being produced every day by the eavesdropping sensors of the NSA and other intelligence agencies. As a result of this “expanding array of theater airborne and other sensor networks,” as a 2007 Department of Defense report puts it, the Pentagon is attempting to expand its worldwide communications network, known as the Global Information Grid, to handle yottabytes (1024 bytes) of data. (A yottabyte is a septillion bytes—so large that no one has yet coined a term for the next higher magnitude.)

It needs that capacity because, according to a recent report by Cisco, global Internet traffic will quadruple from 2010 to 2015, reaching 966 exabytes per year. (A million exabytes equal a yottabyte.) In terms of scale, Eric Schmidt, Google’s former CEO, once estimated that the total of all human knowledge created from the dawn of man to 2003 totaled 5 exabytes. And the data flow shows no sign of slowing. In 2011 more than 2 billion of the world’s 6.9 billion people were connected to the Internet. By 2015, market research firm IDC estimates, there will be 2.7 billion users. Thus, the NSA’s need for a 1-million-square-foot data storehouse. Should the agency ever fill the Utah center with a yottabyte of information, it would be equal to about 500 quintillion (500,000,000,000,000,000,000) pages of text.

The data stored in Bluffdale will naturally go far beyond the world’s billions of public web pages. The NSA is more interested in the so-called invisible web, also known as the deep web or deepnet—data beyond the reach of the public. This includes password-protected data, US and foreign government communications, and noncommercial file-sharing between trusted peers. “The deep web contains government reports, databases, and other sources of information of high value to DOD and the intelligence community,” according to a 2010 Defense Science Board report. “Alternative tools are needed to find and index data in the deep web … Stealing the classified secrets of a potential adversary is where the [intelligence] community is most comfortable.” With its new Utah Data Center, the NSA will at last have the technical capability to store, and rummage through, all those stolen secrets. The question, of course, is how the agency defines who is, and who is not, “a potential adversary.”

 

[Zelig]

There's nothing in that article that supports the Utah data center having anything nearing that scale of storage capacity.

The total amount of storage space in the entire world is probably below 5 zettabytes still.

http://en.wikipedia.org/wiki/Zettabyte

 

[Formaldehyde]

Ironically, all you have to do to arrive at those numbers is some "back of hand math" based on known estimates of current internet traffic. But the actual capacity is obviously classified.

 

[Paul in Saudi]

Seems that they have a lot of room to expand.

 

[Zelig]

Ironically, all you have to do to arrive at those numbers is some "back of hand math" based on known estimates of current internet traffic. But the actual capacity is obviously classified.

It doesn't matter how classified they make the actual capacity, the numbers you're suggesting are just completely impossible.

The F-22 Raptor has a bunch of classified information about its specs, but I can say with some certainty that it can't fly at 1,000,000 km/h.

 

[Paul in Saudi]

OK, then. You're safe.

 

[Zelig]

OK, then. You're safe.

I don't object to anything you've said, I just object to Forma's estimates of their technical capacities being more than the entire world's storage and computational resources.

 

[Formaldehyde]

It doesn't matter how classified they make the actual capacity, the numbers you're suggesting are just completely impossible.

Perhaps you should take up your concerns with the editors of Wired and elsewhere who claim just the opposite.

I don't object to anything you've said, I just object to Forma's estimates of their technical capacities being more than the entire world's storage and computational resources.

And now they are "my" estimates.

Where are your sources who claim it is sheer nonsense?

 

[Zelig]

Perhaps you should take up your concerns with the editors of Wired and elsewhere who claim just the opposite.

They do no such thing.

And now they are "my" estimates.

Where are your sources who claim it is sheer nonsense?

I linked the wiki page on zettabytes, claiming that the NSA has more storage space than the entire world is pretty obviously nonsense.

 

[Borachio]

They do no such thing.



I linked the wiki page on zettabytes, claiming that the NSA has more storage space than the entire world is pretty obviously nonsense.

Well, I don't know. That article claimed the entire web was 0.5 zettabytes in 2009. Which in computing terms is in the dark ages.

Also, there is a possibility (if not inevitability) that NASA has the most advanced technology available, so 5 zettabytes in one facility now, although a great deal, doesn't stretch my credibility beyond breaking point (it would be in their interests to have such a thing). Unlike an aeroplane travelling at 10^6 km/hr - they're different kinds of technology.

But...what we don't know we don't know.

edit:

As of 2009, the entire World Wide Web was estimated to contain close to 500 exabytes.[11] This is one half zettabyte.

Yup. (I thought I'd check.)

 

[Zelig]

Well, I don't know. That article claimed the entire web was 0.5 zettabytes in 2009. Which in computing terms is in the dark ages.

Also, there is a possibility (if not inevitability) that NASA has the most advanced technology available, so 5 zettabytes in one facility now, although a great deal, doesn't stretch my credibility beyond breaking point (it would be in their interests to have such a thing). Unlike an aeroplane travelling at 10^6 km/hr - they're different kinds of technology.

But...what we don't know we don't know.

edit:
Yup. (I thought I'd check.)

Oh damn, I had more of a response and closed the tab by accident...

In short, you can store about 50 PB per 1000 square feet, the Utah center is 100,000 square feet, giving them about 50,000 PB, or 0.05 ZB of total storage capacity. This is already an astoundingly huge amount, but 5 ZB would take a facility 100x larger.

http://blog.backblaze.com/2009/11/12/nsa-might-want-some-backblaze-pods/
http://blog.backblaze.com/2013/06/0...r-prism-surveillance-data-heres-what-we-know/

5 ZB would take about 1000 data centers, covering the whole of New York City.

 

[Paul in Saudi]

Well (to argue against my own argument), the security agencies of the world not only want to keep everything out there now, they want to keep it forever. Ideally, the would want to be able to call up all the e-mails sent by someone of interest fifty years ago.

It would be a colossal task. But the way technology is going. it is not unimaginable.

 

[Formaldehyde]

It doesn't matter how classified they make the actual capacity, the numbers you're suggesting are just completely impossible.

You mean besides stating definitively that the NSA needs that sort of capacity in order to achieve their objectives?

And again, it is not me who is "suggesting" them. Now is it?

 

[Zelig]

You mean besides stating definitively that the NSA needs that sort of capacity in order to achieve their objectives?

And again, it is not "me" who is "suggesting" them. Now is it?

Yes, the NSA has definitively stated that they need more than the entire world's GDP worth of storage capacity.

They've at no point indicated that it's within their near-term capabilities of achieving this.

Lots of people would love to have lots of non-existent technologies, stating that you need them to achieve your objectives doesn't magically make the technologies exist.

 

[Borachio]

Oh damn, I had more of a response and closed the tab by accident...

In short, you can store about 50 PB per 1000 square feet, the Utah center is 100,000 square feet, giving them about 50,000 PB, or 0.05 ZB of total storage capacity. This is already an astoundingly huge amount, but 5 ZB would take a facility 100x larger.

http://blog.backblaze.com/2009/11/12/nsa-might-want-some-backblaze-pods/
http://blog.backblaze.com/2013/06/0...r-prism-surveillance-data-heres-what-we-know/

5 ZB would take about 1000 data centers, covering the whole of New York City.

I agree the figures aren't promising. And in addition, what they spend now will be out of date in 2 years time by something much cheaper. Perhaps.

It looks more likely to me that the producers of information (you and me and everyone) are producing the stuff at a much faster rate than anyone is capable of keeping up with. And I think the authorities are probably alarmed about it.