5 zettabytes of data at Buffdale

[peter grimes]

ITT: Russia Today is taken as read.

Spoiler :

Bluffdale?

Yeppers

 

[Kozmos]

NSA could potentially solve one public key crypto problem but not all of them.

 

[History_Buff]

I think that means they can very well decrypt it using their own highly sophisticated and extremely secret techniques, as well as by using the fastest and most proprietary computers ever invented by far. YMMV.

But that, like, mathematically, isn't possible. Brute forcing one key doesn't get you anything else. And while there are a few methods that work against some keys, properly generated keys tend to be immune to any practicable attack.

Unless you think they have a quantum computer, which pretty much every big name computing firm is working on, and none with any large degree of success.

You are putting entirely too much faith in the NSA.

 

[Grisu]

well, regardless of the outlandish claim of the amount of data stored (a zetta byte really is a huge number, which seems unrealistic) or that actually doing anything with that amount of data would be a huge challenge (and no, they don't have processors that are lightyears beyon what the average joe uses) the trend to store more and more data about its citizens is indeed worrying.

But I'm not really surprised at this, since especially anglo-saxon countries seem to have next to no concept of data privacy like there is in the german speaking areas.

At a conference I was this spring there was an american guy (emc.com) speaking on a keynote about "Harnessing the power of Big Data" and eventually he came to data being stored about individuals and how great a benefit you can get from that, etc. He was reallly enthusiastic about it and I think he expected his audience to be enthusiastic as well...except that most were from Germany/Switzerland/Austria and more or less were shocked into silence for the perceived violation of data privacy

Or take a look how surprised Google was at the opposition to street view in several german speaking countries.

I work for the IT department of a swiss canton (our version of a state/province/whatever you call it) and I know the kind of regulations and limitations we have when storing data about citizens. It's quite a hassle sometimes but in the end, I believe such regulations are worth having and IMHO the US should be adopting similar regulations as well.

 

[nc-1701]

But that, like, mathematically, isn't possible. Brute forcing one key doesn't get you anything else. And while there are a few methods that work against some keys, properly generated keys tend to be immune to any practicable attack.

Unless you think they have a quantum computer, which pretty much every big name computing firm is working on, and none with any large degree of success.

You are putting entirely too much faith in the NSA.

Pretty much this, I'm a first year math grad student with minimal cryptography or programming background. But I could easily implement an arbitrarily huge RSA, including writing Maple code to do all the necessary calculations in a weekend. That I could guarantee would take months to years for the NSA to crack. Note it may run very slowly for me of course, since I would be using a large key etc.

RSA is well known, and essentially unbreakable, Math > NSA.

 

[History_Buff]

And from what I gather, AES is actually even better.

 

[innonimatu]

Great, now I can live with the assurance that my great contributions to philosophical progress posted here at CFC will be forever preserved, thanks to the american government. And indeed all the extraordinary knowledge with which I have have been gracing the Internet, all for the benefit of future generations. Immortality at last!

Thank you, american tax payers! It's well worth your money. A work of the highest order, for the benefit of humanity, surely.

 

[FriendlyFire]

Ah, so, all the things Obama is doing poorly at, it is all because of Bush.
Got it.
For the record, Iraq was already fixed (as it was going to get)... Afghanistan has been a failure with no picture of "fixing" in the future... the economy still sucks... Gitmo is still open... renditions still occur... drone missile use has gone up... national debt has gone up by something like $6T...

BUT
We got ACA! Whooty who!

No Bush did a fantastic job as President, which is why Republicans parade him around everywhere to remind everyone, Republicans especially about the eight years that was Bush.

And you forgot to remind us how 4 US people were murder in Benghazi because Obama is such an incompetent moron, lier and brain damaged surrender monkey.

Republicans sudden worried now about drones ? How about two botched invasions which killed tens of thousands, US hemerage blood and treasure while giving tax cuts to the rich ?

Obama might be mediocre, but he is a dam sight better then the utter disaster that was the Bush Presidency.

 

[innonimatu]

except that most were from Germany/Switzerland/Austria and more or less were shocked into silence for the perceived violation of data privacy

Or take a look how surprised Google was at the opposition to street view in several german speaking countries.

I work for the IT department of a swiss canton (our version of a state/province/whatever you call it) and I know the kind of regulations and limitations we have when storing data about citizens. It's quite a hassle sometimes but in the end, I believe such regulations are worth having and IMHO the US should be adopting similar regulations as well.

Ah, a fellow worker in the same field!

I've held the same strong position on data protection. But, despite my sarcasm above, I've come to see that the people who told us that "privacy is dead" were actually being honest, not only motivated for some desire to advance that agenda. The use of the data collection and storage technology cannot in practice be prevented by law. Data mining can be somewhat controlled, or at least its effects can, but it depends very much on what people want their data used for. Despite my sarcasm above I'm not actually opposed to all that data interception - so long as it comes to be done with public knowledge and access.

You can't enforce privacy if people voluntarily give their data away. And therein lies the problem: there are many objective advantages to giving away one's data. It is rational to do so. Some things will be kept private, obviously: you don't want other people to know details that'll allow them to perfectly impersonate you (though even defending that ground will prove problematic). But the vast majority of what we now hold "private" is hold to be private for no better reason that... tradition! It has always been so, so it must remain. Because if people know this or that about me they'll use it against me. Because I'm uncomfortable with having it know. Because I'll feel constrained in my actions if I feel under surveillance. Because knowing so much about me they'll be able to manipulate me. Those are basically the arguments.

Three of those depend on asymmetry of information: loss of privacy by one individual where privacy is a social norm is a problem for that individual? But what if a much smaller sphere of privacy were the social norm? What blackmail value would some of that private data have if everyone had it public? What discomfort would it cause to share data when everyone is sharing it? Who will do the surveillance in a world flooded with available data? Surely not any strong AI, because that is and will remain for the foreseeable future in the realm of SF.
So the one argument for privacy is the fear of manipulation. But it too has a scale problem in a world swamped by data. In order to actually use the data economically you have to aggregate it: it's too costly to customize your efforts for the individual profiles of each and every person, even if you have the required data. You can target adds for consumers or political propaganda for citizens? So what? It has been done ever since newspapers became common - at least. It has always been, and will remain, a cat-and-mouse game: people get good at filtering those adds.

Some things have been done well in Europe: for example, the mandate for allowing access to one's data held in databases. And why not mandate unfettered access to all information in such databases, once the cost of providing that service becomes low enough? Others not so well: the unenforceable push for the "right to be forgotten". Perhaps it is good that such a right be recognized, but no one will have the power entirely delete something, unless the Internet and all its users were to become very heavily regulated.

Perhaps it is time to stop fearing the database and embrace it?

 

[AlpsStranger]

This reeks of exaggeration to me, but that still doesn't excuse the amount of this crap that I don't doubt for a second has gone on under both Bush and Obama. This kind if BS is a bipartisan problem now.

Unfortunately, everyone who gets really fired up about it seems to want to burn the entire country to the ground and read Bibles and churn butter by firelight. Not exactly a proportionate cure.

 

[History_Buff]

I don't know why everyone is in such a rush to blame the administrations (Bush or Obama). Assuming this were true, it seems to me far more likely that some mission statement came down to the effect of "monitor internet traffic for national security threats" and some civil servant thought the best way to do that was simply to store everything. After all, it wouldn't be their problem to sort it all out later.

I say this because it happens in the corporate world all the damn time. We tend to store all our data, in both code and compiled binary form, forever, despite the fact that this data has zero usefulness to us. If we stored 31 days of source control backups, that would be enough. But it's corporate policy, so we all do it.

 

[innonimatu]

I don't know why everyone is in such a rush to blame the administrations (Bush or Obama). Assuming this were true, it seems to me far more likely that some mission statement came down to the effect of "monitor internet traffic for national security threats" and some civil servant thought the best way to do that was simply to store everything. After all, it wouldn't be their problem to sort it all out later.

I say this because it happens in the corporate world all the damn time. We tend to store all our data, in both code and compiled binary form, forever, despite the fact that this data has zero usefulness to us. If we stored 31 days of source control backups, that would be enough. But it's corporate policy, so we all do it.

I have no doubt whatsoever that several states, the US first and foremost, are storing as much communications data as they can. And the cost and technical challenge of doing so are actually quite manageable. The usefulness of it, though, is as you said: sorting anything useful out of it is just not going to be done. In the end what is collected on purpose from known sources is what will be used for something, I'm guessing. The vast dragnet that collects everything and produces mostly meaningless statistical noise... creates employment and sales for the operators and sellers of the systems involved! And covers the asses of some bureaucrats in the 2intelligence complex". And creates profitable career opportunities once they retire into the private sector. Same old stuff.

 

[Grisu]

Ah, a fellow worker in the same field!

I've held the same strong position on data protection. But, despite my sarcasm above, I've come to see that the people who told us that "privacy is dead" were actually being honest, not only motivated for some desire to advance that agenda. The use of the data collection and storage technology cannot in practice be prevented by law. Data mining can be somewhat controlled, or at least its effects can, but it depends very much on what people want their data used for. Despite my sarcasm above I'm not actually opposed to all that data interception - so long as it comes to be done with public knowledge and access.

You can't enforce privacy if people voluntarily give their data away. And therein lies the problem: there are many objective advantages to giving away one's data. It is rational to do so. Some things will be kept private, obviously: you don't want other people to know details that'll allow them to perfectly impersonate you (though even defending that ground will prove problematic). But the vast majority of what we now hold "private" is hold to be private for no better reason that... tradition! It has always been so, so it must remain. Because if people know this or that about me they'll use it against me. Because I'm uncomfortable with having it know. Because I'll feel constrained in my actions if I feel under surveillance. Because knowing so much about me they'll be able to manipulate me. Those are basically the arguments.

Three of those depend on asymmetry of information: loss of privacy by one individual where privacy is a social norm is a problem for that individual? But what if a much smaller sphere of privacy were the social norm? What blackmail value would some of that private data have if everyone had it public? What discomfort would it cause to share data when everyone is sharing it? Who will do the surveillance in a world flooded with available data? Surely not any strong AI, because that is and will remain for the foreseeable future in the realm of SF.
So the one argument for privacy is the fear of manipulation. But it too has a scale problem in a world swamped by data. In order to actually use the data economically you have to aggregate it: it's too costly to customize your efforts for the individual profiles of each and every person, even if you have the required data. You can target adds for consumers or political propaganda for citizens? So what? It has been done ever since newspapers became common - at least. It has always been, and will remain, a cat-and-mouse game: people get good at filtering those adds.

Some things have been done well in Europe: for example, the mandate for allowing access to one's data held in databases. And why not mandate unfettered access to all information in such databases, once the cost of providing that service becomes low enough? Others not so well: the unenforceable push for the "right to be forgotten". Perhaps it is good that such a right be recognized, but no one will have the power entirely delete something, unless the Internet and all its users were to become very heavily regulated.

Perhaps it is time to stop fearing the database and embrace it?

Dr. Innonimatu: How I Learned to Stop Worrying and Love the Bomb Database

j/k

despite of what I wrote earlier, my position on the matter is not that strong. While I think that the awareness in the english-speaking-sphere on such matters is too low, I sometimes feel that we go overboard with data privacy

I do agree that it probably is a losing battle to limit the amount of data gathered, but what we can do is raise awareness to the principle that the data should be the property of the person it is on. As such, transparancy about who gathers what information about me must be raised and the information they have should be disclosed to me upon request. That would at least somewhat offset the asymetry of information. What you described can only work if the data that governments/companies gather is public. Most often, that is not the case since they want to use it to their advantage (if you know more about a customer than the competition you're at an advantage).

 

[kochman]

No Bush did a fantastic job as President, which is why Republicans parade him around everywhere to remind everyone, Republicans especially about the eight years that was Bush.

Ummmm, ok, nothing to add there I see...

And you forgot to remind us how 4 US people were murder in Benghazi because Obama is such an incompetent moron, lier and brain damaged surrender monkey.

The dust hasn't settled on Benghazi yet, hard to say what happened, so I won't blame him.
*Liar

Republicans sudden worried now about drones ?

Most of them seem fine with it, actually. Not sure where you got this.

How about two botched invasions which killed tens of thousands, US hemerage blood and treasure while giving tax cuts to the rich ?

Ah, Bush had two botched invasions (even though Iraq worked out, though we shouldn't have gone there)... so, Obama, feel free to drop drone missiles at your whim anywhere in the world.

Obama might be mediocre, but he is a dam sight better then the utter disaster that was the Bush Presidency.

So what?
He isn't really a *damn sight better, either... he's slightly better. If he were doing such a bang up job, he wouldn't have lost 6.5 million votes in a growing electorate from 2008 to 2012.

But, perhaps some people just react harshly to any criticism of Obama... damn the facts if they interfere with agenda. No arguing with that.

 

[Farm Boy]

Well, if the NSA has decided it needs to essentially store the internet, it probably is figuring it will get some mileage out of it or it wouldn't be getting built. Either that or somebody's cousin that builds these things needed a new Porsche. Or both. Take your pick on what you want to believe, but it's probably both.

 

[Rashiminos]

It would be one of the most closely guarded secrets in the history of this country.

Close. Yes. Guarded. Definitely. Most. No.

 

[kochman]

Well, if the NSA has decided it needs to essentially store the internet, it probably is figuring it will get some mileage out of it or it wouldn't be getting built. Either that or somebody's cousin that builds these things needed a new Porsche. Or both. Take your pick on what you want to believe, but it's probably both.

Exactly!

Also, there is a push now to have "black boxes" in everyone's cars.
That would mean, 1 - GPS in everyone's car (that they can't use), 2 - record of all conversations...

How far do we go?

Do we really need black boxes in EVERY car?

I guess the answer is, if Obama says so.

 

[JollyRoger]

What is wrong with introducing a peroxide blonde to the backseat of your car?

 

[dutchfire]

Well, if the NSA has decided it needs to essentially store the internet, it probably is figuring it will get some mileage out of it or it wouldn't be getting built. Either that or somebody's cousin that builds these things needed a new Porsche. Or both. Take your pick on what you want to believe, but it's probably both.

The 2009 internet was estimated at 487 billion GB (http://www.guardian.co.uk/business/2009/may/18/digital-content-expansion), or 0.487 zettabytes. So this facility will store 10 internets (2009), which, accounting for the speed of growth of the internet, will probably be around 1 internet around the time this is finished.

I love using internet as a unit of storage space.

 

[History_Buff]

Also, there is a push now to have "black boxes" in everyone's cars.
That would mean, 1 - GPS in everyone's car (that they can't use), 2 - record of all conversations...

• Don't assume GPS is made of magic. It isn't. Anything that's going to be mass installed into cars definitely won't be.
• How exactly are they going to store all that data hmm? Storing a complete audio record would fill any reliable storage medium immediately.

What these things are actually for is storing the last 1-3 minutes of data before an impact event to aid collision investigators.