[RD] What we buy can be used to predict our politics, race or education

Cutlass

The Man Who Wasn't There.
Joined
Jan 13, 2008
Messages
47,758
Location
US of A
What we buy can be used to predict our politics, race or education — sometimes with more than 90 percent accuracy


by Andrew Van Dam July 9 at 10:44 PM

The cultural divide is real, and it’s huge. Americans live such different lives that what we buy, do or watch can be used to predict our politics, race, income, education and gender — sometimes with more than 90 percent accuracy.

It turns out that people are separated not just by gun ownership, religion and their beliefs on affirmative action — but also by English muffins, flashlights and mustard.

To prove it, University of Chicago economists Marianne Bertrand and Emir Kamenica taught machines to guess a person’s income, political ideology, race, education and gender based on either their media habits, their consumer behavior, their social and political beliefs, and even how they spent their time. Their results were released in a new working paper from the National Bureau of Economic Research.

The duo trained their algorithms to detect patterns in decades of responses to three long-running surveys, each with between 669 and 22,093 responses per year. The surveys were tuned and filtered to be consistent over time, which allowed Bertrand and Kamenica to measure how America’s cultural divides have evolved.

To determine how accurately cultural factors predicted a person’s race, education or income tier, the duo tested their algorithms on subsets of the data that the programs had never seen. To keep it fair, they omitted variables that would have been a dead giveaway — if they were predicting whether someone was liberal or conservative, for example, they wouldn’t allow the algorithms to consider the answer to “Which political party do you support?”

Nevertheless, some results are obvious, which indirectly proves that their approach can detect tangible divides. Spending predicts gender with almost perfect accuracy, for example, because men don’t buy nearly as much mascara as women do, and women buy much less aftershave/cologne than men do. But others are revelatory: White people and black people are almost as different in their spending habits as rich people and poor people are, for example.


Differences in social attitudes between liberals and conservatives have been widened over time, Bertrand and Kamenica found. The gap in social attitudes between whites and nonwhites has fallen slightly, but the difference in consumer behavior between races has grown.

Race
In the world of television in 2016, some of the top-ten predictors of whiteness were watching “Rudolph the Red-Nosed Reindeer,” “American Pickers,” “The Big Bang Theory” and the Kentucky Derby. If we’re looking at specific brand names, the top 10 included Thomas’ English muffins, Sweet Baby Ray’s barbecue sauce and Stove Top stuffing.

More generally, in consumer products, the best predictor of whiteness was whether someone owned a pet — followed closely by whether they owned a flashlight. Many of the differences appear to be correlated with wealth and homeownership, areas in which America suffers from vast racial disparities.

[‘We haven’t made any progress’: Black homeownership is stuck near 30-year lows]

The Federal Reserve has found that the median net worth of a white household in 2016 was 9.7 times greater than that of a black one.


Each analysis is binary, meaning that although the authors frame everything in terms of predicting whether someone is white, or high income, or male, the direct opposite is equally true. In other words, “doesn’t own a pet” predicts that someone isn’t white just as strongly as “owns a pet” predicts that someone is white.

To maintain statistical integrity, the authors were able to break the population into only two categories, “white” and “nonwhite,” which may hide differences across a large and diverse population.

Within the surveys they analyzed, social attitudes and media habits were almost as closely linked to race as consumer behavior was. The racial differences in social attitudes were particularly notable.

By a relatively large margin, the view that best predicted being white in 2016 was “approve of police striking citizens.” In previous decades, the best indicator of whiteness was saying that the government was spending the right amount (or too much) on improving the condition of black residents.


On other issues, the gap has closed. “In 1976,” the authors write, “one could correctly predict race based on views towards government spending 74 percent of the time but by 2016 this number was down to 56 percent.”

Income
Attitudes toward police violence are only a few percentage points less effective in predicting high (in the top 25 percent) income than they are in predicting whiteness. The overlap shows how closely related race and income are, probably because of historical disparities and continuing problems with racial bias.

Race aside, consumer behavior is strongly linked to income level. In 1992, Grey Poupon mustard predicted income better than any other brand. By 2016, its place as the key signifier of the country’s economic and cultural divide had been taken by Apple’s iPhone — which the researchers found to be a much clearer signifier of income than the condiment had been.

Politics
Because of limitations in the media and consumer components of the survey they used, researchers couldn’t get reliable data on the liberal-conservative split that was more recent than 2009. But differences up to that time include some of the most interesting findings in the survey.

They start with superficial differences: If someone went to Arby’s or Applebee’s or used Jif peanut butter, you might guess they were conservative. If they didn’t own fishing gear or use ranch dressing, but drank alcohol and bought novels? Probably a liberal.


The researchers find that, across almost every dimension, America’s cultural divide has remained constant. Yes, high-income households buy different things from low-income ones, and white Americans and black Americans watch different television programs and movies. We’re divided. But we always have been and, despite popular narratives to the contrary, it’s not getting worse.

“What’s really striking to me,” Kamenica said, “is how constant cultural divisions have been as the world has changed.”

But there’s one exception. And it’s a big one. The ideological difference between conservatives and liberals is wide and growing.

“This is not a new phenomenon,” Kamenica said. “For the past 40 years, liberals and conservatives are disagreeing more each year. On every topic, liberals and conservatives are disagreeing more than they used to.”

And their analysis of television-watching habits indicates the nature of America’s media divide may be changing, even if its size isn’t. In 2001, you could predict that someone was conservative if he or she hadn’t watched the Academy Awards or “Will and Grace.” By 2009, those cultural signifiers had been replaced by three major Fox News programs: “The O’Reilly Factor,” “Fox and Friends” and “Hannity & Colmes.”

According to Bertrand, cultural factors such as television and movies matter because of how they enable (or disable) conversation and exchange between neighbors of different backgrounds and viewpoints.

“We feel this sharing of culture is important,” Bertrand said. “There is a lot of focus in economics on human capital and social capital. We must also think about cultural capital and the importance it has in our ability to get along.”

Methodology
The researchers' machine-learning approach combined elastic net, regression tree and random forest analysis. As input, they used specific responses to three major surveys:

  • The American Heritage Time Use Study, now conducted by the Labor Department, asked between 669 and 10,210 U.S. residents per year about the time they spend on activities such as sleeping, working, yardwork and video games.
  • The General Social Survey, conducted by NORC at the University of Chicago, polls between 1,093 and 3,735 people per year on their attitudes toward myriad social issues such as trusting others, marijuana legalization, and approving of police striking a man.
  • Mediamark Research Intelligence queries between 15,352 and 22,033 adults a year about thousands of consumer behaviors, such as owning a dishwasher, reading “Architectural Digest” and watching the “Teenage Mutant Ninja Turtles” movie.

https://www.washingtonpost.com/busi...ehind-cultural-divide/?utm_term=.4f12450bfa61


The interesting take on this is that after all these years we are in some ways as much different people inhabiting the same country as we ever were. But in other ways, it's worse. And the part that it's worse tracks back to Fox news.
 
Maybe these people can make the Google algorithms better. I still get...hilariously bad recommendations.
 
In school in 2006 we had a guest lecturer tell us about how the California Republican Party used grocery shopping data to figure out who to send literature to and how many times they would likely have to mail them to get votes. They had one profile for everyone by the coast and another purchase profile for everyone inland. He said that the Democratic Party and labor unions were already doing this with greater sophistication.
 
Yeah, in the 90's I was consulting at a market research company that was mining scanning data from grocery stores. It was amazing to see some of the correlations that were made. And of course, some of the silly ones that were results of other factors.
 
More generally, in consumer products, the best predictor of whiteness was whether someone owned a pet — followed closely by whether they owned a flashlight. Many of the differences appear to be correlated with wealth and homeownership, areas in which America suffers from vast racial disparities.

what the heck, minorities don't own flashlights? You don't have to own a home to need a flashlight and they aren't expensive. That it just such a weird predictor.
 
That was a really interesting article. Who would have ever thought you could know so much based on whether someone owns a flashlight.

By a relatively large margin, the view that best predicted being white in 2016 was “approve of police striking citizens.” In previous decades, the best indicator of whiteness was saying that the government was spending the right amount (or too much) on improving the condition of black residents.

This is really disturbing. :thumbsdown:
 
So, "things that best predict a person is liberal" are just marginally better at it than coin toss?
Color me not particularly impressed. :rolleyes:
 
So, "things that best predict a person is liberal" are just marginally better at it than coin toss?
Color me not particularly impressed. :rolleyes:

Isn't it the combination of many factors that boost confidence in a prediction? It shouldn't be surprising that any one item isn't a strong predictor, especially since most things are sold under the presumption of providing at least some utility.
 
Targeted ads work just fine if you don't attempt to obscure your web presence.

All of my ads on Reddit are relevant to my interests. I ate a whole lot of pizza during the playoffs because of a Dominos deal code only offered through ads. :D
 
Honestly, I think the entire ad industry is a house of cards, with nobody realizing that targeted ads aren't actually effectively.

I've read that the whole thing is largely based on automated fake traffic anyway.
 
Isn't it the combination of many factors that boost confidence in a prediction? It shouldn't be surprising that any one item isn't a strong predictor, especially since most things are sold under the presumption of providing at least some utility.
Sure, if enough of those factors all point in one direction, it boosts confidence. But how often does that happen in practice?
I've recently bought both novels and fishing gear, for example. :D
 
My only problem with targeted ads is that I usually get them for (one time purchase) products I've already purchased. ;)
 
My only problem with targeted ads is that I usually get them for (one time purchase) products I've already purchased. ;)

Oh yes, that always brings a smile to my face.
You bought a smartphone. You like smartphones? Here, look at this smartphone ! Wanna buy it ?
Ehm, no. I just got one. Ask me again in five years.

Youtube is really annoying me right now with all the recommendations for Dave Rubin, Jordan Peterson, and related trash.
 
My wife tells me that it's more effective on women since they're more likely to take more time searching before they buy. I usually find what I want and make the decision in less than an hour. Don't know if there's research that backs it up, but if my wife claims it, I have to believe it is. ;)
 
Targeted ads work just fine if you don't attempt to obscure your web presence.

All of my ads on Reddit are relevant to my interests. I ate a whole lot of pizza during the playoffs because of a Dominos deal code only offered through ads. :D

Gross. Humanity doesn't really lose anything if Dominoes just goes away.

I've read that the whole thing is largely based on automated fake traffic anyway.

Doubtful – there is a ton of fake traffic, but it's tough to fake Google/Facebook accounts at scale, and they tend to employ smarter people than ad fraudsters, while sucking up like 90% of web ad revenue.

My wife tells me that it's more effective on women since they're more likely to take more time searching before they buy. I usually find what I want and make the decision in less than an hour. Don't know if there's research that backs it up, but if my wife claims it, I have to believe it is. ;)

That seems backwards. If a product is fully and competently researched, ads cannot have an influence on the purchase decision.
 
Oh yes, that always brings a smile to my face.
You bought a smartphone. You like smartphones? Here, look at this smartphone ! Wanna buy it ?
Ehm, no. I just got one. Ask me again in five years.

Youtube is really annoying me right now with all the recommendations for Dave Rubin, Jordan Peterson, and related trash.

Youtube is actively dishonest trash. The idea of going with "suggestions" rather than recent uploads from user subscriptions as priority things to display has no coherent justification if the motive is to benefit the user. I don't subscribe to stuff I don't want to watch, and I don't subscribe to channels because I want to delay watching new content from them.

If I'm out of stuff I want to watch, okay put some suggestions below the sub feed. Maybe I'll find a new channel I'm interested in.

That seems backwards. If a product is fully and competently researched, ads cannot have an influence on the purchase decision.

I doubt most purchase decisions are "fully" researched. Competently is a matter of opinion, but I suspect that exposing yourself to more ads gives more opportunity for ads to influence you.
 
That seems backwards. If a product is fully and competently researched, ads cannot have an influence on the purchase decision.

I see what you're saying, but I don't think it's right - to take the smartphone example used earlier, if I look at dozens of different smartphones before choosing which one to buy then I'm less likely to run into the situation where I'm getting ads for a thing I already bought.

Ads you're seeing during the decision process are more likely to affect you than ads you're seeing after you already bought something, no?
 
I once had a colleague that I knew was home schooled. Based on that I was able to deduce that he:
  • Drove a pick up truck
  • Owned a shotgun
  • Watched fox news
  • Listened to conservative talk radio
  • Had a septic system and water tank (i.e. off municipal water)
  • Had satellite TV (dish network)
  • His mother did not wear (was not allowed to wear) pants
  • Did not believe in evolution
  • Believed in the young earth hypothesis and intelligent design
  • Lived next to the woods at the end of a cul-de-sac (really proud of that one)
  • Is an evangelical christian
  • Thought the end times were coming and war would break out in Israel 'soon'

He was pretty pissed and complained that I was stereotyping him. Trends stand out; it shouldn't surprise anyone that buy habits correlate with political bent.
 
Interesting how much the debate about abortion dominated the liberal-conservative-divide even in 2016. Not only is is the strongest predictor, it is also represented a total of 4 times in the top 10:

Allow abortion for single women - 71.8%
Allow abortion for low income women - 71.2%
Allow abortion for married women - 68.8%
Allow abortion for rape victims - 65.0%

It's also interesting how much sense the thing about rape victims makes. Even the average conservative is more likely to say that a rape victim should have the right to get an abortion. I wonder if that means that the child must be a product of that rape though, or whether being a rape victim just gets you the Abortion Card Black Edition. The former would make a lot more sense, but it's phrased in a way that one could almost think that it's the latter.


Don't understand the 'single woman'-part though, do conservatives not want them to have abortions, or do liberals really, really want single women to have abortions so they don't become single mothers? Is that hidden wokeness among the liberals, do they understand how bad it is for a child to grow up without a father? Or just pure distaste on the part of the conservatives?

The real interesting questions.
 
Top Bottom