I have to admit, I'm of mixed minds about this.
On one hand, I think it's possible for good people to do and say bad things. But on the other, people are what they routinely do. If someone does a lot of kind things, we don't think of them as merely a person who does kind things, but as a kind person. If someone says bigoted things, we tend to call them a bigot.
So when a person hears someone tell them that they said something racist, for most people, it's just one tiny step away from being called a racist outright, if not perhaps the same thing. And there is a tendency for people to completely write off a person as irredeemable once their flaws come to light - just look at the routine headlines of people losing their jobs or celebrities losing gigs because of something they said. Combined, this means that a lot of people are told, "What you said is prejudiced," but hear, "You are a racist who should be ostracized."
I'm torn, because while I do believe it's possible and desirable to tell someone what they said is bad without calling them bad people, I can see why people would interpret that as a direct accusation and attack on their worth.