I'm not real big on the waving of "but free speech" as if it were a magic wand, especially when the application is particularly dubious. If I don't like what someone has to say and I cut their tongue out with a pocketknife I haven't "infringed their right to free speech." I'm not the government. I don't represent the government. There is absolutely nothing about the statement "the congress shall pass no law infringing upon the right to free speech" that has the slightest thing to do with me.
People who wave the "but free speech" wand are the ones that always seem to me to not only "not be big on" but to actually not even understand the concept.
Oh I understand the concept just fine. And it does have everything to do with you as an individual because you are likely to vote for politicians who share your views on what speech should be allowed and what speech shouldn't. That means people like you are a direct threat to the Constitutional rights of everyone, including yourself. That's why it's important to get people like you to understand that everyone has the right to express their views without fear of reprisals, even Nazis.