The problem with all the comments to the effect of "punching Nazis could have prevented them from reaching power" is that there were lots of left-wing people punching Nazis in street battles in the years before their seizure of power. It didn't help, because the Nazis were better at violence than the Communists and they were clearly preferred over them by the people with power in late Weimar Germany.
In our particular case, neo-Nazism and (more accurately) overt white nationalism are still fringe positions, if worryingly less fringey than they were up until the past couple of years. Punching Richard Spencer makes people feel good, but it mostly just gives him the exactly the publicity he's looking for.
If anyone can show how left-wing activists punching far-right counterprotesters actually accomplishes anything beneficial for the left, I'm all ears. But under the current circumstances I'm convinced it's a very poor strategy. Further, if push comes to shove, it needs to be remembered that most of the people with weapons (ranging from the police to private citizens) are right-wing. Violence is not something the American left would win at, and it should be avoided from a tactical perspective.