This presumes someone like Milo is intending to engage in an intellectually honest way, i.e. is attempting to present a reasoned case for his beliefs and then engage in a give-and-take with the audience and try to persuade to his point of view. The problem with trolls like Milo is they have no interest in engaging this way. That's part of the problem - where the 1964 Nazi was attempting intellectual rigor in his presentation, Milo is not.
Part of the ethos of "free speech" on college campuses is that it be reasoned speech. This is the part I think most people miss - the marketplace of ideas doesn't mean everybody is invited to share whatever ideas they have, regardless of how damaging or poorly thought out they are. It's fine to have dissenting views, it's fine to present differing opinions, but you better be prepared to do what the 1964 Nazi did - present your argument in a reasoned way and then defend it. Inviting an anti-intellectual provocateur for no other reason than to shock and titillate the campus reactionaries is not in keeping with that tradition, at all. The students are absolutely right to stand up against that.