I dunno, I don't think there should be limits to free speech under any circumstances, but thats my own opinion and i can understand where you're coming from.
For example i would much rather defend a nazi's right to deny the holocaust than petition for ernst zundel to be exiled (think the fact that holocaust denial and other forms of revisionist thought are "banned" in some places is as absurd as holocaust denial and historical revisionism in the first place). I dont think thought or speech should ever be regulated, only actions. The problem with "the limits acceptable of a democratic society" is that the "limits" are almost impossible to identify without creating some kind of double standard somewhere.
Thought should not be regulated. Public speech, however, is a form of action that can, in fact, cause harm to people, and it is the government's place to regulate interaction between citizens.
As for identifying the limits, there is a clearly established judicial test with regard to that.
1. The law must address a substantial (ie, real, serious) objective.
This has two practical effects. The first is that any restriction to fundamental rights must be a mean to a (non-frivolous) end, not an end in itself. The second is that this end
cannot, itself, contradict the charter. You can restrict free speech in the name of protecting minorities, but you cannot restrict free speech in the name of preventing criticism of the government (you
might get away with "unity in time of war". Maybe.).
2. The means must be connected to the objective.
A fairly straightforward one, really - you can't infringe on somebody's rights if doing so has nothing to do with the stated aim of the law. (You can restrict hate speech in a law aimed to protecting minorities. You can't ban criticism of government in a law aimed at protecting minorities)
3. The impairment of rights must be minimal.
Bit more subjective, but basically, this asks the questions: could the objectives of the law be achieved with lesser restrictions? If so, then these lesser restrictions should be used instead. For example, let us assume a perfect hypothetical and not AT ALL something the SC ruled on in the eighties

law that aim to ensure that the primary language in province X remain language Y, which is not English and as such a minority language. You could, of course, simply ban people from making any signs that include any other language than language Y,
but you could achieve the very same objective by instead simply mandating that all signs must give precedence to language Y. Since the second solution is far less restrictive than the first, the first will fail the "minimal impairment" part of the test.
4. The infringement must be proportional to the objective.
Essentially, what does society as a whole stand to gain from the objective of your law? And what do the individuals affected by the restriction stand to lose? Do the loss completely outstrips the gain, or do the gain justify the loss? That's the subjective one, of course, but all laws are subjective don't exist (for example the age of consent is based on the subjective idea of when teens are old enough to give their consent).
However, with regard to point #4, it's worth keeping in mind that if a law fails any ONE part of the test, it's unconstitutional. So point #4 is a final safeguard against disproportionate laws that would otherwise pass the test; it can never be used to justify a law that would otherwise fail the test.
-------------
Cutlass : the problem, to me, with letting the crazies have their say is that it takes common sense to recognize crazies, and common sense, as all the world knows, is neither. Any crazy can and will find an audience of other crazies perfectly willing to listen to their claims and accusations, and "do something about them".
"Do something about it" usually involve rather extreme violations of certain extremely fundamental rights (right to life, for a start...). This is, of course, unacceptable. Unfortunately, it's been shown (times and times again) that penal dissuasion is fairly limited in its effects on crazies. It's also been shown (times and times again) that flooding the streets with police usually ended up impairing the rights of innocent people a great deal, and stopping the misdeeds of not-so-innocent people very little if at all.
That leaves "keep the crazies from making those accusations and claims that convince other crazies to 'do something about it' in the first place".
It's not a perfect world solution, but I tend to think it's the best way to make a great deal of conflicting concepts meet.