The only trouble is that "gunpoint conversions" tend to be less-than-genuine. It certainly wouldn't make much sense for the presence of that religion to improve happiness in the town. Simply conquering a city shouldn't be enough to convert the populace.
Still, it does make sense for the religion to eventually gain converts (sooner rather than later, I'd imagine). The Crusaders are bound to be constructing temples to their foreign god in the conquered lands, and woe be to the foolish soul who dares skip church that Sunday! Of course, if you've got the majority of the population worshipping your god ostensibly, you can bet that their children will lean more towards your side. If you've got an army of loyal troops in the streets converting the heathens, you may not really convert any heathens, but you've got a chance at swaying the city in the long run.
What I believe is that Crusaders (however you implement them: A generic promotion, any soldier under Theocracy, or a separate class of units) should simply weaken the existing religions, allowing more "room" for the state religion to spread (and Theocracy's special powers ensure that nothing else will fill the vaccuum). Unfortunately, this is impossible under the current system, as a city either has a religion or doesn't: There is no accounting for relative strengths (eg "This city is 25% Christian, 10% Jewish, 10% Hindu, 60% Muslim, and 5% Miscellaneous). I and others have proposed systems to remedy this (though I haven't posted in months), but there's not enough room in this topic for me to completely reiterate my "religion slots" idea, but let's assume that the game could be altered so that religions could have varying strengths. In effect, a city which is 100% Muslim could, with the presence of Crusaders, become, say, 90% Muslim, allowing your state religion to spread instead. More Crusaders would weaken opposing religions more quickly (though I don't think they should eliminate it entirely).