It's actually a pretty complicated formula if you're looking for an
exact answer. The formula can be found in the FAQ thread in Section 13 -- found
HERE. There are a lot of variables, but I think that only a few are really important in most game circumstances.
I (and I suspect many other players) use a more "shorthand" sort of formula in the hopes of getting the general idea without having to deal with all potential variables. The primary factors I consider when judging on a proper unit garrison to prevent a flip are: (1) the my civ's total culture versus the enemy civ's foreign culture; (2) the number of enemy citizens in the town and the number of those that are resisting; and (3) the number of the city's 21-tile radius that fall under the enemy cultural border.
Each of the city's 21-tile radius that falls under enemy control counts as 1 point. Each actual enemy citizen in the captured city counts as 1 point, but each enemy resistor counts as 2 points. Add up all three point totals.
If total culture values are about equal between my civ and the enemy civ, and if the city is not in disorder, I can expect that 2 military units per "point" will prevent a flip. If my total culture value was about double the enemy's, I'd expect about 1 unit per citizen is sufficient; the reverse is true as well -- if my culture was half of the enemy culture, I'd expect to need 4 units per point.
Civil disorder doubles the number of troops needed - so it's a good idea to restore order quickly if you can do so safely. WLTKD halves the number of troops needed - great if you can get it quickly. Relative distance to your palace versus the enemy palace also plays a role (assuming there is a chance for a flip -- i.e., it is not suppressed completely).
Rushing a temple or other culture building helps if enemy culture is pressing against your new city's 21-tile radius -- the sooner your new city controls its own 21-tile radius, the sooner your "point count" drops, meaning the sooner your garrison can be safely reduced.
Okay - this has been a long post

, but let's have an example and an alternate solution:
I conquer an enemy city. Our civ's total cultures are roughly equal. The city has 5 enemy citizens, 1 of whom is resisting. Enemy culture controls 9 tiles of my new city's hypothetical 21-tile radius. My point count is 15 (4 normal enemy citizens = 4; 1 resistor = 2; 9 tiles = 9; total is 9+2+4 = 15). Since there's a resistor, the city is in disorder. I'd expect to need 60 troops to absolutely prevent a flip

. But the first turn I just garrison a few wounded units for healing, and I set all citizens to entertainers. The next turn, one citizen has died due to starvation, the resistance has ended, but my borders stay the same. Now my point count is 13 and the garrison to absolutely prevent a flip is approx. 26

. The next turn, another enemy has starved and I've conquered a neighboring city. The first city now only has 2 of it's 21-tile city radius controlled by enenmy culture (because of the capture of another city nearby), and its population is down to 3 citizens. My point count is 5 - the magic number about 10. Maybe now I will move in 10 outdated units to prevent a flip - or maybe not.
Keep in mind this is the number of troops need to
absolutely prevent a flip. If you actually do the math on the forumla, even leaving the city completely ungarrisoned produces a pretty small chance of a flip -- which leads to an alternate tactic employed by many players. When the required garrison is so high as to be impractical, it often makes sense to leave the city undefended (or with just a few troops you can afford to lose in a flip), but leave a healthy attacker or two just outside the city. The rest of your forces move on to the next objective (which hopefully will also reduce the flip chances anyway by eliminating neighboring culutral tile control). If the city in question does flip, you've lost only a few troops (instead of your whole army), and you can immediately retake the city with the pair of attackers you left just outside the city (and that therefore survived the flip).
In the example I gave above, I would almost certainly garrison just some wounded troops for turn one (to suppress the resistance) -- I'm not likely to have available 60 troops with nothgin better to do than garrison the city

. I'd leave only a few token troops garrisoned during turns two -- XX with some attackers nearby in case the city flips, while I continued my offensive elsewhere. Only when my troop "flip suppression" requirement got down to something manageable like 4 - 6 would I bring some troops into the city and thereby free-up the attackers I had stationed nearby. Old spearman / warriors etc. that never got upgraded, or troops "defending" my native cities well behind the front lines (and so out of any near-term danger) make great "behind the front lines" garrisons -- they count just as much as a modern armour for flip purposes, and they free the better troops for front-line action.
Remeber that all of my "shorthand formula" above is exactly that -- shorthand which fails to take into account all variables. There are also additional nuances on flip probabilities that I've ignored. But this post is already too darn long to go any further

.
EDIT: Padma already posted the formula I linked too, but I think there is some additional "color" to the flip nuances in the FAQ.