Yes, running a deficit means that any existing gpt payments to other civs will be continued. They obviously get their gold from some 'invisible fund', thus the deals continue and no rep hits occur (you can check on active deals table). Same goes for AI civs that pay gpt to you (you can use it to bankrupt them, especially if you *feel* they receive gpt from another civ and that money source is about to run out soon for them).
But if you're not in anarchy and run a deficit, you lose one city improvement and one unit each turn as penalty. This may hurt especially in the early game, but lateron, deficit spending plus max research is a very powerful exploit: set tax to 100%, trade all gpt away, then set tax to 0%. You would then lose 20 buildings and 20 units, which is a minor penalty compared to the benefit you could get (imagine a decent sized empire on a not so small map - losing some workers and some granary [say your core cities have already reached size 12] or barracks here and there is a joke compared to the benefits you could get). IMHO this penalty should not be constant, but possibly depend on the deficit itself: E.g. # of units improvements to be disbanded each turn increases with the amount of the deficit (maybe expensive stuff like cavalry or universities will even be regarded
first), and/or map size/empire size dependent (e.g. percentage of OCN reached so far as indicator). IIRC, the penalty was much more hurting in prior civ versions (AFAIK you'd auto-sell as many buildings as possible per turn trying to even out the debt!).
The usual deficit exploit (normal government plus huge deficit spending with a comparibly low penalty at some point) is something I don't use.
But I'm not sure about anarchy deficit spending. No penalty, but that's limited to anarchy length. Although you could possibly have much earlier a republic government (in case you traded a lot) and at that early game phase, the earlier benefit of a republic (or monarchy) could be very vital. OTOH, as mentioned in the first post, the AI civs would certainly benefit from this rule, too, since they tend to revolt more often than the human player (ok, max gov transition times are very low for them on higher levels, but anyways).
So is anarchy deficit spending an exploit (or some bug/glitch that needs to be fixed) in your opinion? Do you use it (and does it work in C3C)?
PS: ainwood, what's wrong with enhanced forestry operations & scientists?
) goes to war when they receive a lot of gpt - not counting in here those gpt deals that you make in a hurry when you see their SoD entering your territory and then ask to leave or declare. 
