OK, so I defied the Apostolic Palace on the resolution to stop trading with my vassal-by-conquest Sitting Bull. (As an aside, that seems kind of cheesy to direct a resolution at a vassal and not the master. As a further aside, Bull still hates me and I get no attitude bonuses for standing with him against the AP guys. Hm.)
I've noticed that since defying the resolution, I don't have a -5 for "TWCYAV" in all of my cities. Is the penalty only confined to cities with the same religion as the AP? If so, can I make it go away or reduce it by introducing new religions to those cities?
I'm pretty sure the only way to make the unhappy people from "the world considers us villains" go away is to vote yes on a resolution that passes. Either that or raze the apolistic palace alltogether.
)

per military unit and he had like 20 units in the tile, would that be a factor? when the game ended, that city was a population of one because that it all it could support.
Each unit on a tile decreases the flip chance by a certain %, depending on the unit's strength. If you have enough units in a city, it will never flip, regardless of the cultural pressure on it.