Units in a city do not get terrain damage if on a damaging terrain. That's of course if you can even found the city on the terrain. However if a unit leaves the city they will get damaged if the city is also surrounded by damaging terrain, even if its in your owned territory.
So no its not easier for the attackers, and if the attackers are you then its even worse since you can get terrain damage, city units attacking you and possibly even defenses attacking you like towers and/or traps.
I can only imagine how crazy it must get in multilayer where both players experience terrain damage. Since AI seems to be immune.