the penalty is never permanent. you can always make it go away by doing nice things for the AI (trade deals that favor them, sharing intrigue, proposing resolution they like, denouncing their enemies, befriending their friends, spreading your religion if they don't have one of their own) and avoiding to upset them (don't lie, don't expand to fast etc).
i'm often branded as a warmonger early on because i almost always steal CS workers, but i still manage to befriend most civs later on.