Japan took a city from me (Kerma in the south) and in peace negotiations I had them give it back to me. When I get it back, the game asks me if I'd like to keep or destroy it (the latter is greyed out). If I choose to keep it, it will count as occupied. So it doesn't grow. In the savegame, ask for peace with Japan and then have them return Kerma.