This has already come up so many times before. Yes, they may have started the war, but it was you who took the city in the end. That was too asymmetrical a response. Also, in the world's eyes, what if you actually coveted some lands of your neighbour and just used the occasion to cunningly provoke them in to declaring on you? It is not just white and black out there, there's a lot of grey. What if in this situation you were in the shoes of Prussia, and your neighbour in the role of France in 1870? Sort of, of course, just a somewhat believable example, don't look too deep into it.
Just accept the fact that taking a city with your army and keeping it is considered a serious business in Civ VI. If you want to punish the aggressor, destroy their armies and burn his lands, take reparations in gold and gpt in a peace deal, but do not keep the cities you've taken with your troops. At least not all of them, give some back. And if you do want that city badly, hurt them until they agree to give it to you in a peace deal, without you actually taking it with your troops.