I would say:
-Before it's finished building, it should be free to wipe it away. You lose the hammers invested, and that's penalty enough. Would at least solve mis-clicks.
-After it's done, I'd suggest a military engineer should be able to clear things away, maybe only after researching a later tech. There are definitely times where I might consider building an aqueduct early, but by mid to late game you don't need it anymore, and now it's taking up a valuable tile. I mean, how many active aqueducts are there in the world anymore? Feels like it should be natural to simply raze it, and either throw a neighbourhood or something else in its spot. It's not like I get a massive benefit from doing that - I still lose the benefit of the district, and lose the turns it took. I guess you would need another mechanism in case if you've build a wonder that relies on the district, but if my civ decided to go atheist, maybe I should be allowed to plow my holy site into the ground.