I took over a city via culture, but it was a piece of junk built on the tundra with a large maintenance cost. I immediately gifted it back.

Is there any way to flip it, then abandon it?

If not, this is my first real gripe with Civ IV. Taking over a city shouldn't be a negative, so it will suck if you can't abondon it. Now it looks like I'll have to declare war and destroy it to get rid of it.

Wow I dont have the game but Im sure there must be some way to abandon it or let it starve or something

The only known way to remove a city is by either raze them when you attack the city or use 'worldbuilder' (which feels like a cheat). There was a discussion about this topic somewhere in this forum (though I don't remember where)

starve them to death .

starve them to death .

You've done this? Doesn't the city square itself produce food, meaning that even if you had one specialist, he would live?