Yeah, i find it annoying, too, but I think that the feature was added to keep a player from gaining "free culture," and making a cultural victory too easy. Still, I think they could've handled it better, like re-setting the city's culture to 0 when captured, but keeping all the improvements and the culture they generate, because it's the other players dumb fault for not defending their town well enough that they lost it, so why should you be punished for taking it?

What I find even more annoying than the loss of cultural buildings is the fact that as a result, you have a size 3 city that's in constant revolt due to the fact there are no improvements to keep the SOB's happy (although if you didn't pillage the terrain improvements prior to capture, have a $hitload of luxuries, AND the AI was kind enough to build a marketplace in the town, this becomes less of an issue), which is why I ALWAYS rush build a temple in newly conquered cities, once the resistance ends (sine I use the Stack o'Doom(tm) strategy to conquer cities, resistance usually only lasts one turn, if even that).
Sorry about how long-winded that was, but I think that this is an issue that Firaxis should address, since IRL, cultural buildings aren't always destroyed during a war.