Well, you want to get culture at all costs, right?
But as Memento says, by either rushing a Worker for 80 gold or disbanding a unit, the penalty is neutralized. Then, if you are lucky, you can rush something cheaper and change to Library in order to utilize the city production on the final turn, so-called short rushing. Say you disband a Medieval Infantry for ten shields, then rush a Medieval Infantry for 30 * 4 = 120 gold, then set production to Library and disband a second Medieval Infantry for ten more shields. You now have 50 shields in box. Since the city is apparently producing 30 shields per turn, the Library is finished in the interturn.
The cost was two MIs and 120 gold, or 280 gold only (Worker 80, Granary 200, ten shields lost).