I think it's useful to distinguish between the historic and artistic value of monuments, and their continuing memorial value. A monument or statute or piece of public art should probably be removed if it still performs a memorialising task for some intersubjectively defined evil. Thus in post-Cold War Eastern Europe, lots of statues were removed, as they were active symbols of the oppressive state, not simply neutral pieces of history or art. But in some places, the statutes were removed to a special park where they could remain on display because of their enduring historic and artistic value. So you can go to 'Memento Park' in Budapest, for example, and see a collection of old realist pieces that used to adorn the streets of the capital, together with a little museum about communist rule (though of course it shouldn't be forgotten that this park and museum themselves perform a political task, particularly in Orban's Hungary).
Similarly, we don't tend to completely tear down historical buildings that stood for oppressive or evil regimes, with some exceptions such as Nazi buildings in Germany, because those buildings themselves performed a memorialising task.
So removing Confederate monuments makes perfect sense, but so too would keeping them in place inside a museum dedicated to teaching people about the US Civil War. What is objectionable is keeping them in place if they still perform the role of memorialising the lost confederacy; to say it's tearing down history would miss the point of the continuing political function served by Confederate memorials in the South.
