Do we have any idea what the prime graphics culprits are, in terms of buildings vs units vs terrain?
It's not buildings any more. I also suspect that the terrain is not THAT major of a factor (more major than buildings, but still not dominant). It's looking more and more like its a pure scalability issue in number of revealed plots. While lots remains unrevealed it seems to stay under control, but once you can see most of the world larger maps really start to cause it to soak up memory. Stopping it displaying improvements and features (which is obviously only useful in testing, not actual play) reduces usage a lot, but even without those displayed it's pretty bad.
I'm still experimenting with Talin's save game. I expect I will be for several days more.