Let me comment some more, building both on the comments above and on IRL Historical mechanisms that the game lacks now, and could use. I'm going to stick to the '4X' framework, because I really think it's the way to increase Tension/Enjoyment/Playability in the mid and late game.
1. EXplore
First of all, the game needs a good 'Terra' or 'Terra Incognita' style map: one in which All Civs start on one continent or set of connected continents, and about 1/3 of the landmass on the map is in a separate Continent/set of continents in the other half of the map, reachable only by Oceanic travel. And that separate landmass has Resources not available at all in the landmass where all the Civs start, so there are really good reasons to want to go there. That would, at least, extend the Exploration 'phase' of the game to the beginning of the Industrial Era.
And I think the other possibility is to make it necessary to 're-explore' the Map. Make, as stated earlier, Resources appear based on new Technologies. Divide Resource deposits into normal and Industrial Quantities, and to feed your Factories the Industrial are required, so that your dinky little surface deposit of Iron disappears in the Industrial Era (or changes to Industrial because you've only scratched the surface so far). Needing new resources by type and quantity would push Exploration into the Modern Era at least.
2. EXpand
We aren't gong to get an Infinite Map, so at some point in the game, Expansion becomes a Zero-Sum Game: for you to expand, it has to be at Someone else's expense. Historically, this meant conquest of new territories and peoples, absorption of same, or Combining with same and holding the Combine together by really good advantages to those involved (Rome) or shared Culture/Ethnicity/language (China, for one). In the Modern and later Eras, Combine, as in International organizations, runs a constant tension with Nationalism/Exclusivity - making for a very complex potential set of interactions and mechanisms between becoming strong enough in some way to 'go it alone' or convincing your people that you can afford to Absorb the neighbors without cultural penalties.
Yet an entirely different possibility is to make City Founding subject to Real Penalties. Nobody started a city in the Tundra until relatively modern times - the technology of food production/distribution just wasn't there, and in fact a slight cooling of the area could cause human settlements to be abandoned and the 'settlement line' to move south (this may have happened in Scandinavia and central Asia, and it happened to the Scandinavian settlers in Greenland with catastrophic results). Settlements in rainforest/jungle areas by people not native to that type of climate/terrain was also a very 'ify' proposition: the mortality rate among early European settlers in the Caribbean exceeded 50% almost everywhere - try building a city in Civ VI when 1 out of every 2 population points disappears every X turns! Introducing mechanisms like these could leave room for settling New Cities in the Industrial and later Eras.
Yet another form of Expansion, mentioned in one of the Posts above, would be to model that massive increase in population and size of cities in the Modern and later Eras. This would both give you a set of new opportunities to get more out of Mega-Cities, but also give you a whole new set of problems in managing Mega-Cities - food, amenities, etc. No new cities or city sites, but a whole new set of considerations in keeping your old cities from revolution and ruin.
3. EXploit
New resources to exploit by type (new continent) or quantity (Industrial Quantities, depletion of old deposits), but also the Religion Diplomacy, Trade mechanics to exploit throughout the game.
First, Railroads should massively change Trade from the Industrial Era on, and so their finance, construction, and negotiation for construction internationally could be a whole new 'mini-game' - with successful results enhancing military, diplomatic, cultural (access to Resorts, Wonders and Parks at a minimum) and Industrial/Production all over the map.
Second, Religion from the Renaissance Era on changes its type of importance: from pure Faith/Diplomatic to potentially Diplomatic/Cultural/Nationalistic (Ideology). Some Beliefs should dwindle in Importance, others arise or 'morph' into something New: Religion, like resources, should be Dynamic to keep you from being able to ignore it all later in the game.
Third, International Diplomacy from early in the game should have more possibilities. Just for starters, City State equivalents IRL, banded together in Leagues for mutual support and protection. Put that in the game, and imagine trying to attack a City State when the nearest 3 other City States add their armies to the conflict against you. Diplomacy Required First, then attack. On the other hand, imagine getting Suzereignty with a City State and it gets you Diplomatic/Trade bonuses with 3 - 4 other City States. Imagine becoming the chief city (a form of Advanced Suzereignty?) of a League (as Athens did), and adopting Inclusive Policies (as Athens Didn't) and getting City States to join your Civ Voluntarily? Exploiting Diplomacy to Expand - why not?
As stated above, Ideology takes the place of, or expands on, Religion. IF Religion becomes Dynamic, with later-game Beliefs becoming available and earlier Beliefs becoming 'obsolete', then some of the later Beliefs are going to cross the line into Ideology, and that in turn will influence Culture and Diplomacy - and internal Civilization/City Management and Government Type.
Potentially, then, you could get entirely new Resource, Religion/Ideology, Diplomacy/Government systems in the late game (certainly from Industrial/Modern Eras on) to deal with, exploit, or succumb to.
4. EXterminate
It's outside the direct scope here, but most of the Units, Unit Factors, Promotions, Resource Requirements, and influences on Units from Diplomacy, Religion and Resources need to change. Putting it bluntly, right now that entire set of systems Sucks Like a Starving Leach, but that's the subject of another one of my interminably long posts.
What can be discussed is what you should be able to do with units: loan or sell them to other Civs or City States, buy or borrow them from other Civs, get Unique Units from Barbarians or City States not otherwise available, 'Promote' Units with new technologies the way the GDR can be now (why did they waste the Tech Promotion Mechanic on an End Game Unit rarely seen in any game?).
At the very least, let us Name Units from their formation, and let actions by the unit earn Honors or Honorifics: I want to have the 7th Legion "Hammer of the Scots" or the 3rd Impi "Spears Washed in British Blood" as a result of victorious battles against Scots and British/English units. Put some Life into the Units.
The other side of that is that Expansion through Extermination should get much more difficult as the game goes on. You can conquer like Miles Gloriousus throughout the first third of the game, but from the Industrial Era on, Nationalism and its malevolent offspring, Guerrilla/Partisan War should raise its head. We had that in older Civs, we need it back to keep late-game Massive Armies busy stamping out fires that, if you've botched your (Mega) City Management, Ideology selection, Religion or Diplomacy after the Industrial Era, should be able to turn your Sprint to Victory into a nasty game of Whack-A-Mole where the Moles Whack Back.
Military systems and mechanisms should go hand in velvet glove/mailed fist with Diplomacy. In fact, botch the Diplomatic part, and you should from early in the game face the prospect of a simple war of conquest/Extermination turning into a war against a Coalition: "Emergencies" should be a lot more Interactive instead of just Something That Sometimes Happens, and the consequences for getting yourself onto the receiving end of an Emergency should be more severe and daunting.
Just my ideas.
I don't think there is any single thing - even something as important as Improved AI - that will completely 'cure' the game of its Mid-Late Game Malaise. It's going to take a bunch of changes, revamps, 'tweaks' and I suspect they will have to encompass many or most of the aspects of play - all '4 Xes' of the game genre/