"The Bad Sequel": Sullla's Analysis of Civ5

[BlueTemplar]

Civ is last great game of the Turn based genre, and the only one this "old boy" can use his wit on instead of my slower reaction times RTS games take.

You haven't played Sword of the Stars, have you? Even if there's a real time component, it's still the strategic turn-based part that it the most important.

Saying that you can't have a excellent combat system with the unlimited stacked units is simply not true. SMACX is a statement to that: damage to all units on the tile when a unit on the tile is destroyed; defense, movement and sight altering terrain; unit power ignoring native life; zones of control; zoc-ignoring probes; custom units; etc...
Even with all the modding, Civ4 combat system has yet to be as good as the vanilla SMACX one...

There might be another answer to the unit carpet of doom - make the units exponentially powerful but also expensive to build.

Yes, there's one thing IMHO in common with all the recent failed sequels: greed.

 

[net55]

You haven't played Sword of the Stars, have you? Even if there's a real time component, it's still the strategic turn-based part that it the most important.

Saying that you can't have a excellent combat system with the unlimited stacked units is simply not true. SMACX is a statement to that: damage to all units on the tile when a unit on the tile is destroyed; defense, movement and sight altering terrain; unit power ignoring native life; zones of control; zoc-ignoring probes; custom units; etc...
Even with all the modding, Civ4 combat system has yet to be as good as the vanilla SMACX one...

There might be another answer to the unit carpet of doom - make the units exponentially powerful but also expensive to build.

Yes, there's one thing IMHO in common with all the recent failed sequels: greed.

Sword of the Stars...No, but I'll give it try

unlimited stacked units...it definitely works well, I just don't care for it, too unrealistic for my taste.

exponentially powerful but also expensive to build...that doesn't sound like fun...waiting 10 or 20 turns.

greed...greed is good IMHO, as long as the product is too!

 

[Hian the Frog]

Civ V is and will remain a failure among the Civ serie.

Sullla is right. He made a good sum-up IMO.


Too many bugs. No religion. The combat system is too tactical like (it's not Panzer General ! Even if it was a great great game). Ridiculous tech tree. And more....
Steam is also a problem in my mind: not the best way to avoid piracy, but a good one to earn more money.
Last but not least: Civ players are playing on a PC (or Mac) and not a "box". With such a tool (a PC) and its fantastic "power", we could wait for more than this crapy basic game.

I'm still angry. I'm 38 and i play to Civ since 1992 (or 93 - don't remember). This is the first time i'm so disapointed by a civ serie game. Some a** need to be kicked !

What i think is quite simple: Civ V is almost dead or close to be. :suicide: Let's wait for more intelligent people (not Firaxis it seems) and a Civ VI. A real Civ !

 

[net55]

Civ V is and will remain a failure among the Civ serie.

Sullla is right. He made a good sum-up IMO.


Too many bugs. No religion. The combat system is too tactical like (it's not Panzer General ! Even if it was a great great game). Ridiculous tech tree. And more....
Steam is also a problem in my mind: not the best way to avoid piracy, but a good one to earn more money.
Last but not least: Civ players are playing on a PC (or Mac) and not a "box". With such a tool (a PC) and its fantastic "power", we could wait for more than this crapy basic game.

I'm still angry. I'm 38 and i play to Civ since 1992 (or 93 - don't remember). This is the first time i'm so disapointed by a civ serie game. Some a** need to be kicked !

What i think is quite simple: Civ V is almost dead or close to be. :suicide: Let's wait for more intelligent people (not Firaxis it seems) and a Civ VI. A real Civ !

it's not Panzer General....No it's not, but they could/should learn from some of the features in PG2

No religion....Yes that is a disappointment, from a large religion influence in Civ 4, to almost none in Civ 5.

Steam....I don't like it, it should be optional.

I'm holding out faith that Sid will come back with a good Civ in the future...I'm sure he could of cashed out a long time ago if he wished. The fact that he didn't says a lot.

 

[cephalo]

I do work in AI, and I can tell you right now that 1UPT needs far-more resources (orders of magnitude worse) as a problem than regular unlimited units per tile. This is just the application of basic complexity theory to the problem. What are you basing your "belief" on, blind-faith?

As for look-ahead, you talk as if this is easy or free, it's a very expensive process. Note that one of the complaints about the current version of CiV 5 is the significant wait time at EOT, if you talk about using the kind of resources that make a better AI, I think you are talking about losing a lot of players to the insufferable wait.

In Civ 4, if you had 20 units the AI could look at only one unit and decide where to move it, the only portion of the program that had to look at broader information was at a higher level and could abstract away details like how to move units. It could deal with units in an arbitrary order, it didn't have to consider moving units in a different order, or how the actions of one unit interfered with another. In Civ5, if you want to move a unit, you have to look at combinations and orders. So instead of a linear problem (which of the 18 tiles that this unit is capable of reaching this turn do I move to?) you end up with a quadratic one (Which order of moving units enables all/as many as possible of my units to reach the tiles I want them to be on?).

With 20 units, each with 2 moves and all tiles having movement cost 1 (simplification for calculation), there are 18 possible tiles that each unit can go to so the computer considers each unit in isolation and considers only 360 options. With 20 units, there are 2,432,902,008,176,640,000 possible orderings of those units. Even if you localize to an area say the 18 tiles in which say 10 of the 20 units are located then there are 3,628,800 possible orderings (It's back to the number for 20 units if you consider moving units 1 tile, moving other units then moving the unit the other tile, which may be necessary for solving the problem). Keep in mind this is just the ordering of the units we still have to multiply by 18 for the possible locations to move to. I think if you're very clever about it and use state of the art heuristics etc, you can probably do what the CiV 4 methods did by considering 360 options by considering 3,600,000 options, or roughly 10,000 times as bad. I imagine they are already doing quite good at this since the wait time isn't anywhere near 10,000 times what it was in civ IV, but the tradeoff is things like archers firing at a scout and blocking in 4 pikemen.

I don't think an effective AI needs to be nearly this complex and brute force. I think that any decent industry AI programmer, given a year or so, could make an AI that can beat an average skilled player. I'm sure I could do it if I took the time. Not by calculating into the future, but simply by keeping to certain individual principles, like pulling artillery back when they are under threat next turn, or resisting the urge to make suicide attacks and keeping the line together.

It is likely that they avoided making the AI difficult on purpose, thinking that centuries long stalemates would be less fun, and then simply missed the mark of exactly how much challenge should come from the combat AI. Their QA told them it was difficult enough... I'm thinking its more of a QA problem than an AI problem.

 

[LordTC]

I don't think an effective AI needs to be nearly this complex and brute force. I think that any decent industry AI programmer, given a year or so, could make an AI that can beat an average skilled player. I'm sure I could do it if I took the time. Not by calculating into the future, but simply by keeping to certain individual principles, like pulling artillery back when they are under threat next turn, or resisting the urge to make suicide attacks and keeping the line together.

It is likely that they avoided making the AI difficult on purpose, thinking that centuries long stalemates would be less fun, and then simply missed the mark of exactly how much challenge should come from the combat AI. Their QA told them it was difficult enough... I'm thinking its more of a QA problem than an AI problem.

You've missed the point here, you're talking about the tactical level of the engine. I'm talking about actually figuring out how to move units anywhere near optimally. I'm talking about solving the "archer blocks in 4 pikemen" problem, yes you can use a lot of good abstractions to cover the higher level thinking but in terms of actually moving units around in 1UPT, figuring out the whole "I move unit A 1 square, move B,C,D through the hole this creates, move unit E (a horseman) through hole left by unit B,and up in front of unit A before getting to his destination, then finish moving unit A. This problem is hard enough that its not realistically achievable for this AI. It's not that ciV has done it badly, it's that the state of the art in AI research does it just as badly. It's very similar to the storage domain from IPCC (International Planning Committee Competition) and lots of quality planners take massive amounts of time to solve relatively simple planning problems in that domain.

 

[civ_king]

I do work in AI, and I can tell you right now that 1UPT needs far-more resources (orders of magnitude worse) as a problem than regular unlimited units per tile. This is just the application of basic complexity theory to the problem. What are you basing your "belief" on, blind-faith?

As for look-ahead, you talk as if this is easy or free, it's a very expensive process. Note that one of the complaints about the current version of CiV 5 is the significant wait time at EOT, if you talk about using the kind of resources that make a better AI, I think you are talking about losing a lot of players to the insufferable wait.

In Civ 4, if you had 20 units the AI could look at only one unit and decide where to move it, the only portion of the program that had to look at broader information was at a higher level and could abstract away details like how to move units. It could deal with units in an arbitrary order, it didn't have to consider moving units in a different order, or how the actions of one unit interfered with another. In Civ5, if you want to move a unit, you have to look at combinations and orders. So instead of a linear problem (which of the 18 tiles that this unit is capable of reaching this turn do I move to?) you end up with a quadratic one (Which order of moving units enables all/as many as possible of my units to reach the tiles I want them to be on?).

With 20 units, each with 2 moves and all tiles having movement cost 1 (simplification for calculation), there are 18 possible tiles that each unit can go to so the computer considers each unit in isolation and considers only 360 options. With 20 units, there are 2,432,902,008,176,640,000 possible orderings of those units. Even if you localize to an area say the 18 tiles in which say 10 of the 20 units are located then there are 3,628,800 possible orderings (It's back to the number for 20 units if you consider moving units 1 tile, moving other units then moving the unit the other tile, which may be necessary for solving the problem). Keep in mind this is just the ordering of the units we still have to multiply by 18 for the possible locations to move to. I think if you're very clever about it and use state of the art heuristics etc, you can probably do what the CiV 4 methods did by considering 360 options by considering 3,600,000 options, or roughly 10,000 times as bad. I imagine they are already doing quite good at this since the wait time isn't anywhere near 10,000 times what it was in civ IV, but the tradeoff is things like archers firing at a scout and blocking in 4 pikemen.

Nice post

One of the problems along this line is workers in Civ V, they have to choose between every tile in the empire and also what to build, the larger the empire the more choices it has to make increasing exponentially the turn time, then you multiply it by the number of workers plus the same thing for every civ.

I think that if the turn times were five seconds instead of forty many more people would like Civ V or at least hate it less

 

[scratchthepitch]

The combat system in Civ4 was garbage and that, along with the absurd way religion was done killed that game for me. There were many other smaller reasons why that game sucked in my opinion, but those were the worst offenders.

The problem with the Civ series is greed. Rather than spend the effort, time and money to renew the game and bring it into the 21st century, they tweek their 20 year old AI program, spruce up the graphics (or cheese them up for the kiddies, as in the case of Civ4), and give it a new number. The changes to Civ5 were a bold move, but unfortunately, the suits wouldn't OK the required support to make these changes work. So you ended up with just another edition of Civ1. Like all the others. This company has done the same thing with their other titles.

Hopefully, some one else will see the void created by firaxis/t2 greed and realize that the market is wide open for a decent turn based historical strategy game, and take up the reins (it's been wide open for quite a while now).

BTW 1up is a vast improvement over the cheesy combat in Civ4, but it requires major changes to the game format which the suits refused to approve (most importantly, a real AI, not the series of cheats the Civ uses for an AI). This is what has made the system so clumsy in Civ5. In other games which use similar systems, it works. But those games were designed to use those systems, not some very basic 1980's combat system, originally probably programed as a last minute afterthought.

 

[The T knight]

im not new to civ. not at all. I played civ 1 and thought. "wow this game is great it will never be topped!" I was happily wrong of course in this matter. When i got my hands on civ 2 I was hoping for a few different things. I hoped for the 1upt to be gone. I got it and all was well. I was hoping for more unpredictable AI. I got that too. yeah the AI was insane but it had to be to make up for its glaring weaknesses. I never got civ 3, though I heard it was only a slight improvement on civ 2, still a slight improvement is great! Now we move on to my favorite of the franchise. Civ 4. I play civ 4 every day. Every. damn. day. Ive never done that with a strat game. wether it was beating the game on vanilla, conquoring lands in warlords (I played the alexander strategy a bit. and srtill play it even though i cant beat it on noble ) and BTS. oohhh bts dont even get me started on that. yeas the AI was lacking, but remember what it was like 2 games earlier. Insane and weak. now it actually made sane decsions 40% of the time. yeah youd get dogpiled, but there was always that chance youd be the one dogpiling another power too. the religion addition was utter genius in my mind. true it had its faults, but it actually ran. it made the game more involving because you wanted to this with that and that with that. Anything to make your religion stronger 9mainly for the money ). I could go on and on about how great civ 4 was and still is. Hell as i write this there are still mods being made for it!. hell there are mods for civ 3 still being made! that shows that there is a involved community of people willig to make a game money 5, 10, and even 15 years after its been realsed. but back to what i was saying before I go off track. Then there was CiV or civ5. I had promised myself that i wouldnt buy it because the reviews were scetchy at best and stuff. But when a BRAND NEW version of it goes on sale for $ 20 you dont pass that offer up. ever. Well I know now why it was on sale. It simply put. is a horrible mockery of any civ game before. I loved the hex grid dont get me wrong the hexes made everything seem better looking and more realistic, but BUT everything else. Everything else makes you wonder what exactly the devolpers were on when they made this? Did they really think 1upt wouldnt clogg up the maps? Did they really think making the AI dumber then it was in civ1 and more insane then it was in civ 2 was a good thing? Did they think limiting resorces wouldnt stop me from maybe...i dont know......CONQUORING THE NATION NEXT TO ME TO GET ITS IRON!No that only made the decsion a very easy one. Ive said it before. I can barely handle noble on civ 4 BTS or any version of civ 4 to be exact. I FRIGGIN BEAT CIV 5 ON DEITY 3 MONTHS AGO! yeah if i can beat it on deity then its not very involving is it? no it isnt. its a piss poor example of a game, hell I wouldnt even call it that. So the other day i was cleaning out files from my computer and decided since civ 5 was a waste of space id uninstall it. best decision I ever made.


To other matters. Is civ about the money? hell yes. BUT its supossed to be about building a fanbase and bringing new fanbases in and making a game that will at least last until you make the next one in 5-6 years. NOT 3 Friggin months. Are civ games basically the same at thier core? I dont know cause im not a coder. civs 1-3 had corruption, 4 had matnience, and 5 has a broken happiness system. Oh silly me how I could forget problem number 1 billion with the damn game. yes the combat system is basically the same. but the combat system really isnt broken and you know the adage about something that isnt broke right?

Well im done ranting for now. Ill be back again to rant more maybe. I can only hope civ6 will be made, and be the great ressurection of the franchise now that this trash that is civ 5 is out.

 

[gustavocobra]

Really no different than the way most consumer products are designed today. It is not in a company's best interests to sell aproduct that will last forever!!

If you only had to buy one car in your whole lifetime the car companies would have been begging for handouts a long time ago. Company engineers design the vehicles to fail after an 'acceptable' amount of time and use has gone by.

Video game design is no different...You want to produce a game that goes off with a bang and then fairly quickly leaves you bored to tears(or waiting for the fix-all patch) and wanting something, well, different...like another video game=profits on a year over year basis.

I Disagree with that...
I have a few games that i think will last for ever, Civ 4 is one of them, I played the older Civs but they are not in my list anymore because Civ 4 is way better and have all the features that I like in the other ones and much more.
A real fan lasts a life time, you can't compare a computer software with a car, a car needs maintenance, needs new parts, you can't keep running with a Ford T model because you will have to replace parts that is not manufactured anymore, if they were still selling a brand new Ford T for much less price than the new ones I am sure that you would see a bunch of them in the streets.
I still play Civ 4, but Civ 5 I do very rarely.
I will not hide my disappointment with Civ 5, I see it as a big fail and I can't imagine how they can fix so many problems in this version. I am already waiting for Civ 6.
And please Cid! next Civ, if I have to move to mars, please let me bring my Civilization CD(or whatever) with me...

 

[code9]

I agree with the "What went wrong" article. The biggest point for me is the unpredicatbility of AI, and how bad it is in general, and the punishments we get for trying to build our empire. Why do roads cost, what is the gameplay benefit of that? Can anyone explain? Guess not.

Civ5 was rushed, and I'd like to know why. Did they run out of money to pay the dev team? I know it's hard times, economy is down and all, but still, is that it?

The biggest crime the design team comitted was that they didn't play their own game enough.

 

[Katananga]

Hmm, imagine Civ 4 with:

* Hexes
* graphics of Civ 5
* culture border growth system of Civ 5

I would buy the game right away again

 

[LegioCorvus]

Hmm, imagine Civ 4 with:

* Hexes
* graphics of Civ 5
* culture border growth system of Civ 5

I would buy the game right away again

Having recently gone back to Civ4, I'd say the graphics are just as good if not better. The only 'better graphics' I see are the animated leaders, and honestly, they annoy me.

I'm surprised no one is working on a mod to make something like this. It would take some work, but the Civ4 modding community is one of the best out there. I've been trying out new mods left 'n' right, and I think I have enough new stuff to try out to keep my 4x itch scratched for the next few years.

 

[mamba]

I do work in AI, and I can tell you right now that 1UPT needs far-more resources (orders of magnitude worse) as a problem than regular unlimited units per tile.

I am not in AI programming, just programming in general (I did write an AI for Risk in my spare time for fun though, good enough to beat the standard AI every time - but that AI was truly stupid, which is why I did it to begin with..) and I can tell you right now I do not buy this

With 20 units, each with 2 moves and all tiles having movement cost 1 (simplification for calculation), there are 18 possible tiles that each unit can go to so the computer considers each unit in isolation and considers only 360 options. With 20 units, there are 2,432,902,008,176,640,000 possible orderings of those units.

True for brute force math and an AI whose behaviour I would call random (i.e. the very opposite of an AI). You are neglecting a few things from my perspective.

A) even in Civ 4, you do not want to have all units in one stack, esp. if you have multiple wars going, so this increases the effort for Civ 4 somewhat

B) your units do not wander around randomly, there should be an overall goal, i.e. attack civilization X, but keep peace with everyone else. So moving units to all borders makes no sense, most units should move towards where the AI wants to attack.

C) You have some pretty good ideas on how to get your destination from any point, so of the options you list, >75% make no sense outright, and the AI (pathfinding) should know it and eliminate them without actually testing them (if you know going north makes no sense, then there is no need to test north + any adjacent tile as a possible path either).
As the movement costs are pretty much the same on most tiles, for most tiles, the direct route is the most sensible route, automatically eliminating all non-direct routes of equal movement cost.

E.g. going north when your target is to the south almost never makes sense, unless it is a lot less costly to go north than S, SE, SW, W or E. So NE, NW and N are usually out by default, even E and W you can usually ignore in this scenario.

So, there are a lot of ways to eliminate useless paths early on, reducing the number considerably.

 

[forty2j]

With 20 units, there are 2,432,902,008,176,640,000 possible orderings of those units. Even if you localize to an area say the 18 tiles in which say 10 of the 20 units are located then there are 3,628,800 possible orderings (It's back to the number for 20 units if you consider moving units 1 tile, moving other units then moving the unit the other tile, which may be necessary for solving the problem).

Yes, this sounds like a daunting number of possibilities. But, without doing the math, I would guess that the raw number of possibilities for Risk variants, MTG (Duels of the Planeswalker), Chess (when looking ahead enough moves in advance), and many other games come up with a similar number of raw possibilities. And yet AIs that make reasonable decisions 90% of the time exist for all of these. A few simple rules start eliminating them by orders of magnitude, and soon it's a manageable number. I don't think 3 million combinations is too many to evaluate; on modern hardware, it should only take a millisecond or two to consider each one, and you can thread this to take advantage of multiple cores. And not every AI opponent will have anywhere near 3 million options to choose from.

Re: the current lag between turns; I think the product was just rushed out the door before any performance tuning at all could be done on it; I expect this to improve in time.

 

[BlueTemplar]

There's this module for Civ4 called realistic culture spread - It's not completely like border growth in Civ5, but it's closer.

 

[LordTC]

This isn't the number of possibilities its the number of move orders. It's far worse than that for number of possibilities, as you have to factor in that there are 7 possible tiles for each 1 point movement (if you allow stationary as an option). So if you want to consider the number possibilities you have to multiply by 7^20 (558,545,864,083,284,007). Note that 3,000,000 possibilities at 1 millisecond each would take 3,000 seconds (just under 1 hour). Even if you spread it out equally and perfectly among 4 cores you're looking at over 10 minutes, which I think is completely unfeasible even if its a rare event.

I think people are failing to understand the difference in difficulty between the problems. The Civ IV pathfinding algorithm can ignore all other units. For each unit in the order it encounters it it has to ask what the tactical engine wants to do with the unit and execute it. The execute it portion of the instruction doesn't care about other units other than opposing units which block yours.

The Civ V pathfinding algorithm can't ignore all your other units, and has to deal with the fact that obstacles are not static. Moving a unit to a square can stop your other units from being able to move where you want. Similarly moving a unit from a square can open up ways for your other units to move where you want (For the AI, consider "you" as the tactical planner, which is one level above the pathfinder). This means that unit orders need to be considered, if you have 7 units moving towards the enemy in a hex formation, and you try selecting the backmost unit first you end up with no good movement options for it. In Civ IV this didn't matter, in Civ V its a huge problem!

The International Planning Committee Competition (IPCC) routinely has problems of a similar nature for state of the art planning programs (a major research branch in AI) to solve. A typical problem that is similar in structure is "storage" and any meaningful problem in that domain takes a long time for planners to solve. Most planners routinely hit a 30 minute timeout on problem instances in that domain. It would be nice if this type of problem was approachable in the say 10 seconds we want between turns, but it quite simply is not. You're welcome to continue to object based on "feel" and "opinion" but I haven't seen any data that suggests the kind of thing being suggested is feasible on the timescale and consistency necessary for a videogame and I work in AI at a major research university.

 

[LordTC]

Yes, this sounds like a daunting number of possibilities. But, without doing the math, I would guess that the raw number of possibilities for Risk variants, MTG (Duels of the Planeswalker), Chess (when looking ahead enough moves in advance), and many other games come up with a similar number of raw possibilities. And yet AIs that make reasonable decisions 90% of the time exist for all of these. A few simple rules start eliminating them by orders of magnitude, and soon it's a manageable number. I don't think 3 million combinations is too many to evaluate; on modern hardware, it should only take a millisecond or two to consider each one, and you can thread this to take advantage of multiple cores. And not every AI opponent will have anywhere near 3 million options to choose from.

Re: the current lag between turns; I think the product was just rushed out the door before any performance tuning at all could be done on it; I expect this to improve in time.

The nature of the problem is different. Chess AI works using a ply-seach with a position evaluator. It calculates all the possible positions up to 2X ply's (X moves by white, X moves by black) ahead, and applies a position evaluator to each of them, then picks the move-order that gives the best result for its side given the best play by the other side. It also uses opening tables and endgame tables from study of the game by humans (basically turns off calculation and relies on prerecorded moves). In the middle-game this kind of search is fairly managable. In particular, one turn involves moving only 1 piece, and a 12 ply search (the typical maximum level) involves moving only 12 pieces. The timescale is also very different, playing in a match one typically has a time control like 2 hours to complete 40 moves and an additional 10 seconds after every move. This means its fine for the computer to spend 10 minutes on a critical position, a luxury certainly not available to a ciV 5 AI.

Similarly in RISK while a large number of possibilities exist quite quickly, the game has a general method of evaluating a position (a static board without looking ahead) and typically evaluates only a few moves (often fractions of a turn) ahead.

I'm suggesting a problem that occurs entirely within one AI's own turn in ciV, which is substantively different since it is much harder to avoid looking ahead. Algorithms that say "ok, only consider 6 units at a time, in the 6 unit groups I select" would do quite poorly, if say there were 7 units in an area and one of the frontal ones got left out.

I'm also suggesting a much simpler part of a turn than say a chess move or a risk move. I'm asking the computer to calculate what unit moves are actually possible, or what portion of the moves it wants to do it can actually execute. This is very different from actually searching for the best move in a controlled setting where the possible moves are very easy to calculate (Chess and Risk AI's always know what moves are possible, a ciV AI given desired end locations for all its units, couldn't easily tell you if its possible for its units to get to those squares or not.. it doesn't even know what moves are possible).

 

[LordTC]

I am not in AI programming, just programming in general (I did write an AI for Risk in my spare time for fun though, good enough to beat the standard AI every time - but that AI was truly stupid, which is why I did it to begin with..) and I can tell you right now I do not buy this



True for brute force math and an AI whose behaviour I would call random (i.e. the very opposite of an AI). You are neglecting a few things from my perspective.

A) even in Civ 4, you do not want to have all units in one stack, esp. if you have multiple wars going, so this increases the effort for Civ 4 somewhat

B) your units do not wander around randomly, there should be an overall goal, i.e. attack civilization X, but keep peace with everyone else. So moving units to all borders makes no sense, most units should move towards where the AI wants to attack.

C) You have some pretty good ideas on how to get your destination from any point, so of the options you list, >75% make no sense outright, and the AI (pathfinding) should know it and eliminate them without actually testing them (if you know going north makes no sense, then there is no need to test north + any adjacent tile as a possible path either).
As the movement costs are pretty much the same on most tiles, for most tiles, the direct route is the most sensible route, automatically eliminating all non-direct routes of equal movement cost.

E.g. going north when your target is to the south almost never makes sense, unless it is a lot less costly to go north than S, SE, SW, W or E. So NE, NW and N are usually out by default, even E and W you can usually ignore in this scenario.

So, there are a lot of ways to eliminate useless paths early on, reducing the number considerably.

You misunderstand the problem severely here. Given a set of objectives from the tactical level, execute the desired unit moves. Just figure out how to get your units where you want them. The problem is you have non-static obstacles. In civ IV, units can completely ignore all your other units in terms of finding a path to the desired square. In Civ V, your units rely on other units moving out of the way for paths to open up, and can be stymied by other units moving into the way of their objective before they move where they want. You need to consider moving units in different orders to be able to achieve anywhere near optimal pathfinding (walking around your own units as if they were mountains doesn't work at all).

I'm not saying in civ IV all units have to be in one stack, I'm saying if I know I want to move my spearmen as far as possible northwest, I don't have to consider what my catapult is doing when I move my spearmen. In civ V if I want to move my spearmen as far as possible NW my catapult can be in the way, so it matters what order I move my spearmen or catapult in. If I move my catapult first I get better results from my pathfinding with my spearmen. Just figuring out the correct order to move units in can be very problematic in some cases that typically occur with 1UPT.

 

[LordTC]

I am not in AI programming, just programming in general (I did write an AI for Risk in my spare time for fun though, good enough to beat the standard AI every time - but that AI was truly stupid, which is why I did it to begin with..) and I can tell you right now I do not buy this

This is just basic complexity theory, executing movement in civ IV is worst-case linear in the number of units, executing movement in civ V is worst-case exponential in the number of units.