http://www.youtube.com/watch?v=rSw499GNHUE&hd=1

no, don't really wanna see, but yes, it sucks.

Just out of curiosity: Did you ever turn FoW on to see whether or not the turn really takes just as long?

Assuming the Mod removes FoW for all players, that'd mean the AI checks every tile on the map for every worker to figure out which one to work on (thanks to the cultural border line of sight bug)

Not that it'd be fast even if this is the case and you turn FoW on again, but it'd be... less excruciatingly slow ;D

I'm unsure if your assertion that FoW has no impact on performance is correct. Try it?

Just out of curiosity: Did you ever turn FoW on to see whether or not the turn really takes just as long?

Assuming the Mod removes FoW for all players, that'd mean the AI checks every tile on the map for every worker to figure out which one to work on (thanks to the cultural border line of sight bug)

Not that it'd be fast even if this is the case and you turn FoW on again, but it'd be... less excruciatingly slow ;D

The AI is blind and has same rules pre- FoW mod. This is human/client side only. So as I said before, it does not influence CPU time.

Do you know how many workers there are on the map?

Try strategic view. Pretty funny that after all that whooping about what lovely graphics the new game is gonna have, we're all reduced to something that looks and plays like an Amstrad game.

Wait, not funny, the other one - depressing.

You actually built farms?? :p

Sorry dude...your Fraps froze while your were recording your game.

http://www.youtube.com/watch?v=rSw499GNHUE&hd=1Something's wrong with your rig bro. My specs are worse than yours and I play with high graphics and the longest turns ever take is about 30 seconds.

Stop downloading porn is all I can say...
I have it maxed out with 2x AA at 1280 x 1024 and it takes maybe 10-15 seconds to process each turn late game on a huge map.
My rig is:
i5 760 2,8 GHz Quad core
4 GB 1600MHz RAM
Nvidia Geforce 8800 GT 512 mb (gonna upgrade it soon probably)
WD 1 TB 7200 RPM HDD
Running on Win 7 Ultimate 32 bit (yes i know im not using all of my RAM, I accidently got the 32 bit version instead of the 64 bit =()

I have late game lag, which includes screen going black ibt, but nothing over 30s.

Of course your system is going to have trouble if you're playing the game and recording video at the same time. That's nothing new. It's part of the reason that the pre-release live demo run by the Firaxians/2Kians didn't go as well as they'd hoped since the machine was doing multiple intensive tasks at once.

Stop downloading porn is all I can say...
I have it maxed out with 2x AA at 1280 x 1024 and it takes maybe 10-15 seconds to process each turn late game on a huge map.
My rig is:
i5 760 2,8 GHz Quad core
4 GB 1600MHz RAM
Nvidia Geforce 8800 GT 512 mb (gonna upgrade it soon probably)
WD 1 TB 7200 RPM HDD
Running on Win 7 Ultimate 32 bit (yes i know im not using all of my RAM, I accidently got the 32 bit version instead of the 64 bit =()

You do know that if you have windows 7 you can choose to install it in 32 or 64 bit right? So you can change it to 64 bit by doing a clean install.

My computer is about on par with yours (I have 64bit Windows 7, 6GB of RAM and a slightly better video card I think, but regular SATA drives) and my turns take maaaybe 15-20 seconds during the late game. I think you've got some other issues going on.

If you guys start a huge map with 19 Civs and max out City-States... maintain yourself a small neutral Civ in the middle. When you hit late Rennassaince in the game it will run slow as a crawl.

You guys are talking to a 25-year seasoned hardware benchmarker. I'm not stupid. I ran the same set of turns WITHOUT FRAPS, and WITH FRAPS. The results are the same.

And if you think I'm wrong, then I'll send you the savegame file and you prove me wrong. The game is not stalling and the Fraps is not stalling it. I also have the following:

* SSD drive JUST for Windows.
* SSD Drive JUST for Games.
* SSD Drive JUST for FRAPS.

I'd also like to point out that I have a Quad Core rig. Div V doens't use more than 2 cores, so runnign FRAPS is NOTHING to a system like this since it's only at 50% load to begin with.

And if you think I'm wrong, then I'll send you the savegame file and you prove me wrong.
I suppose one of the crucial things is the mod. I still don't buy the assertion that it'll add no CPU load. If the savegame is compatable with an unmodded game why don't you post it on this thread? I'd be interested to see what performance my PC gets.

According to GameSpot performance guide civ 5 does utilize all 4 of the CPUs but no more than 4. I have much lower rig but runs standard games much smoother. All the Let's Plays on Youtube don't have that extreme lag time either. Perheps worker AI bug causing extraordinary lag when there are maximum number of CIVs and City-States?

As someone with a quad-core super-machine that runs Civ from a RAID0 of SSDs, I too can confirm that Civ5 doesn't use anywhere near all of my system resources, and it slows to a crawl on large or huge maps, especially at any high difficulty level.

According to GameSpot performance guide civ 5 does utilize all 4 of the CPUs but no more than 4.

You know, talk to an expert not Gamespot. Civ V runs 2 threads. When it's on a quad-core, each cpu at 50% utilization or below. This is called core sharing. This is how low/non-threaded games work on multi-cores in a 64-bit environment. They do NOT sit on one or two cores at 100%. So that link is absolutely incorrect.

I'm really perturbed by people with no analytical experience and no foundation in science claiming it's "my fault". And FoW or not, it shouldn't be running this slow. You naysayers are doing yourselves and everyone else a big disfavor by protecting Firaxis through sheer ignorance. I guarantee those of you who say the game-time wait-state isn't so bad are not playing a Huge game, 19 Civs, 24 City-States.

If you're going to criticize my point, then for God's sake be honest that you're not going to bother comparing apples to apples. Most people won't sim to this point with so many Civs as it will take you ALL DAY.

LAST... Fraps doesn't use CPU cycles bound for the game. It saves and logs the files to a dedicated SSD. That's the great thing about a highly OC'ed quad core. Now if someone with a dual-core processor, moderate RAM, and a crappy old hard drive were to do this... then yes - they would see a massive performance hit.

Savegame is attached, the FoW mod is made by XiDragon.

My PC isn't nearly as powerful as the rig in that video and, even late game with tons of units and cities, turns for me have never taken more than ten seconds...

You may also want to scan PC for viruses....I've had Diablo 2 run like a 280 pound marathon runner taking long breaks from fatigue on my PC back when it was infected. And a thorough defrag helps speed things up as well.

But I'm betting it's Civ 5's shabbiness.

JP, okay so you've proven (to me at least) that your computer is strong enough to run the game but that doesn't explain why CIV has slowed down so much for you but not everyone else?

I personally don't have the newest computer and don't play with Huge maps but if this is a problem I'd like to know why it happens..

Well few games ever make good use of cutting edge technology. Almost every version of Civ has had trouble running the largest maps on the rigs that were out at release. There are a lot of things you can do to speed the game up.

I have a 5 year old below minimum spec computer and my turns are never over 10 secs and rarely over 5.

A) Play in strategic view
B) Don't play on the huge maps
C) Don't play on immortal or diety because the AI has too many units.

Huge map with that many city states will always create excruciatingly long wait times between turns, even much earlier in the game. For people who are posting that they never get that long of a wait time, you aren't trying to create a worst case scenario with huge map, max city states and 1500+ AD. The game is very poorly optimized.

What is your evidence it is poorly optimized? I haven't seen any? It runs slowly, but I think it is pretty clear it is doing so because it is a VERY cpu intensive game, as all civs have been.

I know you wish it made full use of 4 cores, but almost no software does a good job of that right now.

If wishes were horses beggars would ride.

You actually built farms?? :p
Hey, don't pick on farmers. They look really pretty! :ack:

JP, okay so you've proven (to me at least) that your computer is strong enough to run the game but that doesn't explain why CIV has slowed down so much for you but not everyone else?


1. Did you skip the threads that corroborated my synopsis? Is it that syndrome where people only see what they want to? ie. What did Jaguar and a few others post?

2. As has been proven over and over, Worker AI is terrible and greatly increases turn time.

3. AI Pathfinding is slow and poorly optimized.

-----other notes to respondents-----

4. It IS that slow for everyone else if they'd run my savegame file and run the exact same parameters.

5. People telling everyone else you have to play smaller maps with way fewer civs... that's a cop-out for shoddy programming.

6. There have been several people in this thread that have corroborated my findings. So don't say "jpinard is the only one seeing it". Just checkout all "terrible performance" threads on Civfanatics, Take2 forums, Apoly, etc. Or do I have to find and link them all for you?

7. This video is a quantitative expose on just how slow the game is with normal settings. If you think I'm wrong, then I want to see you post the same thing. proving the game is so nice and speedy. Especially when Civs go full turbo on workers.

-----

Is there an option to disable the movement animations? That's what's taking all the time, the computer didn't seem to be hanging in between turns, it was moving units constantly, and there were a LOT of units. Even in huge map games in Civ4, I'm betting there weren't nearly as many moves required - a lot more units, but moved in stacks.

Is there an option to disable the movement animations? That's what's taking all the time, the computer didn't seem to be hanging in between turns, it was moving units constantly, and there were a LOT of units. Even in huge map games in Civ4, I'm betting there weren't nearly as many moves required - a lot more units, but moved in stacks.

It is NOT movement animations taking all the time. The long pauses you see are the CPU calculating where to move workers & units to, and it's build queues. It's not an animation problem. THe AI is taking a huge chunk of time calculating each race and then taking action to move the units in sequence. The actual "units moving" is about 5% of the pause time - at most.

I've got a quad core and a pretty decent (I think) video card yet i have all the graphics on their lowest and if I play a huge map with 8 civs and 18 city states the turns take 10 seconds + from the end of medieval age + and then takes about a minute from the begining of the modern age :\ its rediculous as I got excited thinking id beable to do a huge map with 18 civs and 20 city states with it only becoming lagged when well into modern age. A huge letdown and if its true that the game only uses 2 of the cores then thats rediculous as it means i could play with more civs but the game restricts itself :\

I hope they will speed up the turn times eventually. On my computer sometimes the time is so long the screen actually turns black for 10-20 seconds. This doesn't happen every turn, but happens more often in the later turns when AIs have more units and cities to think about. Really, the graphics should be on a separate thread from the AI processing but it seems not. I can't understand why they didn't put them on separate threads.

It is NOT movement animations taking all the time. The long pauses you see are the CPU calculating where to move workers & units to, and it's build queues. It's not an animation problem. THe AI is taking a huge chunk of time calculating each race and then taking action to move the units in sequence. The actual "units moving" is about 5% of the pause time - at most.

I'll admit I only watched the first two minutes, but I did watch with the sound on, and even though you have the view positioned so you can't see all the units moving around on the map, you can hear them. During that first two minutes, most of the time between turns there were marching and unit movement sounds playing.

If they remove the animations and just teleport units to where the AI decides they need to go, they can speed up situations like this where the delay is caused by the time it takes to move around hundreds of units one at a time through every hex between where they start and where they go.

Come on folks: you can test this yourself. I found a large map on my computer not playable because the late game time lag was absurd - yet on a standard map it's reasonable and it's pretty good on smaller maps.

The game shipped with sizes where it doesn't work; it simply wasn't adequately tested to work at all of the available settings.

I7 860 2.8ghz 8gig ram

Civ5 huge map turn ONE, 10 seconds.

CIV4 Huge Map end game: 15 seconds.

CIV5 is a dog. Period. The moment I saw the 1upt rule I knew that pathing would kill this game. What is sad is that its not really even pathing for that many objects, and only pathing them 1-3 spaces. Its actually quite ridiculas. Maybe Firaxis should contact Toady One of Bay12 for his pathing engines.......(shameless plug of my fav game-DF).

I'll admit I only watched the first two minutes, but I did watch with the sound on, and even though you have the view positioned so you can't see all the units moving around on the map, you can hear them. During that first two minutes, most of the time between turns there were marching and unit movement sounds playing.

If they remove the animations and just teleport units to where the AI decides they need to go, they can speed up situations like this where the delay is caused by the time it takes to move around hundreds of units one at a time through every hex between where they start and where they go.

I just explained exactly what's up, and you still stick to your guns while wrong. You're like the anti-vaccine crowd. All the evidence that it cannot cause Autism, but steadfast with a dangerous opinion while presenting no evidence or facts.

More for you:
http://forums.2kgames.com/forums/showthread.php?t=94139

http://forums.2kgames.com/forums/showthread.php?t=95661

http://forums.2kgames.com/forums/showthread.php?t=93151

http://forums.2kgames.com/forums/showthread.php?t=91719

I don't get it. Someone does an analysis, describes what he did, posts a video to show the evidence, and even posts the savegame so that everyone can reproduce his findings (or demonstrate that he doesn't have the problems). And people don't even try to make their own tests (let alone film their results or provide savegames), but attack him based on pure assumptions about his setup.

This is markedly different from how many Civ4 problems were discussed. Would the staunch defenders of Civ5 care to take a look at this thread (http://forums.civfanatics.com/showthread.php?t=147334)? You'll see a discussion between people who claim that the Civ4 AI is cheating when selecting city sites, and others who are defending the AI and claiming that it doesn't. This was a rather loaded question at the time, similar to the Civ5 debates we're witnessing now. The difference is, that in the thread I linked to, everyone - no matter what he thought of the claim - agreed that it would be good to have objective and reproducible tests performed on the issue. Which is what we did, we even found a bug while doing so, which Soren then corrected in the next patch. Compared to this thread, there seems to be surprisingly little willingness of some Civ5 defenders to even consider objective facts. It's a bit sad actually.

Jpinard, thanks for the professional approach. However, please don't allow yourself to be drawn into ad hominem arguments, it detracts from the quality of your test.

Huge maps are pretty unplayable. Large are only tolerable. I find it hard to believe anyone would argue this.

I played a huge map, default number of civs and city states for huge. Played through domination around 1965. High settings, a few things toned down to medium, like shadows and ermm something. Since it isn't a graphics issue, that doesn't really matter anyway.

And even late game, each turn took around 10-15 sec. I'm not running anything extraordinarily powerful, a regular 32bit vista, Q9650 quad non-OC-ed, 4gigs ram, gtx 275.

The game seems to put a good deal of strain on the graphics card, though waiting time between turns goes smoothly. The game isn't universally broken, it seems to have issues with specific setups, rather.

I don't get it. Someone does an analysis, describes what he did, posts a video to show the evidence, and even posts the savegame so that everyone can reproduce his findings (or demonstrate that he doesn't have the problems). And people don't even try to make their own tests (let alone film their results or provide savegames), but attack him based on pure assumptions about his setup.

This is markedly different from how many Civ4 problems were discussed. Would the staunch defenders of Civ5 care to take a look at this thread (http://forums.civfanatics.com/showthread.php?t=147334)? You'll see a discussion between people who claim that the Civ4 AI is cheating when selecting city sites, and others who are defending the AI and claiming that it doesn't. This was a rather loaded question at the time, similar to the Civ5 debates we're witnessing now. The difference is, that in the thread I linked to, everyone - no matter what he thought of the claim - agreed that it would be good to have objective and reproducible tests performed on the issue. Which is what we did, we even found a bug while doing so, which Soren then corrected in the next patch. Compared to this thread, there seems to be surprisingly little willingness of some Civ5 defenders to even consider objective facts. It's a bit sad actually.

Jpinard, thanks for the professional approach. However, please don't allow yourself to be drawn into ad hominem arguments, it detracts from the quality of your test.

I can agree with your position and I hope my reply above was not one of the ones you were referring to. Absolutely there are signifcant (I can't emphasise that enough) performance problems with the game, but if running recording software at the same time it is going to exacerbate the problem, much moreso than usual. I have not the best connection at the moment, so can't sit through a 10 minute youtube video, but can someone please tell me if this same test was done without running fraps at the same time?

I notice the OP has stated fraps should have no effect because it's a quad core, but I don't agree with that, unless there's evidence for it.

I can agree with your position and I hope my reply above was not one of the ones you were referring to.
No it wasn't. :) In fact there are several posts in this thread which pose reasonable questions - I just get the feeling that the heat of the debate is forcing people into trenches, from which they then attack each other, instead of constructively trying to find a good way of clearing up the factual issue (i.e., what is causing the slowdown that jpinard witnesses, how can this be reliably tested, and can the issue be solved?)

I notice the OP has stated fraps should have no effect because it's a quad core, but I don't agree with that, unless there's evidence for it.

Personally I don't know enough about multi-core CPU technology to have an opinion here. It does seem that jpinard took care to minimize the impact that Fraps could have (directing its output to a separate disk so that it doesn't slow down disk access times for Civ5 is a pretty hardcore way of testing already ;) ), so I'd guess that he accounted for that, but as said, I don't really know enough about these things.

If the CPU is not 100%, then there's plenty of leftover power to run FRAPS.

I have a Xeon 5506 quadcore and have much the same experience -- my CPU is generally IDLE for the majority of the game. Although not clocked anywhere near as high, it's still spinning around with plenty of time to do things.

As an example, I can sit and watch a 720p HD+AC3 audio MKV file on my 2nd monitor while playing CIV5 and not notice any extra slowdown in the game. It really is not a matter of it "requiring high end hardware", it's a matter of not being able to take advantage of the resources properly.

It's true that the threading is poor, but without seeing the code it's tough to say exactly why.

But there's no reason that there shouldn't be a single thread per AI running all of the time calculating moves -- there are finite tiles and finite moves, this isn't complicated. The fact that it saves all of its calculations (and not just actions) until its turn is just a poor choice along the way.

PieceOfMind - I actually was in the "performance is just fine" camp until I started trying out larger maps with more Civs. In fact, I went back to this save because turns times weren't as bad as other spots in the game. Fraps primary impact is on moving images and video memory bandwidth - not cpu intensive applications. You can test this by running Fraps test on/off in 3dMark cpu benchmarks.

I guess I could get my camera out and try to take 9 minutes of video to superimpose etc. But really this is stupid and a waste of my time. If a person wants the truth they just need play a game like I did. It's not rocket science.

I can agree with your position and I hope my reply above was not one of the ones you were referring to. Absolutely there are signifcant (I can't emphasise that enough) performance problems with the game, but if running recording software at the same time it is going to exacerbate the problem, much moreso than usual. I have not the best connection at the moment, so can't sit through a 10 minute youtube video, but can someone please tell me if this same test was done without running fraps at the same time?

fraps may have a very slight hit on performance, but nothing noticeable. Since the game is only taking advantage of two cores, fraps can muse one of the two remaining without any issue, however, windows will take a very small hit on the overhead, but nothing you could notice..

I notice the OP has stated fraps should have no effect because it's a quad core, but I don't agree with that, unless there's evidence for it.

I run two monitors and can see the effects in task manager/resource manager. Stopping fraps doesn't change the time turns take either.

My experience is I have the exact same issue. I play huge maps all the time and about half way through the game it starts lagging. Soon I hit lag spikes so bad it locks up the game for over a minute. Task manager reports windows stops responding and only 1 core is being used. There is a whole lot more than just optimization or worker issues going on.

ps, I have a fresh copy of windows 7 32-bit and 64-bit installed and both behave the same. no programs are installed but this game.

OP is obviously correct, there is really nothing to argue.

I notice the OP has stated fraps should have no effect because it's a quad core, but I don't agree with that, unless there's evidence for it.

Taking videos with fraps hurts FPS - but has no affect on how long the AI takes to figure out where it wants to move its workers. (on quad core at least. although I don't think it would matter much on any CPU.)

at sunday I was finishing my latest game standard archipelago culture attempt (so I had 5 cities only), turn time were increasing over time, at the end of the game I usually had to wait 2-3 minutes each turn...

there definitely seems to be some problem with increased number of units over time (did anyone saw the threads about workers?). I will eventually try the worker limiter mod (each civ can build only 1 worker) to see how it goes, maybe OP is willing to try it?
or just worldbuilder his save and delete all workers from all civs except for 2-3...

According to GameSpot performance guide civ 5 does utilize all 4 of the CPUs but no more than 4. I have much lower rig but runs standard games much smoother. All the Let's Plays on Youtube don't have that extreme lag time either. Perheps worker AI bug causing extraordinary lag when there are maximum number of CIVs and City-States?
I am running Civ0.V on a i7 920 with an Logitech G19 keyboard attached (Win7 x64, 8 GB RAM, DX10/11 settings with 2AA, everything else set to "high") and can say that from the logitech performance monitor it is clear that Civ0.V utilizes 2 (in words: two) of my four cores at ~50%. A third core is utilized at ~30% (I assume this is for OS, virus protection, whatever) while the fourth one is idle at 90% of the time.

I don't know what the Gamespot guys have tested but Civ0.V does not make use of more than 2 cores, at least not on my installation.

You know, talk to an expert not Gamespot. Civ V runs 2 threads. When it's on a quad-core, each cpu at 50% utilization or below. This is called core sharing. This is how low/non-threaded games work on multi-cores in a 64-bit environment. They do NOT sit on one or two cores at 100%. So that link is absolutely incorrect.
This I can confirm.

Well few games ever make good use of cutting edge technology. Almost every version of Civ has had trouble running the largest maps on the rigs that were out at release.
What kind of argument is this, now?

"Don't expect a game released in 2010 to have a proper coding! You know, CivIII already was poor in these aspects..."


There are a lot of things you can do to speed the game up.

I have a 5 year old below minimum spec computer and my turns are never over 10 secs and rarely over 5.

A) Play in strategic view
B) Don't play on the huge maps
C) Don't play on immortal or diety because the AI has too many units.

A) the graphics were advertised to be one of the main points during the development of Civ0.V.
So, if I would have to go for the "strategic view" to have a smooth running game, then I would call this a major failure on Firaxis' side (not that I would be astonished about such a failure...)

B) huge maps (unmodified) are something which is delivered WITH the game.
It doesn't seem to be unjustified to expect them to work properly. If they don't: poor coding on Firaxis' side.

C) The number of units only has an impact on turn times IF these units HAVE to be moved. This is highly questionable.
Good coding (AI wise) includes the determination of which unit HAS to move to WHERE. And more often than not a lot of units (especially on huge maps) does NOT have to move.

What is your evidence it is poorly optimized? I haven't seen any? It runs slowly, but I think it is pretty clear it is doing so because it is a VERY cpu intensive game, as all civs have been.
I really have to wonder why especially Civ0.V has to be a cpu intensive game.
Less major civilizations than in Civ4, less units, less everything (no international trade routes for instance, lower graphics)

Before they released the game, they were advertising how beneficial their new, self-made graphics engine would be.
Seems that the opposite is the case.

Of course your system is going to have trouble if you're playing the game and recording video at the same time. That's nothing new. It's part of the reason that the pre-release live demo run by the Firaxians/2Kians didn't go as well as they'd hoped since the machine was doing multiple intensive tasks at once.

I think that demo was a very accurate demonstration of the game. They had about 1 minute lag in between turns, same as everyone always does. People complained here, Greg brushed us off, and that was that. Did they really expect us not to notice that?

In this thread, http://forums.2kgames.com/forums/showthread.php?t=94139, 2k games already acknowledges the issue and 2k Greg passed it along to the devs.

Since these ridiculously long load times are caused by workers only, it seems like something that should be pretty easy to fix.

In the interest of bringing some more hard evidence to the table, here's a perfmon graph of Civ5 running while using OP's save game.

1: Just loaded the game, browsing the map, loading textures (I get the half-loaded textures on my GTX 460)
2: I pressed "Next turn". As you can see, it was about 3 minutes.
3: I was able to issue commands on my next turn.
4: I clicked next turn.

Unfortunately, instead of carrying on, the game crashed. I happen to get somewhat frequent crashes and the only resolution is to go in to NVidia control panel, change the quality settings to the opposite of what it currently is (performance -> quality), and play again -- this usually solves the problem until the next reboot, whereupon I must do this again.

The graph ceiling is 400 because I have a Xeon 5506 quadcore processor. In terms of perfmon, that gives you 400% processor time.

When you hit Next Turn, the game goes in to a state where it is using less of your available resources than when you can browse around the game, which makes sense, since you can't go and click cities and the like while the computer is calculating.

This does somewhat imply that the same thread that the UI is running on is also being used (or frozen) by the thread that the AI is doing calculations on -- not a great architecture.

You can do this too, on your own computers. Perfmon comes free with Windows and the graphs make it easy to understand the data even if you don't understand the values. You can also go inside of perfmon and add a counterset Thread, which can monitor CPU usage on a per-thread basis.

You can see in my original graph that some threads were created somewhat in to the "next turn" phase. It didn't last the duration and it certainly isn't anywhere near what I'd expect with that number of AIs/CSs.

If Firaxis were to take the route Bohemia Interactive has, they have all the capabilities at their fingertips to re-manage data stream code to max out dual core threads, AND quad core threads, AND hexacore threads. BIS has a team that's only 1% the size of Firaxis, yet their game and code-based is probably 1,000x more complex than anything Firaxis is doing. In ArmA you can't just edit a few files in Python, XML and be off to work like you can in Civ. And since BIS can control thre threading so finely now (note: they started with an engine that could not manage four cores - note Arma2's ~ 1,000x more complex than Civ V... then Civ V programmers could do it too... unless they don't know what they're doing OR didn't hire/train an expert in parallel processing on x86 x64.

To lay down how it should work.

* Thread #1 = Worker movement
* Thread #2 = Military movement
* Thread #3 = Build queue
* Thread #4 = Diplomacy

You break these down, and Even if all AI iv have to wait to take their turns, you've made time turn faster by a factor of 4! That is huge! Not 4x faster, but a full factor of 4. You've then reduced the turn time by rather large 25%-75%

Right. I'd actually go a step further.

There should be a strategic AI per game AI, akin:

Game Master
- Overlord #1
-\ #1
-\ #2
-\ #3
-\ #4
- Overlord #2
-\ etc

no overlord should lock/freeze on game master (parent) thread. They should also be constantly activated so they can move as quickly as possible. There's still plenty of idle time when the real player has its turn.

Right. I'd actually go a step further.

There should be a strategic AI per game AI, akin:

Game Master
- Overlord #1
-\ #1
-\ #2
-\ #3
-\ #4
- Overlord #2
-\ etc

no overlord should lock/freeze on game master (parent) thread. They should also be constantly activated so they can move as quickly as possible. There's still plenty of idle time when the real player has its turn.

Gah - Strategic AI. I previously split that between Build Queue and Diplomacy, but Strategic Overview AI yes. It could pony onto the least resource intensive app (Build Queues)

I can't open the link. Could someone tell me what the video is all about? Thanks

PieceOfMind - I actually was in the "performance is just fine" camp until I started trying out larger maps with more Civs. In fact, I went back to this save because turns times weren't as bad as other spots in the game. Fraps primary impact is on moving images and video memory bandwidth - not cpu intensive applications. You can test this by running Fraps test on/off in 3dMark cpu benchmarks.

I guess I could get my camera out and try to take 9 minutes of video to superimpose etc. But really this is stupid and a waste of my time. If a person wants the truth they just need play a game like I did. It's not rocket science.

Note, I am not in any way in the "performance is just fine camp", so hopefully you didn't intend to imply that. Rather I think the performance issues are the biggest problem with the game at the moment, as they are the problems most making me think this game is near unplayable. I can live with relatively slow turn times on large maps due to more AI calculations or whatever, but even to a layperson as myself, it is pretty obvious that something is going wrong in civ5.

Originally I wanted someone to actually tell me what you had said in the video because I didn't have the bandwidth to download it. I'm not sure if it was ever summarised, but I think I should be able to watch it now anyway.

I doubt I will bother trying your savegame as I already know what to expect, and I don't have a quadcore so I can't test the assumption that a quadcore and fraps would have no effect on the result.

You have a better grasp of the hardware details than I do, but in a good testing methodology I think it is just a good idea to eliminate , or at least provide a control for, other factors that could influence the result. I remain skeptical of untested statements about how quadcores, fraps and the civgame would interact, despite the reasonable explanations of how they would interact.

I can't open the link. Could someone tell me what the video is all about? Thanks

It shows about 4 turns, in which the player is clicking "Next turn", while the AI needs 2 minutes per turn to move its units. Personally such a delay wouldn't bother me much, but my patience for inter-turn wait times is almost legendary, and it's indeed contradicting the advertised efficiency of the new graphics engine and AI programming. Also, there's the question why exactly the worker AI takes so long.

As someone with a quad-core super-machine that runs Civ from a RAID0 of SSDs, I too can confirm that Civ5 doesn't use anywhere near all of my system resources, and it slows to a crawl on large or huge maps, especially at any high difficulty level.

I can confirm that on Quad, with 4 Gb of ram and a 9400 GT it takes a lot. The core problems are the subpar use of CPU, the excessive request of RAM and the stressing work on the GPU, that is very strange considering the overall graphic of the game...

Performance patch is requested no matter what.... I can't have problems on a Quad, and i cn't be in total difficulties playing(it's near impossible to be honest) on a dual core, with 4 Gb of Ram and an ATi 3570 (where Battlefield 2 in multiplayer has no problems)... IT IS NOT CRYSIS!

It shows about 4 turns, in which the player is clicking "Next turn", while the AI needs 2 minutes per turn to move its units. Personally such a delay wouldn't bother me much, but my patience for inter-turn wait times is almost legendary, and it's indeed contradicting the advertised efficiency of the new graphics engine and AI programming. Also, there's the question why exactly the worker AI takes so long.

Thanks Psyringe.
I agree inter-turn wait time is too long. And when it gets to later eras of the game, it crashes too often.:(

To lay down how it should work.

* Thread #1 = Worker movement
* Thread #2 = Military movement
* Thread #3 = Build queue
* Thread #4 = Diplomacy

You break these down, and Even if all AI iv have to wait to take their turns, you've made time turn faster by a factor of 4! That is huge! Not 4x faster, but a full factor of 4. You've then reduced the turn time by rather large 25%-75%
Wish it were so, but it really isn't :/
You're assuming that those 4 processes require a roughly similar amount of time, which is not the case.

The savegames I fiddled around with had time between turns of around 20 seconds on my machine (after subtracting the time required for unit movement)
Around 1-2 seconds of this time were used for military decisions, city building (this was like 5ms) and diplomatic decisions put together!
In your example, Processes 2-4 would be idle most of the time while 1 wouldn't be able to finish its work.

No idea how they are handling their multi-core-usage at the moment, but parallelizing their current logic shouldn't be a problem, as its a linear worker by worker process.
(Maybe they already did so)


It's actually rather surprising how fast the engine can handle all its action besides worker movement calculations (remember: the AI isn't all knowing any more, and calculating all kinds of assessments about their enemies, plans and overall tactics each turn!)
So let's hope they replace their sucking tile assessment system by a simple weighted and sorted list of tiles - simple. fast. efficient.

Running a single thread for workers isn't really a problem, providing you run a worker thread per computer player / city state.

It's true that not all operations are equal, but leveraging more threads is not going to hurt.

They can even do a simple query on startup to determine the number of logical CPUs and have that be the volume of threads generated -- there are many scenarios to solve this problem. But it's very clear that upon hitting "next turn", the performance gets locked to a single CPU, and even locks processing of UI elements / parent thread. That's just poor design.

I think 1 UPT will slow things down a lot. In my experience the performance roughly matches Civ 4 on release, so... give it 5 years and all our computers will be good enough to handle it!

I wonder how much Civ 5 is actually parallelised. I wouldn't be surprised if it's nothing much more complicated than having the graphics and animation handled by one thread, while another handles the AI. The trouble is that parallising the AI is not an easy task as each step in a turn can potentially greatly influence the next step in the turn. There are things you can split off (city deciding which tiles to use, for example) but I would imagine that the nastiest bit of the AI is pathing and combat, and that's not pretty to do efficiently in parallel. If one of the units discovers, for example, a massive enemy army the other units being calculated in parallel need to know about this, or we end up with an AI that looks stupid. There are things you could do with parallising a single AI which might produce good results, but nothing that would be easy in terms of initial programming or in terms of maintainability.

One thing that could be fairly easily done is city states. It'd be fairly easy to work out in the city state phase which city states could influence other city states then parallelise them in such a way that no mutually influencing states would be running at the same time. How much performance you'd see from this is not obvious, especially as there might be gotchas in the implementation as I imagine CS use the same functions as normal civs in a lot of places.

There are many places where small level parallelism might be possible, however it's often the case that this isn't actually worth doing. You actually need to send a reasonable chunk off in a parallel unit for any significant benefit to be seen.

BTW: Save game crashed for me. I've been getting that a lot with Civ 5 though. Hoping the patch will fix that...

There isn't much that is parallelized. You can use perfmon on your computer to observe both the process and the individual threads within the process.

If Firaxis were to take the route Bohemia Interactive has, they have all the capabilities at their fingertips to re-manage data stream code to max out dual core threads, AND quad core threads, AND hexacore threads. BIS has a team that's only 1% the size of Firaxis, yet their game and code-based is probably 1,000x more complex than anything Firaxis is doing. In ArmA you can't just edit a few files in Python, XML and be off to work like you can in Civ. And since BIS can control thre threading so finely now (note: they started with an engine that could not manage four cores - note Arma2's ~ 1,000x more complex than Civ V... then Civ V programmers could do it too... unless they don't know what they're doing OR didn't hire/train an expert in parallel processing on x86 x64.

To lay down how it should work.

* Thread #1 = Worker movement
* Thread #2 = Military movement
* Thread #3 = Build queue
* Thread #4 = Diplomacy

You break these down, and Even if all AI iv have to wait to take their turns, you've made time turn faster by a factor of 4! That is huge! Not 4x faster, but a full factor of 4. You've then reduced the turn time by rather large 25%-75%

It would be very nice if it was that simple. But ask yourself, how can you do any decision about worker/military/build order before you know your diplomacy?
You can't, if you don't know that if you are gearing for a war or not you just can't move your soldiers/workers or set any build queues. And that goes for all four of these. You need to know where your military is before moving workers etc. Multi threading is not so simple that you just could put tasks like that for each core. If you could, you still would need to be calculating all three million paths for military on one core while other is calculating on diplomacy and then decide which one of those three million pre calculated military movements would be the right one to execute. Simply waste of time, not processor time but time that the programmers could use better, like fixing bugs.

Also my phenom X4 at 2.5 GHz can get up to 75% at all cores. So the game does use more than 2 threads at some times. Not too well, but it can. Some times rarely it just freezes for ten or twenty seconds, then it takes 100% at one core and other three are idling, seen that many times from running task manager at second monitor. Maybe you cant see that on faster CPU:s, but with this old chunk it sure uses at least some power.

I'm not concerned about the load times, but the oblivious bugs that must have come up at beta phase, like the resource trading bug and the 69-limit. I really feel like I was a beta tester that didin't get paid for it but rather had to pay for testing the game.

Edit:
Also the Ai is so damn awful at fighting on sea that all the maps other than pangea are pretty much useless with out mp. AI:s just bring out their troops embarked for you to pick up one by one with any ship that you might have built.

I have much lower specs and much better performance.

did you try the savegame?

performance kills this game... even if teh game was so poor as it is, if you waited few seconds for turn ends, than you could at least waste more time thinking about silly strategies on how to challenge yourself to win (ie 1 city in normal game on deity), artificial restrictions of various kinds etc... but the performance as it is at the moment, simply does not allow the game to be played on large/huge maps past medieval times. It's a nail in the coffin.

You propably said that to the above poster. But I tried the the save and it took about one minute and forty seconds to finish the turn. Usually at that point I would need ten minutes to run my turn,(so couple of minutes waiting wont do much) but because the city limit I can't play games like that.

well if you are geting half the time the OP, that is certainly interesting... there may be something more to this end game lage than just basic hardware requirements.

Well my system is certainly inferior to the the rig that op is holding. I have the game installed on intel 80 GB SSD. With phenom cpu and 4 GB of ddr 2 memory. Now I have run it twice and it takes about the same 1:40 to finish the turn. It might also have some randomness. or maybe more :). If it has, it is going to be near impossible to make any worthwile benchmarks for this game.

I've found some interesting things going on late game.

My recent epic game I was getting lag about mid game(375). It was the same as my normal speed(250). about 50% way through the game seems to be happening instead of turn number.

At this point I get lag spikes and you can see animations freeze for a long time (1 full minute). I decided to delete some navy units as this lag started when sending out navy to find wonders and the more map revealed, the most lag increased. After deleting navy the lag was the same.

I then deleted all workers and my 1 minute lag spike was gone completely. I even couldn't find one of my worker and the lag spike was still one full minute. deleting that last worker caused a full 1 minute lag spike itself.

Now my turns takes 35 seconds waiting for AI stuff, but I have no workers to improve tiles. not that I need any right now as I'm so far ahead.

Here is a link to my saves both with workers and without so you can see the difference.

http://forums.civfanatics.com/showpost.php?p=9802937&postcount=49

also, I can see in the OP video all the lag spikes occuring for the Civ AIs in the game(the citiy states go by quickly), but no lag on his phase of the next turn phase (just after barbs turn or CS if no barbs as I see none) for me, when I get the mini lag spikes during the AI Civ's phase, I can see task manager reporting not responding and ony one core being used. It seems like there is some coding bugs for the not responding to show up.

anyway, hoping patch comes soon. or at least a mini patch to fix some things.

Ok a couple of things what exactly are FRAPS? (some people said having a quad core means that fraps are no problem but I'm still curious as to what they are) and has anyone tried the mods that only let each civ have 1 or a certain limit of workers? and is it worth bothering with?

FRAPS is a frame rate display utility for Windows based PCs. FRAPS = FRAmes Per Second

Ok a couple of things what exactly are FRAPS? (some people said having a quad core means that fraps are no problem but I'm still curious as to what they are) and has anyone tried the mods that only let each civ have 1 or a certain limit of workers? and is it worth bothering with?

FRAPS is a two part utility.

First, it is a benchmark program to measure how frames of video are being displayed by your video card every second. Or frames per second (FPS). It is a novellty thing to let you know how your system is running or if it's running well enough for a game you want. 45 average FPS is a good measurement. It depends on screen size as well, as larger dksplays take longer to draw.

second, it's a video capture device. Kind of like your DVR to record your favorite TV shows for later viewing, except it's for your computer. It work well for most games and other applications.

Here's a link if you want to check it out.
http://www.fraps.com/

http://www.youtube.com/watch?v=rSw499GNHUE&hd=1

Wow, what a slow comp, something must be wrong with his system and running fraps don't help. I can't believe how much faster my well maintained Althon 64 X2 5200 with 3GB RAM, Vista 64 and NDIVIA 9600GT 1GB DDR3 is. He is playing on a huge map, I don't play on maps that size but the game is maxed out on my system and runs speedy. Takes about 2 sec a turn on my comp. I am wondering if the game also runs faster in DX9 mode, seems so.

Wow, what a slow comp, something must be wrong with his system and running fraps don't help. I can't believe how much faster my well maintained Althon 64 X2 5200 with 3GB RAM, Vista 64 and NDIVIA 9600GT 1GB DDR3 is. He is playing on a huge map, I don't play on maps that size but the game is maxed out on my system and runs speedy. Takes about 2 sec a turn on my comp. I am wondering if the game also runs faster in DX9 mode, seems so.

I'm going to assume you're just kidding around trying to bait me. Using nothing even remotely comparative.

Perhaps someone with such fast turn times on huge maps can prove it by uploading a video in a similar style? I do not believe for one second that even with a powerful computer would you be getting 2 second turn times with a savegame like that.

I think the key passage in the post in question is "I don't play on maps that size". The question of Dx9 vs Dx11 could be worthwhile testing though. Although there's other evidence hinting at a worker AI lag, it wouldn't hurt to explore other possibilities as well. Imho. :)

Hi,

I just want to point out 1 thing that has been repeated many times in this thread which is very WRONG!

The number of units can not be the real slowing down factor with respect to the case you all are disccusing...

There are 2 factors:
1. The number of units
2. The amount of codes/logics to go through in order to decide where to send a unit or what a unit should do.

For 1., you all are taking computer CPU power as slow as humn brain power. Let me tell you this, you will need to have more than 1 million units on the map to really see an performace impact.
It is the 2. that is the actual cause of the slow down problem.

If 2. is badly coded, then each unit will take lot of decision making time, that cause adding 100 units seems to be a big matter. (but adding 100 unit isn't a real big issue for CPU to go through.


Programmers (if any of you is one) I mean

FOR nUnit = 1 to N
// Call: a routine to determine what happen/where to move a unit in this turn
WhatToDo(aUnit[nUnit])
NEXT

there won't be a differrence if N is 1 or 10000 in the above codes, it is the time WhatToDo() routine takes per iteration... see my point?
If WhatToDo() is really optimized, which means it takes very little time, then you will need to increase N to say 20 million (20,000,000) in order to feel a lag.

Btw: (I would like to add the below)
Chances is their path finding routine, or a routine that gives all possible tiles a unit can be reallocated has bugs or there are codes that call it inefficiently.
Otherwise that must be the routine that gives all possible tasks a unit can be assigned has bugs.

The above are my guesses, anyway, I suppose they are good one.

Hi,

I just want to point out 1 fact that has been repeated many times in this thread which is very WRONG!

The number of units can not be the real slowing down factor with respect to the case you all are disccusing...

There are 2 factors:
1. The number of units
2. The amount of codes/logics to go through in order to decide where to send a unit or what a unit should do.

I'm a programmer, and you're wrong. No matter what each unit has to do, doubling the number of units is going to make it take (at least) twice as long. Probably more. Of course optimising the inside of the loop is going to make it faster, but so is more units. You're never going to optimise the inside to zero, and it's always going to be a trade-off.

workers must be it... I wander what did they do to that guy who was hired to write worker routies :D

if it is as simple as that, that may be the true big improvement after the patch.

I've found some interesting things going on late game.

My recent epic game I was getting lag about mid game(375). It was the same as my normal speed(250). about 50% way through the game seems to be happening instead of turn number.

At this point I get lag spikes and you can see animations freeze for a long time (1 full minute). I decided to delete some navy units as this lag started when sending out navy to find wonders and the more map revealed, the most lag increased. After deleting navy the lag was the same.

I then deleted all workers and my 1 minute lag spike was gone completely. I even couldn't find one of my worker and the lag spike was still one full minute. deleting that last worker caused a full 1 minute lag spike itself.

Now my turns takes 35 seconds waiting for AI stuff, but I have no workers to improve tiles. not that I need any right now as I'm so far ahead.

Here is a link to my saves both with workers and without so you can see the difference.

http://forums.civfanatics.com/showpost.php?p=9802937&postcount=49

also, I can see in the OP video all the lag spikes occuring for the Civ AIs in the game(the citiy states go by quickly), but no lag on his phase of the next turn phase (just after barbs turn or CS if no barbs as I see none) for me, when I get the mini lag spikes during the AI Civ's phase, I can see task manager reporting not responding and ony one core being used. It seems like there is some coding bugs for the not responding to show up.

anyway, hoping patch comes soon. or at least a mini patch to fix some things.

I'm a programmer, and you're wrong. No matter what each unit has to do, doubling the number of units is going to make it take (at least) twice as long. Probably more. Of course optimising the inside of the loop is going to make it faster, but so is more units. You're never going to optimise the inside to zero, and it's always going to be a trade-off.

Actually hclass has a point. He is not stating that number of units is not a factor at all, which is fairly clear if you read the post carefully (unfortunately you cut off his quote which is a bit unfair). He is saying it is not the main factor that is the cause of the lag. In other words, the problem is not the large number of units on large maps. Larger number of units on large maps just reveals the problem more obviously.

The 2nd point he speaks of is where there is something going wrong in civ5. There is, in my view, most probably a very poorly optimised or just plain bugged step in the worker logic that is causing an insane number of loops.

I bet if Firaxis revealed the SDK source tomorrow, one of the programmers/modders on this site would find the problem on the same day.

Actually hclass has a point. He is not stating that number of units is not a factor at all, which is fairly clear if you read the post carefully (unfortunately you cut off his quote which is a bit unfair). He is saying it is not the main factor that is the cause of the lag. In other words, the problem is not the large number of units on large maps. Larger number of units on large maps just reveals the problem more obviously.

I'm not quite sure what the edit was after I posted. Perhaps I was a bit blunt saying "you're wrong", but I still get a fairly strong impression that he's suggesting that the number of units is not significant. You're not going to be able to optimise the function hugely, and, as far as the user is concerned, the performance is always going to be directly linked to the number of units.

I'm not quite sure what the edit was after I posted. Perhaps I was a bit blunt saying "you're wrong", but I still get a fairly strong impression that he's suggesting that the number of units is not significant. You're not going to be able to optimise the function hugely, and, as far as the user is concerned, the performance is always going to be directly linked to the number of units.

I think he's saying that the number of units would be insignificant if there wasn't any code associated with moving them. But I'm not sure myself, because then he'd stating something so obvious that I don't understand why he's saying that others are wrong. I.e., in that case, to make his point, he must have taken others' statements so literal that he actually misunderstood them.

Everyone who's saying "the number of units is a problem" is of course implying that this problem is created by the code that's being looped through for every unit, but if one takes the statement very literally, one might say that it's wrong, because the code creates the lag, not the number of units itself.

At least that's how I understand him. I have to admit that it took me a while to arrive at this conclusion because at first his perspective seemed rather odd to me.

Yes, if the number of units were the real cause of the problem, then the solution to the problem would naturally be to find some way to reduce the number of units. But this is not a satisfactory solution.

Reducing the number of units isn't a solution either, at least not for long. I deleted all but 1 worker and still had 1 minute spikes. I delete the worker and the spike went away.

It doesn't seem to be the worker routines for automation to decide what they do as none of my workers are automated. I manually control all of my units.

I think it has to do with some part of the program trying to manipulate the worker that shouldn't. My guess would be advisors may be doing this to offer suggestions for your turn. I wish there were an option to disable advisors completely as they are useless, except for noobs, anyway.

I also find that 1 great person seems fine, but a second one starts the lag spikes again. I guess two great people equal a worker. Probably same with UU that have worker ability, but haven't tried yet.

I've continued on with my game with little success. now the game just locks up constantly. I have to autosave every turn because I rarely make a dozen turns now with no workers. If I capture a worker then lag spikes hit and now they last almost two minutes. but I have to keep the worker until the next turn before I can delete it.

woot, game patch released to v63. now we'll see how things go now.

good luck everyone.

v63? I thought it was 62.

My system specs:

AMD Phenom II x4 955 @3.21GHZ
4GB (2x2GB) Corsair XMS3 PC3-12800 1600MHz 240-pin DDR3 CL9
Diamond ATI Radeon HD 4890 1GB DDR5
Windows 7 Home Premium x64

My settings are all at max, huge map. End game takes about 1 minute between turns, sometimes when moving a unit the game freezes for 30 seconds before I'm able to move the next unit, which is always a worker. I believe the issue is somehow related to the workers, and the AI moving them.
Only crash I had is related to the 70 city bug.

Well, my lag spikes no longer happen. I captured a three workers and zero lag spike. Even my turn times have lowered. It was 45 seconds, now it's only 20seconds and more war happening around me. 5-6 battles a turn I have to watch.

anyone replay this game yet? I can't because it's not a vanilla civ v save.

good news, would be great that this issue has been improved significantly.

I'm a programmer, and you're wrong. No matter what each unit has to do, doubling the number of units is going to make it take (at least) twice as long. Probably more. Of course optimising the inside of the loop is going to make it faster, but so is more units. You're never going to optimise the inside to zero, and it's always going to be a trade-off.

Good to have a programmer here!
My point is, deciding for the best location to go, or what to do for a unit should be merely code branching... e.g. let say on average it takes 2000 IF ELSE END logical branching to come to a result. Remember, if the decision making process is modal correctly, then it should not involve:
1. Any graphical display
2. Access to HDD or other storage
The above means, in practice, it should not be very time consuming, unless, of course, there is bugs in the branching codes.

BTW, in the description of the patch just released, it proves my guess is correct. The worker's path finding bug(s) was the problem maker.

Technically speaking, I think Firaxis is a poor game developer. For a game like Civ, I wonder why is there a need to always introduce new game engine in every major release and (even if there is a good reason to do so) how can that force them to rewrite the whole game from scratch. Can't they modal the game correctly and at least port the base/core modal codes from version to version?

Also, there isn't any 3D animation intensive graphical process involved at all, why is such a high graphics device requirement?

Nowadays, even a small company, e.g. those who produce video format converter program can write very good codes to support multi-cores machine... how come such a, so call big game company fail to do so (even if they have claimed Civ5 does support multi-cores, but many have pointed out it actually didn't). Big company like ADOBE has almost all their products support multi-cores nicely... they don't even bother to mention that as a selling point, it is a norm now for them.