Optimization suggestions

Hey guys, long time. I'm back, and with lots of suggestions to improve the speed of this mod. I was looking at the Python code recently and the reason it's so slow is because there are many big loops that are either completely unneeded or can be slimmed down a lot. I'll go through all the suggestions I can think of, but you get the idea. Most of these have to do with UHV conditions.

Persia

Control 8% of world territory in 600 AD (Turn 151)
This is more of a fix than an optimization, but it says "in" 600 AD, not "by" 600 AD, but the code checks for the condition every turn until 600 AD. One of those should be fixed.

Control 7 world wonders in 700 AD (Turn 161)
Instead of looping through every wonder and every city, every turn, use events. Just increment the counter when Persia builds a wonder (onBuildingBuilt, and onCityAquired for cities with wonders in them), and decrement it when a wonder is lost (onCityLost). Then you wouldn't have to do any calculations or loops every turn, only when wonders are built/lost. You already do this with the Maya, the only difference here is you have to keep count, which is easy with pickle.

Rome

Build 5 Barracks, 5 Aqueducts and 5 Amphitheatres by 450 AD (Turn 143)
Again, use the onBuildingBuilt event instead of looping through each city every turn and checking the number of each building.

Russia

The Power of General Winter: Enemy units in Russia take damage every turn
This is just a small thing, but how about skipping the loop if Russia is not at war with anyone? I don't know how easy this is to check though...if it requires looping through each civilization and checking, then maybe it's not worth it.


Custom events

This is a fairly big suggestion, so bear with me. What I'm proposing is to make custom events using the Civ SDK (CvGameCoreDLL.dll). For example, for all the "...control...by..." objectives, you could make a plotOwnerChange event that fires whenever the owner of a plot changes. This could be easily done by just adding a couple lines of code to CvPlot::updateCulture or CvPlot::doCulture (probably the latter) to fire the event if the new owner is different. My reasoning for this is that plot owners are not changed every turn, and even if they were, you wouldn't have to loop through a bunch of plots in a region, which would (theoretically) increase the speed of the mod substantially. This would help especially with RFC RAND, since there are no static coordinate-based "regions"; everything is dynamic, which makes it slower (theoretically...I haven't played long enough to find out).

I'm not an expert on this stuff, and I'm pretty new to the Civ SDK, so let me know if there are any problems in doing this, or if, with the way Civ implemented events, this would actually be slower, etc. I am a programmer, but I've never really studied Civ's algorithms before, so like I said, I'm not an expert.

Here are some examples:

Carthage

Secure through trade or take control of 3 dye resources by 350 AD (Turn 138)
Create the events resourceAquired and resourceLost, and have a counter that increments every time Carthage aquires a source of die, and decrements every time they lose one. This isn't really that big, since the current check is pretty fast, but it's a speed improvement either way.

Arabia

Control or make vassal states of Egypt, Spain and Carthage by 1300 AD
There is already a vassalState event, so use that together with the onPlotOwnerChange event I mentioned above. When one of the Civs is either controlled or vassalized, check it off using pickle, and stop checking it in the future. Then, when Arabia gains a plot, check against the remaining Civs and see if another can be "checked off". For example, if you've already vassalized Spain, there's no need to check its territory, so you only need to check if the plot you gained is in Charthage or Egypt.

Mongolia

Control China by 1300 AD
Control 10% of world territory by 1500 AD
Use the onPlotOwnerChange event mentioned above. When Mongolia gains a plot, calculate the land percent and store it using pickle. If it becomes 10%, Mongolia completes the objective and the check can be skipped in the future.

The Netherlands

Secure or get by trade 7 spices resources by 1775 AD
Same thing as Carthage, but Spices instead of Dye.

America

Secure 10 oil resources by 2000 AD
Same thing as well.


I hope this helps in making the mod faster. Please note that I've never tried using custom events and am unable to, as I don't have Visual Studio on my computer, and am not planning on getting it. (Is there a "lighter" version of it somewhere? Preferably free? I'm not talking about VS Express.) I have, however, tried a couple of the Python optimizations, and they've helped noticeably. Unfortunately, my changes accidentally got overwritten when I updated the mod, so I am unable to share the exact code differences with you.

Cheers.

EDIT: I'll attach the latest version of my little "RFC Optimized" project here so it's easily accessible.

In the words of the Bible: Listen, all who have ears.

I'm going to try implementing some of these changes now. Someday maybe I'll spend a few hours to optimise the Python code in my mod... (yeah right )

I think Rhye is out of the loop in Japan at the moment, but given how crappy my PC is I'd be all for anything which speeds up RFC. These suggestions sound good at first read.

this sounds very helpful, it would be fantastic if the slowness can be reduced as that is RFC's only big problem

Interesting, Very Interesting.

Indeed. I stopped played after I finished to long games as Japan and England in which the time in between turns ended up being more than five minutes in each one.

Anything like this would be a Godsend.

Yeah I read the "Going to Japan" thread right after I posted this. Well, I guess I'll try to implement some of these myself. I'll let you guys know if there's a major speed improvement, and if there is, I'll send Rhye the source code so he can implement these changes into the official version.

Theoretically, this should just about double the speed of the mod, but like I said, I'm not an expert on how Civ does its thing, so I'll have to actually try it out to see. But it seemed to work last time I did it.

Stay excited.

You did it before(gape)!

Make that two, not 'to'.

Anyway, I'm excited. The time between turns is the only thing that turns me off about the mod.

Don't mean to hijack, but are there any user settings that would speed up performance?

kravixon said:

Make that two, not 'to'.

Anyway, I'm excited. The time between turns is the only thing that turns me off about the mod.

Don't mean to hijack, but are there any user settings that would speed up performance?

Click to expand...

Nope, there was a discussion about this in another thread. Even getting a better machine won't do much. Hardware only helps up to a certain point; from then on, it's the software that needs to be optimized, which is what I'm planning on doing.

i say great idea RFC optimized

So I have good news and (kinda) bad news. Bad news is that I'm still working on the fixes I mentioned in my first post. Good news is that I found a small thing that made the mod a hell lot faster; namely, using "cPickle" for storage instead of "pickle". cPickle, according to the Python documentation, is 1000 times faster than normal pickle, so I thought I'd give it a try, and it worked! I just changed two lines and now the game is considerably faster, although still not what I would call "fast". (Maybe about a 50% speed increase?)

Note that this is not an official release of what I'm working on ("RFC Optimized"), but this little change made such a big difference, I had to post it here. Rhye can include the change himself, but he's currently in Japan, so I thought I'd share it now so you guys don't have to wait.

Enjoy, and stay tuned for more optimizations!

Oh, I forgot to mention some things:

1. I have only made the change on the BTS version (1.181 to be exact). However, there are only two lines changed, so if you want to implement the changes yourself in a different version, go to RFCUtils.py and StoredData.py in the mod's Python folder (RFC\Assets\Python) and change "import pickle" (somewhere at the top) to "import cPickle as pickle" in both files. Then you're all set.
2. This does not make any changes to gameplay. The only difference is that the game will run somewhat faster.
3. This is totally backwards compatible, so it will not break any current saved games.
4. These changes can also be made on RFC Rand. In fact, it seems to work even better on that (I'd say about 3x as fast).

Give it a try and let me know what you think. In the meantime, I'll be thinking of other ways to speed up the mod.

any chance of a multiplayer friendly version?

I replaced the files with same names in the RFC Python folder, did I do it right?

If this thing works, I see absolutely no reason why Rhye wouldn't add it in the next update.

Great work, musicfreak!

ive been playing a bit of a game with it and it has sped up the game the biggest difference is with congresses, they are now much faster

cPickle is an excellent discovery, I can't see any reason why Rhye wouldn't implement it in the next version. The OP suggestions also sound very sensible.

I replaced the files with same names in the RFC Python folder, did I do it right?

Click to expand...

Yes.

any chance of a multiplayer friendly version?

Click to expand...

For now, no. But if you're feeling adventurous, you can use the instructions in my last post (list item 1) and apply it to every file that has that line in the multiplayer version. The thing is, I don't really have time to make a separate download for every version of RFC (it's hard now that Rhye added RFC RAND to the mix), so I think I'll just post instructions so anyone can do it on any version. Once I'm finished with my little "RFC Optimized" project, I'll post it here for all single player RFC versions, and if Rhye wants he can implement the changes in the other version for everyone, but I simply don't have the time to do it myself. Sorry.

Great work, musicfreak!

Click to expand...

Thanks!

ive been playing a bit of a game with it and it has sped up the game the biggest difference is with congresses, they are now much faster

Click to expand...

Yeah, Congresses use pickle a lot (probably more than anything else in the game), so that's why.

cPickle is an excellent discovery, I can't see any reason why Rhye wouldn't implement it in the next version. The OP suggestions also sound very sensible.

Click to expand...

Thank you. I'm also going to experiment with marshal instead of pickle, which is supposed to be even faster, but we'll see what the tests say.

Update!

I found more files that could use this optimization. I've also done a couple other small things that I'm not going to try to explain. Again, this is for BTS only, other version will come soon, don't worry.