[Religion and Revolution]: Mod Development

Status
Not open for further replies.
I haven't looked at what was copied yet, but I can reply based on what I read in this thread.

3) Advisor Screen from Nightinggale
The current advisor file in Medieval Conquest is written to be copied to RaRE without modifications. In other words it is the "correct" one. I wrote it generic enough to show both correctly (setting columns and stuff based on XML values) as it allows writing new features for both mods at once. It also mean a bugfix for one likely works on both.

However I'm not sure you will be able to tell the difference in RaR if you used the newest from RaRE or M:C. The difference is likely just coding. (didn't bother to check right now)

What I did change from RaR:
  • Made it more generic, allowing addition of new code more easily.
  • New screen with combined production storage of all yields for all colonies.
  • Fixed wrong column width on production page 3 (copy paste error)
  • Removed buildings page 3 (it was empty)
  • Added missing column on buildings page 1
  • External events to redraw the screen will now always recalculate the front page and never any background page (if you were on say production page 2, that page wouldn't update, but trade and natives would)
  • The clicked button is highlighted when it's the active button (RaR only had highlighting for page 1)

I can't give exact numbers, but it feels faster.
But I can't tell for sure.
My optimization strategy was to improve the slow part of the AI code to reduce wait time for next turn, meaning this is the only part I actually measured as a result. I managed to reduce the wait time by 25%.

Some of it will make the player's UI faster as well, like creating the list of professions for a unit inside a colony. I didn't actually measure that, but I do imagine at least some people will notice a smother colony interface, specially on low end computers.

The things I did, do improve performance only marginally.
(Basic code improvements to have logic a bit more efficient.)
The code is generally not that badly written when it comes to speed. The bad parts is that it recalculates values, which are often the same though the entire game or accessing constant XML values. This is why caching appears to be the most efficient approach to increase performance.
I did find one place I can improve by changing logics. I didn't as I (at least at that time) had a plan to add a feature to that particular piece of code and I easily ended up improving a function I would no longer call. I decided to postpone that until I'm fairly sure it's there to stay for quite a while. Having said that I can tell for sure that it will not have the overall impact like the caching I made.

Several things Nightinggale had done, I can tell by looking at the code, that they increase speed since we did that already before.
(Caching of Globals)

With others it is really hard to predict by simply looking at the code.
(Inlined functions, the "Just-In-Time" arrays, ...)
Here I simply trust Nightinggales tests and examinations.
The inlining was done before I had the profiler working. I don't know how much it actually improves either. However it's a schoolbook example of what should be inlined (all functions returns a variable) as it reduces code size and generates a small speed boost. It reduces the DLL file size by 8k. Speed is unknown, but likely not that great. It is likely not a bad speed improvement when it comes to increase compared to coding time as it was mainly copying one block of code from a cpp file to a header file and then search and replace to add inline. I'm not sure how long it took me, but it's in the area of a few minutes and testing can more or less be reduced to "can it compile".

"Just-In-Time" arrays are not designed to increase speed on their own, but rather memory. The point here is that they aren't allocated until they are written to and unallocated arrays are full of 0. It's primary used for arrays, which are full of 0 from the start and the AI will not change that. The speed goal is to match "normal" arrays, which is more or less true, specially for reading. It does have one speed boost though as it can check quickly if an array is allocated, which in turn is a single comparison for "is all content 0".

The first commit with a working profiler was caching CvPlayer::getYieldEquipmentAmount(). On it's own this cache is responsible for half the speed improvement. However in addition to the caching I did I did notice that whenever this function is called, it is often like "for each yield, call function, check that yield is in colony etc". In other words this is safe to ignore when all are 0. Skipping those checks when the array aren't allocated contributes to 20% of the speed boost. There is absolutely no way to predict this without proper testing and I had no idea how well it would work before I was done and tested it.
This mean this single commit has more than twice the impact than all the other speed improvements in RaRE combined.

The key to writing faster code is to measure and profile. If you ever compile a DLL yourself I highly recommend the makefile I wrote. It not only compiles with all CPU cores, it also allows stuff like profiling, allowing easily identification of which functions and even lines in the code, which uses the CPU time. Without this, I wouldn't have been able to identify the parts of the code to optimize.
 
All I did was:
....
D) Merged Speed and Performance Improvements from Nightinggale
....

Summary:
This is more or less simply a clean-up, to create a better version for all mod-modders.
But yes, it will most likely be published as Release 1.6.
(Just don't expect any new major features.)

Any improvement to speed and performance would be worthwhile! :bowdown:
My computer takes ~ 79 seconds per turn :hammer2:
 
Any improvement to speed and performance would be worthwhile! :bowdown:
My computer takes ~ 79 seconds per turn :hammer2:
Assuming Ray moved everything from RaRE (I still haven't read the changed code) the new speed would likely be around 59 seconds.

It would actually be interesting if you
  • save the game
  • measure how long it takes to reach next turn
  • update to 1.6
  • loads the same savegame
  • measure how long it takes to reach next turn
This way we will have a speed boost measurement from more than one computer. It would be interesting to know if you get the same number as I did. Also given that your computer is slower, your measurement will be more accurate.

The approach I wrote is important as it mean measuring the same next turn event. Replacing the random seed or not using the very same turn could cause the AI to have different input of the calculations, which could affect the result. Maybe it's boring to do the same measurement twice, but that's the way to get a proper reading.
 
I tested the new additions a bit because I had a suspicion that two things might have gone wrong. I found one issue and fixed it while the other was false alarm. To my knowledge the speed improvements should be bug free.

I did encounter one issue though. The new total production and storage button. When hovering the mouse on the button, the popup string is in German even though I'm running the game in English. Not a major issue, but it looks a bit silly.
 
I did encounter one issue though. The new total production and storage button. When hovering the mouse on the button, the popup string is in German even though I'm running the game in English. Not a major issue, but it looks a bit silly.

Yeah, I was tired. :)
(It is corrected in the newest revision.)

@team and mod-modders:
Another revision with further small code improvements has been uploaded.
 
I made some improvements to the code that determines the professions native villages teach, and Ray asked me to post them here, so here they are.

You can find my changes in the codes for "bestTeachUnitClass" in CvCity.cpp and for "AI_idealProfessionForUnit" in CvPlayerAI.cpp.

I've also made changes in CIV4CivilizationInfos.xml to the TeachUnitClass values of each profession for each native nation, because it needed to be rebalanced after I made my changes.

I hope you like it!:)
 

Attachments

  • NativeSpecialties.zip
    158 KB · Views: 48
@agnat86:

Thanks for sharing your changes. :thumbsup:
I have just taken a first look.

bestTeachUnitClass --> Will need to check in detail tomorrow.
AI_idealProfessionForUnit --> What are you trying / intending to do with your changes ?
 
AI_idealProfessionForUnit --> What are you trying / intending to do with your changes ?

This function is used by the code in CvCity.cpp to determine what the default profession for a unit is.

However, in the vanilla version, each unit had only one yield it got a bonus for, either on land or at sea, but in R&R, many units have more, like a Miner getting a bonus for both ore and salt, and a trapper for premium fur both on land and water.

The problem is that the original code in CvPlayerAI.cpp returns the value NO_PROFESSION if two ideal professions end up being both the best profession. For example, because an Expert Miner was equally good at Ore Miner and Salt Miner, the game decided that the best profession for an Expert Miner is no profession at all.

And in the code for BestTeachUnitClass, if the ideal profession is no profession, the part where it looks at the surrounding terrain is skipped, and a standard value is returned, regardless of the village's location. This is why Trappers, Miners, Prospectors and Farmers appeared in places that made no sense: the surrounding terrain was ignored entirely.

I've changed the code in CvPlayerAI to make it look at the best three profession instead of two, because there is no unit that is equally good at three professions. And now, the professions mentioned above appear in places again where they do make sense.

In CvCity.cpp, I only made a minor change in the part of the code that checks for terrain features. It adds or subtracts from a value called iValue, while it should be iPlotValue, as iValue is redefined entirely a few lines down. This bug was already present in vanilla, and it explained why hunters and cocoa planters never appeared in forests and jungles respectively.
 
@agnat86:

Thanks for the explanations. :thumbsup:
I will check all your uploaded changes in detail tomorrow.
(It is midnight here in Germany and I had a pretty exhausting week. :sleep:)
 
@agnat86:

The logic that has been programmed for AI_idealProfessionForUnit is total crap in cases the unit is designed for more than one profession.
(I am not talking about your changes by the way, but about the code from Vanilla.)

Simply saying "Hm well I cannot decide which profession is best if he is skilled for more than one so I will rather decide that there is no best profession." is really poor programming.

I will try to program something useful instead. :thumbsup:
 
This is commited to SVN. :)

No need to start a new game.
Savegames (from revision before) will still be compatible.

Please everybody (with access to SVN) get the newest revision. :thumbsup:

In my reading through various forum posts, I see many references to "SVN." I haven't been able to find what this stands for or how you gain "access."

By the way, thanks for the update of v1.5.
 
In my reading through various forum posts, I see many references to "SVN." I haven't been able to find what this stands for or how you gain "access."

SVN is short for "Subversion".
(It is a configuration management system hosted on a server.)

We used it to do cooperative work within the team and with partners and to share work with a few selected modders from other projects.
(Basically it was the main technical platform for all our development.)

Only these people have access to it.
All others always needed to wait for public releases.

But don't worry, I am planning to publish Release 1.6 once it is finished and properly tested. :)
(I simply do not have a lot of time left for modding, so I am progressing slowly.)

Spoiler :

Release 1.6 is simply a (last) clean up from my side without any new features.
(Intended to give an even better base for future mod-mods.)
  • fixes of minor bugs reported
  • general code improvements
  • performance improvements
  • memory usage improvements
  • AI improvements
  • balancing improvements
  • improvements to screens
  • ...
 
@team and mod-modders:

There have been several small new revisions uploaded to SVN.
(AI improvements, speed improvements, balancing improvements, ...)

@agnat86:

I have included small parts of your code changes considering TeachUnits at Natives.
The rest of your code I could not use, because I had completely reprogrammed the method.

I have also adjusted Founding Fathers a bit, since that was one of the points you had mentioned.
(Quesada and Orellana have been changed a bit. Also Quesada has switched Position with Drake.)

By the way:

I think I am done with the improvements.
So basically the current revision in SVN will probably become public Release 1.6.

If I find the time, I will play a few rounds and probably publish this in about 2 weeks or so.
(I have had several games running in Autoplay to find bugs, but everything was fine.)

As always, if you want to play with the current revision, please clean cache and start a new game.
 
Could you also add the war paint?
Even if it's just a small aesthetic change, I find it really appealing
 
Could you also add the war paint?
Even if it's just a small aesthetic change, I find it really appealing

Sorry, but I probably won't. :(
I simply have almost no time for modding and don't want to introduce graphics I have not properly tested myself.

I only got active again because (potential) mod-modders have told me about performance issues, a few balancing issues and a few small bugs.
That is all I really want to take care for now.

Anything else is "Nice-To-Have" which I will leave to mod-modders or to the other team members. :thumbsup:

If some other team member would like to implement it, properly test it in game and then upload to SVN so I don't need to spend time myself, that would be fine for me of course.
 
@team and mod-modders:

New revision available in SVN.

AI was sometimes stupidly cutting down a TerrainFeature (Forrest, Jungle, ...) without taking care of the improvements already built and thus sometimes rendering them useless.
(E.g. Lodge without Lumber, Trapperhut without Fur, Cocoa-Plantation without Cocoa.)

Now of course, this does not happen anymore.

I am currently trying to focus on finding such small AI flaws in my tests even if I don't have that much time for intensive testing anyways.
(I never really had a lot of time for "fine tuning" like that, because I always had more important todos, like fixing a real big AI issue, programming a feature, fixing a bug, improving performance, ...)

Edit:
Otherwise everything is going pretty well in my tests. :)
It is just a pitty that nobody else is testing a bit as well ...
 
@team and mod-modders:

...Otherwise everything is going pretty well in my tests. :)
It is just a pitty that nobody else is testing a bit as well ...

I was just going to comment about that the other day. Thanks for your dedication. I'm sure it's discouraging when you're not getting alot of support or help.
 
Thanks for your dedication.

Ah well, it is simply that I would really like to see some of the mod-mods finally take off.
And I think that an even better base mod might help.

I'm sure it's discouraging when you're not getting alot of support or help.

There are simply no "flashing" new features in development.
That is usually not very interesting for modders since improving and testing is boring work.

If some of the mod-mod projects really get going, this will probably be much more interesting for modders again.

However all together a lot of code changes were made to improve speed, improve memory usage, improve AI, fix minor bugs, ...
(Especially some of the changes I adapted from Nightinggale were quite heavy.)

I would have had a better feeling to release this next week, if some more people would have tested.
But everything should be fine anyways. :)

Edit:
I finally got first feedback considering tests from others. :thumbsup:
Everything is fine so far. :)
 
Hi Folks!

I have made some changes of the mod for my personal taste and due to the fact that some players also mentioned that it would be nice if some small things could be changed (the train for example), I have decided to release a small Mod-Mod of the next version of RaR which is going to be released on short notice.

This Mod-Mod will contain the following changes:

- Train = changed to „carriage“
- Trainstation = waystation /coaching inn
- Route Models correspondingly changed:
- - Plastered road = country road
- - Railray = plastered road
- Some buttons changed
- Two new unitys implemented:
- - Settler's milita (weak defense unit)
- - Line infantry (strong infantry unit / in addition to colonial troops)
- Colony Names of England changed (for example „Virginia“ instead of „Jamestown“)
- Unit art styles of England changed into „red“
- Unit art styles of England in revolutionary era changed to „Continental style“ - but this is optional (file path's have to be changed manually in the beginning of the revolution)
- Unit art styles of England Ships in revolutionary era (also optional)

Regards

Schmiddie
 
Status
Not open for further replies.
Top Bottom