1. We have added a Gift Upgrades feature that allows you to gift an account upgrade to another member, just in time for the holiday season. You can see the gift option when going to the Account Upgrades screen, or on any user profile screen.
    Dismiss Notice

[mod] Religion and Revolution Extended (modmod of Religion and Revolution)

Discussion in 'Civ4Col - Mods and Files' started by Nightinggale, Jun 11, 2013.

  1. raystuttgart

    raystuttgart Civ4Col Modder

    Joined:
    Jan 24, 2011
    Messages:
    5,516
    Gender:
    Male
    Location:
    Stuttgart, Germany
    There is nothing touched more often in Civ4Col logic than the logic about professions.

    That is why I was extremely reluctant about having new features that add new computation intensive logic about professions.
    (e.g. Inventions that change the professions allowed)

    Well, I guess all DLL modders share that impression. :)

    Well, as a DLL modder, it would probably be quite easy to change that.

    But it will be a really bad design decision for others that don't do DLL modding and might be interested to use your modmod for their own private version.
    The majority of people that occassionally do modding does not know anything about the DLL.

    Thus you might lower interest of others to participate in your project ...
    Maybe you might reconsider this. :dunno:

    Things like that are usually a good idea. :thumbsup:

    I tried to do that with many XML settings to avoid reading from XML or the relatively slow FVarSystem.
    This is one of the main points by which we have increased performance in RaR.
     
  2. Nightinggale

    Nightinggale Deity

    Joined:
    Feb 2, 2009
    Messages:
    4,356
    It needs to hardcode the array length at compile time. However now that I think about it, nothing really bad will happen from using a bigger number. The array contains pointers and if coded correctly those pointers will not be accessed and will remain as NULL pointers. This mean each player will use 4 bytes for each extra pointer, which is within acceptable waste. It will slow down player init a little bit though, but that's so little and so rarely called that I don't really care about that.

    It would then have to check if the allocated size is the same or bigger than expected by the XML. It would also mean I should review my additions to remove the new hardcoded number from some other places.

    Right now the XML loading ends with a check and it will make a popup like this if you add a profession
    CvEnums.h has a line saying
    Code:
    NUM_PROFESSION_TYPES = 69,
    If you get that error popup (not assert, popup), then change that line to 70 and recompile. It doesn't take much C++ knowledge to do that.
     
  3. raystuttgart

    raystuttgart Civ4Col Modder

    Joined:
    Jan 24, 2011
    Messages:
    5,516
    Gender:
    Male
    Location:
    Stuttgart, Germany
    That is true, but a huge part of the modders does not even know how to compile a DLL.

    It is your choice of course. :thumbsup:
     
  4. Commander Bello

    Commander Bello Say No 2 Net Validations

    Joined:
    Sep 3, 2003
    Messages:
    3,858
    Location:
    near Koblenz, Germany
    I have to assist Ray in that.

    Many people are shy of meddling with the DLL.
    Furthermore, as mods have a tendency to be copied and re-used and changed and copied again, you'll never know which way your DLL will go. It could theoretically end in a mod "XYZ" being appealing to a new modder who just doesn't make use of it because of the (theoretical) need to change the DLL.

    So, this idea of yours
    sounds like a good idea. :goodjob:
     
  5. Nightinggale

    Nightinggale Deity

    Joined:
    Feb 2, 2009
    Messages:
    4,356
    Luckily the cache mean the actual calculations are done quite rarely. I don't mind making the calculations more complex, as long as it works with the cache, which mean it profession X for player Y needs the same each time and isn't affected by say buildings in the colony. I completely agree that it really is a dangerous function to make just a bit slower without the cache.

    That would actually be easy to do. I have considered inlining some of those small often called global functions and reducing the return value to a cached int would make it even better. I think that would be just as fast as hardcoding the value (well almost) except that it involves a risk of cache misses. However the more often it's used, the more likely it is to be in the cache. I will also place such caches next to each other in memory, which mean the most likely scenario would be to have all or nothing in the cache, in which case it would be unlikely to get a cache miss.

    I will implement this and do the same to some other XML generated arrays (unit types and so on). There is one way to find out if the impact is great enough to make us able to tell the difference.

    I decided to remove the hardcoded numbers. Maybe I should allocate memory for 84 professions? That would leave 15 free slots, enough for most people. Sure I could set it to something insane like 200 and it would never run out, but that sounds like overkill to me.

    Btw the number of yields is hardcoded and I have no intension of changing that.
     
  6. Nightinggale

    Nightinggale Deity

    Joined:
    Feb 2, 2009
    Messages:
    4,356
    I learned something new. Turns out that the player constructor is called before reading the XML files while player init is called after reading the XML files. I'm not 100% sure, but it's likely what caused me to look for a hardcoded value in the first place. Either way I managed to find a way to remove the hardcoded value and the DLL is once again open for XML changes.

    I also got rid of the hardcoded value for number of units, though the price for that one is adding 4 bytes to each city (colonies+native settlements). I couldn't find a clean way to get around that.

    To my knowledge there is no longer any hardcoded values like that except for the number of yields.
     
  7. Commander Bello

    Commander Bello Say No 2 Net Validations

    Joined:
    Sep 3, 2003
    Messages:
    3,858
    Location:
    near Koblenz, Germany
    Thanks not only for what you are doing, but for your explanations as well! :goodjob:

    I think a good explanation for the reasoning behind the doing is worth nearly as much as the doing itself.
     
  8. orlanth

    orlanth Storm God. Yarr!

    Joined:
    Nov 17, 2001
    Messages:
    1,776
    Great! :thumbsup: My thanks as well for your modding contributions. Reducing hardcoding will really enhance the versatility for modmods, adding a few bytes seems a small price to pay for the added flexibility.
    IIRC, one other remaining bug in RaR/TAC was something about the French civilization trait not applying correctly during player creation - Ray could you refresh my memory about that, I wonder if what you found there could be related to that issue?
     
  9. raystuttgart

    raystuttgart Civ4Col Modder

    Joined:
    Jan 24, 2011
    Messages:
    5,516
    Gender:
    Male
    Location:
    Stuttgart, Germany
    It is related to the trait "Cooperative" which gives a small relationship bonus to Natives.

    To problem occurs only for human player when he chooses the French.
    (It does not occur to AI.)

    It is related to initialization order of Players.
    (Human Player is initialized before Native AI players have been.)

    Discussion starts here.
     
  10. Nightinggale

    Nightinggale Deity

    Joined:
    Feb 2, 2009
    Messages:
    4,356
    I managed to get profiling to work with the stack. Now I can see how long a function is used including the time it spends waiting for the functions it called itself.

    I noticed calculating warehouse capacity took 1 minute 54 seconds. This changes when buildings are added/removed and on colony creation/load. Fairly simple to cache, so I did.
    It actually took another two seconds from the next turn events.

    I merged it with the cache for teaching level as they should update at the same time. I made a function to call those two functions meaning if some future cache needs to update on building change it only needs to be added in one place and it will hook into the code at the right places.

    That's unrelated. My issue was about the order stuff done inside one player while the French issue is the order of players. Two completely unrelated issues even though they sound similar.
     
  11. Kailric

    Kailric Jack of All Trades

    Joined:
    Mar 25, 2008
    Messages:
    3,095
    Location:
    Marooned, Y'isrumgone

    This is an interesting thread, I'll have to go back and read from the start. I myself am no programmer but rather a tinker. I completed my first major project, the Privateer mod, not even knowing anything about Debug. It was nuts, I know. I guess I just have a knack for "if this, then do that", and just knowing that for the most part the computer only reads one line at a time, so you have to keep order in mind always. I am, however, enrolled in college and will eventually take some programing classes.

    One thing in my Medieval mod that I had to add that gets called a lot is a for loop of the Civics(I used Civics for Techs). It has to loop through nearly every civic to test if say, a profession is allowed or disallowed. Then checks to see if the Player has learned that tech. I started thinking on ways I could do this without having to cycle through the Civics each time. Creating arrays for player allowed Buildings, Professions, Units, Promotions, Yields, etc was one idea. I made sure though to that if a Tech disallows a Profession that it allows a replacement or upgrade. Then the code just cycles through the players units and forces a profession change. This fixed a lot of errors I was getting for professions not allowed.

    Anyway, like I mentioned to ya, a lot of my code needs overhauled. I create stuff in a hurry just to test if I liked it or not or it would even work then if I liked it I left it as is for the moment... things like that.

    Thanks for all the info you are sharing, I'm loving it.
     
  12. orlanth

    orlanth Storm God. Yarr!

    Joined:
    Nov 17, 2001
    Messages:
    1,776
    Yes very interesting discussion - even for someone with the bare rudiments of Perl like me :crazyeye:
    If Nightinggale is using a cached array that stores yield requirements of profession X for player Y, I wonder if it could be possible to extend that for your tech requirements as well. (eg a cached value of -1 means that player can't currently access that profession.) Then you could just update the cache when a tech is gained/lost, and again get a big savings. :cool:
     
  13. raystuttgart

    raystuttgart Civ4Col Modder

    Joined:
    Jan 24, 2011
    Messages:
    5,516
    Gender:
    Male
    Location:
    Stuttgart, Germany
    Until now, most discussions about performance were by private messages between a few modders interested and "skilled" enough to understand what it was all about.
    If there is really a lot of interest for a public discussion about improving performance, than maybe a separate thread could be opened for it.

    Isabelxxx for example was also very interested in that topic.
    (I am not sure however, if she is currently still modding Civ4Col.)

    Generally however, there are a few major topics involved:

    1. Logging
    2. Caching / File Access
    3. Graphics
    4. Game Logic
    5. Structuring of XML

    Each of those we tried to improve in RaR.

    We had spent a complete release on the topic performance.
    We simply had to do it due to the incredible size of the mod (Mapsize, Number of Players, Number of Graphics / UnitArtStyles, ...)

    But of course, a lot more can be done.

    Basically it comes down to a very simple summary:

    If you keep your mod tidy and well structured, you will have good performance.
    You need to take care of everything. DLL, Python, XML and graphics.
    DLL alone will not solve all your performance issues if your graphics are messed up.
     
  14. Nightinggale

    Nightinggale Deity

    Joined:
    Feb 2, 2009
    Messages:
    4,356
    I have done stuff like that too. Modifying your limits to fit the task rather than reduce the task to your limits can be a good thing. However it's a fine balance between making something great and making something slow which is impossible to test for bugs.

    While I agree that it sounds like it could be really slow, I would like to profile before judging it. Sometimes the slowdown is in the place you least expect. The slow looking code might be fast enough while the fast code is called 7000 times instead of 5 because of a typo in a loop condition.

    I tried looking at the logic, or rather recode how certain stuff is calculated. It turns out that RaR is actually decent in that regard and "improvements" turned out to make no difference, or in one case made it slower. In fact I considered all 5 of those and found 2 to be the only place where I could make noteworthy performance improvements.

    Yeah. Slowdowns can happen in all kinds of code. Luckily the stack based profiling looks into this as well because it measures the time the DLL waits for external stuff, like python. Profiling will not help fix the problem, but it will tell where it went wrong.
     
  15. Nightinggale

    Nightinggale Deity

    Joined:
    Feb 2, 2009
    Messages:
    4,356
    I cached the profession IDs for fishing and whaling boats to avoid reading the XML every time. This reduced the wait around 1.4-1.5 seconds (according to profiling). The savegame which originally took 40 seconds to get to next turn is now down to 30 seconds.

    I don't get it :confused:

    Can I get a savegame where this happens? Preferably one where all I have to do is to hit end turn and then it happens every time.

    I looked into CivilizationIV.ini and there is a line saying
    Code:
    AutoSaveInterval = 4
    It's likely a good idea to set it to 1 to make sure you have a savegame just before the assert took place.
     
  16. Nightinggale

    Nightinggale Deity

    Joined:
    Feb 2, 2009
    Messages:
    4,356
    I committed project files and makefile. No change to the DLL, but much easier for people who wants to compile themselves.

    The makefile is the new one mentioned in my signature. I really should make a more interesting looking signature, but it will have to do for now.
     
  17. agnat86

    agnat86 Warlord

    Joined:
    Jun 30, 2011
    Messages:
    218
    Hello Nightinggale. Great that you are working on this mod!:goodjob:

    I'm playing R&R a lot, and I'm currently trying to learn how to make modifications myself. I've already done a lot of balancing changes in XML, and also a few in cpp.

    I've done the following things:

    - Changed the order and abilities of Founding Fathers. In R&R, lots of great FFs were added, but in my opinion some of them appear too late in the game, while others are underpowered. After playing a few games, I noticed that I never took some founding fathers (like Orellana), while other appeared at a point when they weren't very useful anymore (Villegagnon).

    - Changed the traits and attitudes of many native civs, to make them more diverse.

    - Rebalanced the negative influence of population growth on native attitude.

    - Allowed units with mountaineer to enter peaks.

    - Many minor balancing changes to units, resources, improvements.

    I still have a long list of things I still want to change, like adding events, further diversifying native civs and rival colonies and improving the AI of rival colonies, but these generally take up more time, and first I would like to get a clearer idea of who else is working on R&R, and what their plans are.

    Please let me know if you are interested in these things for your Modmod, then I'll post my changes here.:)
     
  18. Kailric

    Kailric Jack of All Trades

    Joined:
    Mar 25, 2008
    Messages:
    3,095
    Location:
    Marooned, Y'isrumgone
    I was told Nightinggale is out of pocket at the moment (you better not be reading this while I'm typing!) but from the sounds of it you guys need to talk. It can be tough trying to do things as a team though because everyone has their own ideas on what they do and do not want. So I would suggest to anyone out there trying to work together to concentrate on what you do agree on and then make Modmods for the things you don't :)
     
  19. Nightinggale

    Nightinggale Deity

    Joined:
    Feb 2, 2009
    Messages:
    4,356
    Sadly I'm a bit stalled right now. However I haven't stopped and will commit stuff again soon.

    All of what you wrote sounds interesting. I like the sound of the result and it's something I haven't even considered modifying meaning I wouldn't have done it myself.

    I'm reading it now :p

    I will deal with one problem at the time. First problem is getting potential team members with skills and ideas. That appears to be a far more serious problem than disagreements. However at least a bit of agreement will always be needed and it looks to me like we fully agree so far.
     
  20. agnat86

    agnat86 Warlord

    Joined:
    Jun 30, 2011
    Messages:
    218
    Great! I'm planning to add my changes to the newest version of R&R (probably including Schmiddie's changes). Please let me know it when you want some of them, then I'll share them.

    I'm mainly occupied with changing the natives right now, and after that, I think I'll look if I can make other Europeans more interesting. I've already added new traits and abilities for natives. The next thing I want is to change some native civs into others. I think that currently, the American midwest is overrepresented with 7 civs. I'm planning to remove the Comanche, Navajo and Crow, and add the Haida (Alaska/Western Canada), Ohlone (California) and Timucua (Florida). Of course, this is all rather ambitious for a novice modder, so I don't know if I'll ever get there...:(
     

Share This Page