v1.61.0002a (2nd release)

[Impaler[WrG]]

Is anyone else experiencing that Memory leak or have any idea what might be causing it?

I will be fixing up the UU mod with the modified Python and possibly including some other small XML mods. Can we aim for a release this coming weekend?

 

[SpoiledFruit]

Thats what I was planning to do. I will release a new package this Saturday, so get your changes ready and load them up before Saturday evening.

Let me know if there are any new files I need to add to the base package (not the example mods).

 

[SimCutie]

UN Vote seems to be broken. All zero votes..
Please test it to confirm that it is reproducible error or not.

 

[SimCutie]

Why we should not take "big-combined kitchen-sink DLL approach"?
It is matter of resource.
Yes, Firaxis or any one could have various "nice-to-have" features added to DLL. But many of them are not welcomed by or not utilized by most game player.
Even vanilla Firaxis Civ4 is quite a resource hog that many not-so-top-calss machine has hard time to run it smoothly, especially machine with 512M RAM. Many player already complains on Civ4 for late game slow down and excessive memory usage even with 3.0GHz CPU and 1G RAM machine and suspects memory leak.

So we , and other MODer, must very careful when adding burden to memory.
Code memory size is nothing. Adding few 100K bytes of code will not make big difference in memory foot print.
But data member addition or array expansion is different. For example, adding arrayfor class member or new commerce type or yield type has quite large impact on each class and each instance.
Adding few mod will not make big difference. But if we take "kitchen-sink approach", it will become larger and larger as we adds more and more mods to project and bloat like rollling snow ball and take a big byte on size of memory working set. The size increment may not be make or break difference but it can be last straw to some players.
Extra CPU time for handling new MOD would be highly dependant on nature of the MOD. But we also must be careful not to burden CPU, expecailly in turn time.
As DLL become heavier, more and more player and modder will show cold shoulder to our project becauase our project DLL is too big and slow.
So we must maintain skinny, lean and mean DLL for COMMON platform.
Common DLL shoud be minimum common denominator for most of players.
This is why we should keep DLL as lean as possible. We should keep extra keen eye on resource usage of our project. We must always weigh benefits of MOD for general players and extra resource it uses.

This is why we should not take kitchen-sink approach.
If we add particular MOD to project, the mod should be general enough and should be welcomed and can be utilized by most players so that it can justify extra resource it uses.
Just adding XML attribute or switch to turn off the the feature is far from enough.

 

[Impaler[WrG]]

UNVotes: Though I had fixed that, will double check it

I have found that the DLL currently on the CVS is running and leaking BUT it is NOT the DLL you get from compiling current files! When I compile current files the resulting DLL is a MB larger and unstable and will imediatly freeze/coredump when pressing continue from DoM (all other Assets the same). If I play DLL only its stable, so Python interaction is the problem. I will try to see if it also leaks memory with DLL only.

I ofcorse disagree that we are coming anyware near a CPU usage issue, most especialy one caused by XML style additions. Warlords might even have higher system requirements then current game and Mores law is on our side. At the present rate of Moding we wont have any problems.

 

[Impaler[WrG]]

Memory leak is the same with DLL only run, by watching the Task Manager I can see the memory consumption go up each time I hit End turn, a large game will consume 50-100MB of additional memory every turn so its more acurate to say we are Hemraging memory.

 

[SpoiledFruit]

It probably is a new or modified function that is not releasing its calls once it is done with is. Are you shure all the destructors are in place and releases the memory block when it is done?

Also we need to get the source code stable so I can compile all the new changes, the old DLL does us and me no good cause its too old.

 

[Impaler[WrG]]

I've been running the DLL whial watching memory trying to see if their was a pattern, all I can say is that in the early game memory consumtion goes up almost every turn often a small amount like 50KB sometimes lots more. Occasionaly it dosn't go up at all, it always goes up durring combat.

If one of the game objects wasn't releasing its memory then we would expect memmory to stay high when a unit/city ect dies and is deleated but were seeing memory go up just from doing nothing so I suspect its a python call not releasing a pointer. What do SimCutie and 12Monkeys think? Its going to be up to yall to fix these issues if their in the Python (the crash at DoM definatly is), the memory issue is still a mystery to me.

And SimCutie, stop making non-substantive changes to my code to satisfy your personal tastes in how to coment out a line of code, you need to stop being so anal about other people work and start bug fixing your own code which has never been anyware near stable. Do you do any testing before you commit these changes? We have seen code that wont compile and DLL's that imediatly crash when played being commited. I am not perfect myself but I feel I have keep it reasonable and have made shure I atleast dont make the DLL unplayable. I think we should go back to individual developer braches so these instabilites dont bring the whole project to a screeching halt.

 

[SimCutie]

I've been running the DLL whial watching memory trying to see if their was a pattern, all I can say is that in the early game memory consumtion goes up almost every turn often a small amount like 50KB sometimes lots more. Occasionaly it dosn't go up at all, it always goes up durring combat..
<.....>

Memory consumtion on end turn itself is not problem. Original DLL also consumes memory at end turn. What is important is the amount of consumtion. Please test with same save file with both DLL (original /Project)
If difference is big enough, it may be sign of problem.

And SimCutie, stop making non-substantive changes to my code to satisfy your personal tastes in how to coment out a line of code

It is NOT non-substantive change.
You are not understanding how CVS update/merge work. If you does not preserve original line same as original, CVS update can not find orginal line and think the line is deleted and fails to automatically merge changes from multiple developer. other developer should manually merge the changed line. It failes purpose of CVS. I will not care your commenting style as long as
1) Original line should be preserved as it was. No change by line itself.
2) Each insert/change block should be marked clearly with START.. END
so that identfiable without parsing C++ block syntax.
3) Each change/addition should have no ambiguity where the change ends.

These are imortant substantive matter that can't and should not compromised.

you need to stop being so anal about other people work and start bug fixing your own code which has never been anyware near stable. Do you do any testing before you commit these changes? We have seen code that wont compile and DLL's that imediatly crash when played being commited. I am not perfect myself but I feel I have keep it reasonable and have made shure I atleast dont make the DLL unplayable. I think we should go back to individual developer braches so these instabilites dont bring the whole project to a screeching halt.

Compiler failure is your personal problem. how can I do development if it would not compile. You didn't understand how preprocessor symbol work. Read C++ manual on preprocessor symbol.
Instability? I can finish game from start to end without hitch with my code ( except UN vote mentioned above).
Your code seems not tested in debugging mode. It fails on some debugging assert statement. I fixed them all and put these change into the CVS. But you overrided/ignored the change and backed to your buggy code. I guess that you CVS update failed to merge changes from my code. ( due to missing "original line" ). This illustrates why we should preserve original line for CVS merge.

 

[SimCutie]

How could I add tags with out giving them a specific action/effect in the game? Perhaps your thinking of having data which is functionless at the C++ level and is just used for export to Python ware it gets used for what ever each indiviudal moder wants? I basicaly try to copy the Firaxis methodology (datamember, InfoClass read/write of the Tag, Schema with the Tag optional, code to effect the change, and Python export of data).

See "[SDKMOD] Scriptable Leader Traits" (http://forums.civfanatics.com/showthread.php?t=175537 ) by TheRopez.
This is just version 0.1 and rather primitive for now.
But it looks much more promising and flexible than Impalers leader trait mod.
I think it is the approach we should take.
This Mod is much appropiate MOD to include in our project than unmoddable Impaler's leader trait mod.
IMHO, it would be better if impalers should announce your mod as separate personal SDK MOD, not as part of our project to enhance modability of SDK DLL.

 

[SimCutie]

I conducted a simple, crude experiments on memory leak..

I played a peliminary session with a save file to stabilize disk cache buffer.
Each main session was about 20 turns long after 10 "warm-up" turns starting with same game save file.
I played game with vanilla DLL ( with same save file) and mesured increment of total memory usage between these 20 turns.
Then I played with our project DLL. Again I played with vanilal DLL, and played again with project DLL.
For each session, I measured memory usage increment during same 20 turns.
Conclusion:
The memory usage is little different depending on DLL. But it was far from meanigful amount (less than few percent of vanilal DLL increment).
I can not say that there is meaningful amount of additional memory leak due to project DLL.
This game save file is early-mid ear( about turn 150 out of 430) of quite standard game, mostly peaceful world.
This is very simple and crude experiment so I can not conclude that there is no memory leak, too.
But it seems that there is no apparent and big memory leak hole out there.
It will take longer and more controlled expreiment to detect more subtle memory leak.

 

[Impaler[WrG]]

Ok I have reached a conclusion because I am sick of delays caused by other peoples unstable code and elitist atitudes, I am going to stat my own branch on the CVS and include only the content I wish on it, how I want it, when I want it. I will aim for public release every 2 weeks and will rely heavly on outside mods. If it is agreeable to TGA and other members of the project these can be released under a "CCCP all inclusive" title and SimCutie can focus on "CCCP lite" or what ever he wishes to call such branch as he creates and maintaines. Anyone may copy materials from my branch but no one but myself will commit to it (Such configuration is I belive possible with SF and am asking for it to be done once I have the branch up). If and when code made by other members of the project is stable and usable I will consider it for inclusion.

 

[The Great Apple]

As much as I hate to see division, I feel that an "all inclusive" .dll is much more what I had in mind when I first posted about this project. My original intent was not to target the modders, but the players, giving them more options to improve their game, with the emphasis being on the fact that the options are options. Modding content has it's place, but is not the be all and end all.

I would certainly support many more releases, along with much more content, as long as they are kept stable. If other people's code causes errors, then it just doesn't get included, and we release the mod without the features.

If we stick to a set release timing, say, every two weeks then everybody will know what is going on. If somebody sumbits code which doesn't work, it gets dumped, and they have two weeks to fix it. We should keep a dev branch, where work in progress is added, and tested as it is made, and a stable branch, where only work which works fine with the existing .dll is added. Stable is then released every 2 weeks regardless (unless there is nothing added at all, but I can't really see that happening).

 

[Maniac]

Have you guys considered starting a public thread where modmakers can request addtions that would allow them functionality without changing the core game?

I have posted this in the Mod Components Request thread, but of course I'm actually hoping these would be included in CCCP.

Btw, sorry for not being active here. After the expansion pack is released though and the CCCP works for it, I hope to start planning a liberal conversion of SMAniaC to Civ4 and will use CCCP for it.


I've got some requests, most to make SDK stuff moddable in XML.

Great People

Currently the GPP threshold increase is 100 for the first ten, then to 200, 300... Except for the first ten threshold increases, the GPP threshold is unmoddable. So I'm wondering if it would be possible to mod in GlobalDefines.xml:

• set the increase (or decrease) rate yourself instead of it automtically switching to 200.
• define after how many GPs the threshold increases changes (instead of the current fixed ten).

Hurrying Production

Currently you can set units to hurry production with < iBaseHurry>0< /iBaseHurry> in Civ4UnitInfos.xml.

However currently you can only hurry building production this way. I'm wondering if it would be possible to make units hurryable.

Also currently production just adds an absolute number of hammers. Would it be possible to let this be affected by production modifiers at the city where you want to hurry?

Eg a supply crawler adds 100 hammers, but you're playing an industrious civ and have a resource which doubles production, so the supply crawler adds 250 hammers. Or the other way around, you have a civic which gives you -20% military unit production, so the crawlers should only give 80 hammers when hurrying military unit production.

As far as I can see there should be some exceptions though to avoid that modifiers are counted twice.

• Buildings shouldn't add their production modifier. Eg a forge with +25% production shouldn't make the supply crawler give 125 hammers.
• Other general production modifiers, eg any civic or leader trait that affects ALL production.
  
  
  Specialists increasing the unit upkeep limit
  
  Allow specialist to increase the unit support limit. Useful for eg a Soldier specialist that allows you to support three extra units and give some GP points to get a Great Warlord.
  
  I guess this could be done in the SDK by adding these tags from Civ4CivicInfos.xml to Civ4SpecialistInfos.xml
  
  < iBaseFreeUnits>0< /iBaseFreeUnits>
  < iBaseFreeMilitaryUnits>0< /iBaseFreeMilitaryUnits>
  
  
  Terrain improvements and units providing resources
  
  Or take them way, by giving a negative value. This could allow a quantified resources system. Thus make these tags from Civ4BuildingInfos.xml usable multiple times in Civ4SpecialistInfos.xml and Civ4ImprovementInfos.xml:
  
  <FreeBonus>NONE</FreeBonus>
  <iNumFreeBonuses>0</iNumFreeBonuses>
  
  Multiple times so you could have a specialist take away raw resources and convert it into an industrial resource.

 

[SpoiledFruit]

release 1.61.0003a is now out. next version will be v1.61.0004a

I'm also starting a new thread with our next version.

 

[Impaler[WrG]]

I'm trying to play the downloaded release and am getting Python errors from all of 12Monkeys new game Utility calls (this never happened before), its seems that his CvGameUtil.py is not being found/used. And as always I get the CustomButton problem at the Dawn of Man screen which forces me to quit Civ4.

I am wondering do people test with Python Error supressed? This is the only way these bugs could have gone un-noticed by their creators for so long. Its critical to have all the Python Error popups and logs on when testing the game or else your not acomplishing anything.

 

[The Great Apple]

The Python exception at DOM

I too have this.

From the log:

Traceback (most recent call last):

  File "CvScreensInterface", line 598, in forceScreenRedraw

  File "CvMainInterface", line 672, in redraw

  File "CvSimCutieMainInterface", line 606, in updateSelectionButtons

  File "ActionButton", line 314, in DisplayCustomButtons

  File "PyCivSim", line 333, in triggerEvent

  File "CvEventInterface", line 44, in onEvent

  File "CvSimCutieEventManager", line 455, in handleEvent

  File "DEMO_ActionButton", line 43, in onShowButtons

  File "ActionButton", line 291, in createButton

  File "ActionButton", line 178, in __init__

AssertionError: CvPyActionInfo().create() failed
ERR: Python function forceScreenRedraw failed, module CvScreensInterface

 

[SimCutie]

I changed source to fix Python exception at DOM.
But I am not sure that it is really fixed or not bacuse I can not reproduce the DOM error in my computer ( Python exception Error report enabled, of course). I got such error before long in 1st release days but I have thought that it had been fixed in 2nd release and did not get the error from then ever. It was race condtion error.
So please heavy stress test this new version in your installation to make sure that it will not cause trouble in most of players computer and tell me the error happens again or not.

I also added AI Auto play by jdog_5000 and some of LunarMongoose's bug fix
and (tried to) fixed some strange error that rarely happens at start up when I switch between vanilla DLL and our project DLL (hard to reproduce error so not sure of its cause or if it is fixed completely or not but I tried)

PS) all of 12Monkeys new game Utility calls should be activated by python mod before use (in 'Init' time for example)
This is change to avoid memory leak. ( Not extensively tested yet)
ex) PyCivSim.activateGamEntryPoint("canPlayerFoundCity")

 

[Impaler[WrG]]

On first inspection the Momory leak seems to be fixed on latest build, I will test it with my own compile and on longer games to make doubley shure.

Good job on the inclusion of jdogs AI auto-play, I know TGA is going to be using that as a base for some of his "Auto-AI-game" stuff. Of Mongooses stuff did you include just his StrikeChances fix or other mods? Most of his game play altering mods could be usefull as options but they will likley need work before they can be used.

 

[Dale]

BTW, you need to include the firstStrike fix of mongooses. I had it confirmed by Alex that it's an error, and it will be fixed in Warlords too.

Dale