incremental patch discussion

[Deliverator]

Mmmm. So that means there is something added between 1.7.3 and 1.7.3.2, probably between 1.7.3.1 and 1.7.3.2, that is causing a crash in Vista but not XP.

There are quite a few Python and SDK changes between 1.7.3.1 and 1.7.3.2. Adding them in one at a time could get time-consuming.

It would be good to narrow it down to a SDK or Python issue by doing the following:
1. Create a 1.7.3.2 install and rename the Dune Wars folder to Dune Wars-1.7.3.2.
2. Create a 1.7.3.1 install.
3. Delete or rename the assets/XML folder and the assets/art folder from the 1.7.3.1 install, and copy in the assets/XML folder and assets/art folder from the 1.7.3.2 install.

In other words, create an install the has the Python and SDK of 1.7.3.1 with the art and XML of 1.7.3.2. This could give some Python errors due to renaming of XML tags.

The point is to establish whether there is any issue with the XML and art changes that have been made.

 

[Deliverator]

OK, this sounds promising. It sounds like there might be an issue with compiling the SDK using Codeblocks (which is what I have been using) and Windows Vista. See this link.

I haven't yet posted the source changes for the last tweak I made to the code (removing the celebrity happiness stuff). If I do this perhaps david can compile a new DLL using the makefile and see if that DLL will work in Vista.

 

[Deliverator]

From Xienwolf in that linked thread:

Vista is more picky than XP is about some errors, like failure to initialize variables. So there is an error somewhere in your code, but XP is cool enough to let it fly. If you switch over to VS2008+ to compile a debug DLL you will probably find an Assert firing somewhere that is linked to your Vista issues. Otherwise it is nearly impossible to locate it without actually running on vista yourself.

Will using VS2005 or higher fix the Vista issue?

Not directly, but it can assist in finding minor "nuisances" (not enough of a bug to cause an XP crash, but enough of one to cause a Vista crash) without having to run Vista personally. Mostly this will happen with FAsserts, but sometimes the debug DLL will notice that something isn't quite right and it stops the program mid-flight to let you know.

Thinking about these points, I see two possible explanations for the Vista crashes from what we know.

1. One thing that changed between 1.7.3.1 and 1.7.3.2 was the introduction of the new tile variable that holds the amount of fresh water in a tile. Possibly, there is an initialization problem with that variable or some other C++ coding error that XP is OK with, but Vista cannot cope with.

AND/OR

2. Compiling using Codeblocks may result in a DLL that will work in XP, but not Vista. This seems less likely since the DLLs in 1.7.3 and 1.7.3.1 were both compiled by me in Codeblocks and have been reported working by Vista users.

 

[davidlallen]

OK, this sounds promising. It sounds like there might be an issue with compiling the SDK using Codeblocks (which is what I have been using) and Windows Vista.

The Visual Studio approach to compiling is also free, and described in a lot of detail in refar's guide. Was there a specific reason you picked codeblocks?

I can compile code to try stuff out in about 10 hours from now. But, I don't have vista, so I can upload a new dll which either JF or Ahriman (or others) will need to try.

 

[Ahriman]

For the record, I am using Vista, so this seems a promising line of inquiry.

 

[Deliverator]

The Visual Studio approach to compiling is also free, and described in a lot of detail in refar's guide. Was there a specific reason you picked codeblocks?

I read in Xienwolf's SDK modding guide that Codeblocks was easier for SDK newbies. Probably I should move to VS.

I can compile code to try stuff out in about 10 hours from now. But, I don't have vista, so I can upload a new dll which either JF or Ahriman (or others) will need to try.

I think this is a good plan. I'll see if I can spot any initialization problem in the code too. I can post the latest source within the next 4 hours.

 

[cephalo]

When I first got Vista, I had a ton of problems with modding civ because how it handles the 'Program Files' directory. It doesn't actually let you write to those files, it only tricks you into thinking that you can. For example, if you edit a Civ file with notepad, and then with something else, Vista will maintain 2 copies of the file for each application, which of course leads to disaster if you thought your notepad changes would change something.

The installer might be writing 'fake' files instead of real files if the user does not have full write permissions on the Program Files directory. I can't remember the location where the fake files are stored. All I know is that anything you inside the Program Files directory can become extremely complicated.

 

[Deliverator]

OK, I have a new DLL. This was compiled in Visual Studio 2008. Also, there was definitely some dodgy code around the fresh water parameter which I have fixed.

(It seems to be worth using VS anyway since the compiled DLL is about 1.5MB smaller than the Codeblocks one.)

To test:
1. Delete or rename your Dune Wars directory.
2. Run dune-wars-1-7.exe. (Say "OK" to Do you wish to uninstall?)
3. Run dune-wars-patch-1-7-3.exe.
4. Run dune-wars-patch-1-7-3-2-incremental.exe.
5. Overwrite the file Dune Wars\Assets\CvGameCoreDLL.dll with the one in this ZIP file.

I'm hopeful this one will work in Vista...

 

[Ahriman]

Great, I will test this tonight (4-5 hours from now).

 

[Ahriman]

To test:
1. Delete or rename your Dune Wars directory.
2. Run dune-wars-1-7.exe. (Say "OK" to Do you wish to uninstall?)
3. Run dune-wars-patch-1-7-3.exe.
4. Run dune-wars-patch-1-7-3-2-incremental.exe.
5. Overwrite the file Dune Wars\Assets\CvGameCoreDLL.dll with the one in this ZIP file.

Followed this. 1-7-3 works fine.
Install 1-7-3-2. Load the mod; get a long string of XML errors on load: tech water_discipline, memory_Manipulated_Us, Memory_Political_Marriage.
Mod loads.
Start a game: CTD during initialization.

 

[Jester Fool]

I followed the instructions in post #1168 and although I didn't get XML errors I still get CTD during initialization. So I tried compiling the DW SDK with Visual Studio 2008 and was successful, but then found out it was 1.7 code when I started the game Doh!

 

[Ahriman]

Any thoughts on why I could be repeatedly getting XML errors on some installs that Jester isn't?

Maybe my version of 1-7-3 that I keep using is a beta version, or corrupt?

 

[Deliverator]

I followed the instructions in post #1168 and although I didn't get XML errors I still get CTD during initialization. So I tried compiling the DW SDK with Visual Studio 2008 and was successful, but then found out it was 1.7 code when I started the game Doh!

Sorry. I should have posted the up-to-date source for 1.7.3.2 as well. Here it is.

If you can compile the latest source and still gets the CTD then there must still be a Vista-killing coding error in the C++ somewhere. I think that might still be the case. My changes in the code are commented Deliverator, but I haven't done more elaborate comments yet. Patch 1.7.3.1 contained all the source changes with it, so if you install that and rename it you can potentially do a diff to see what has changed between 1.7.3.1 and 1.7.3.2.

Any thoughts on why I could be repeatedly getting XML errors on some installs that Jester isn't?

Really don't know. It would suggest that you are ending up with an install that is missing some XML changes, but I'm not sure how this is happening. Possibly you do have the beta version of 1.7.3 - you can get the final version here.

If we can crack this bug then I suggest we put together a step release (1.8) to bring together all the patches that are floating around. It's a long time since 1.7 was released now.

 

[Deliverator]

I've found another possible error in the code. I think there needs to be a line added after line 18895 in CvInfos.cpp that reads:

if (isWormFriendly() == bDefault) m_bWormFriendly = pClassInfo->isWormFriendly();

 

[Jester Fool]

I compiled the up-to-date source code for 1.7.3.2 with "if (isWormFriendly() == bDefault) m_bWormFriendly = pClassInfo->isWormFriendly();" added after line 18895 in CvInfos.cpp. I got no errors or warnings. Inserted new dll but still get CTD during initialization.

Just to make sure, I uninstalled Dune Wars and reinstalled 1.7. Using the source code from the 1.7 installer, I re-compiled the DLL (did get 1 warning from an unreferenced local variable) and replaced it. The game starts normally.

This would seem to indicate Vista-killing coding error(s) in the C++ somewhere in the 1.7.3.2 source. I will now try compiling 1.7.3.1 and see what I can find out.

 

[Deliverator]

I compiled the up-to-date source code for 1.7.3.2 with "if (isWormFriendly() == bDefault) m_bWormFriendly = pClassInfo->isWormFriendly();" added after line 18895 in CvInfos.cpp. I got no errors or warnings. Inserted new dll but still get CTD during initialization.

Just to make sure, I uninstalled Dune Wars and reinstalled 1.7. Using the source code from the 1.7 installer, I re-compiled the DLL (did get 1 warning from an unreferenced local variable) and replaced it. The game starts normally.

This would seem to indicate Vista-killing coding error(s) in the C++ somewhere in the 1.7.3.2 source. I will now try compiling 1.7.3.1 and see what I can find out.

Good on you for investigating, Jester Fool.

From memory, there are basically only 3 new areas of functionality added to the SDK between 1.7.3.1 and 1.7.3.2.
1. The bWormFriendly tag for TerrainInfos and the new Sandrider art switching logic.
2. The new system for fresh water.
3. The GlobalBuildingClassCommerce tags that makes the Semuta Supplier and Semuta Den work.

Given that the crash seems to happen on startup I would think that an issue with 1 or 2 is more likely to be the problem, but I could be wrong. If you remove either of 1 or 2 from the code, you will need to alter DuneWars.py to avoid Python errors. For 1 you could just switch isWormFriendly to isWater, for 2 it will be a bit more complication to revert back. Both isWormFriendly and isFreshWater are called from Python when the Spice is being placed at game start so I can see their being an issue there. Perhaps try removing isWormFriendly first.

I know it is possible to compile a debug DLL which might help pinpoint the crash. I've never done it myself though and I can't recreate the crash on XP anyway. There are details at the bottom of this guide. It would be obviously much better if we can pinpoint the issue with the new features rather than just have to remove them because we can't get them to work in Vista.

 

[Jester Fool]

Ran 1.7.3.1 with a re-compiled dll and everything worked fine - no problem there.

Although 1.7.3.2 compiles with no errors or warnings, compiling a debug version ends in a fatal error.

Spoiler :

CvUnit.cpp
CvUnit.cpp(12395) : error C2296: '>=' : illegal, left operand has type 'int (__thiscall CvUnit::* )(void) const'
CvUnit.cpp(12395) : error C2297: '>=' : illegal, right operand has type 'int (__thiscall CvUnit::* )(void) const'
NMAKE : fatal error U1077: '"C:/Program Files (x86)/Microsoft Visual C++ Toolkit 2003/bin/cl.exe"' : return code '0x2'
Stop.
Project : error PRJ0019: A tool returned an error code from "Performing Makefile project actions"

Compiling 1.7.3.1 as a debug dll gets the same error as 1.7.3.2. However, the debug version of 1.7 works correctly. Does the 1.7.3 exe install the 1.7.3 source code? It might be helpful to see if a debug version can be made of the 1.7.3 source code.

 

[Deliverator]

I don't know C++ too well, but line 12395 in Unit.cpp looks like it could be wrong:

int CvUnit::getNoXPCount() const
{
	return m_iNoXPCount;
}

bool CvUnit::isNoXP() const
{
	return (getNoXPCount() > 0);
}

void CvUnit::changeNoXPCount(int iChange)
{
	m_iNoXPCount += iChange;
	[COLOR="Red"][B]FAssert(getNoXPCount >= 0);[/B][/COLOR]
}

Now, there is no variable called getNoXPCount so I think that line should be:

	FAssert(getNoXPCount[B][COLOR="Red"]()[/COLOR][/B] >= 0);

The brackets seem to be missing.

In fact, I'm sure that line is wrong if you compare that function with CvUnit::CvUnit::changeOffensiveVictoryMoveCount or CvUnit::changeOneUpCount.

Perhaps with the change will make a difference or at least allow the Debug DLL to compile.

Also, Xienwolf said in that thread that incorrect FAssets can cause issues for Vista. I'm not even sure what the FAssert line does to be honest.

 

[davidlallen]

When compiling in normal mode, the text inside "FAssert()" is thrown away without even being looked at. That is why the missing parentheses are not caught in normal compiles. In debug compiles, that text is compiled, and that is why the errors are flagged. In debug compiles, all of these "assert" statements test assumptions that the developers are relying upon. For example, if there are 10 techs, it is an error to ask something about the tech with index -1 or the tech with index 11. The program may work, or it may crash, when these illegal index values are passed in. Without asserts, the runtime is slightly faster but there is no helpful message when these illegal values are passed. With asserts, the runtime is slightly slower and a message is printed whenever the illegal values are passed in.

Add the missing parentheses, and the debug version will compile. There is a lot of philosophical debate among programmers about the proper use of asserts, but it doesn't matter for the problem we are trying to solve.

 

[Jester Fool]

Okay, Debug version working. These errors come up before the title screen (in order listed) -

Spoiler :

Assert Failed

File: CvGlobals.cpp
Line: 4025
Expression: iExisting==-1 || iExisting==idx || strcmp(szType, "ERROR")==0
Message: xml info type entry RELIGION_IMPERIAL already exists
-------------------------------
Assert Failed

File: CvGlobals.cpp
Line: 4025
Expression: iExisting==-1 || iExisting==idx || strcmp(szType, "ERROR")==0
Message: xml info type entry RELIGION_LANDSRAAD already exists
-------------------------
Assert Failed

File: CvGlobals.cpp
Line: 4025
Expression: iExisting==-1 || iExisting==idx || strcmp(szType, "ERROR")==0
Message: xml info type entry RELIGION_IMPERIAL already exists
---------------------------
Assert Failed

File: CvGlobals.cpp
Line: 4025
Expression: iExisting==-1 || iExisting==idx || strcmp(szType, "ERROR")==0
Message: xml info type entry RELIGION_TECHNOCRAT already exists
---------------------
Assert Failed

File: CvGlobals.cpp
Line: 4025
Expression: iExisting==-1 || iExisting==idx || strcmp(szType, "ERROR")==0
Message: xml info type entry RELIGION_QIZARATE already exists
---------------------
Assert Failed

File: CvGlobals.cpp
Line: 4025
Expression: iExisting==-1 || iExisting==idx || strcmp(szType, "ERROR")==0
Message: xml info type entry RELIGION_SHAIHULUD already exists
--------------------
Assert Failed

File: CvGlobals.cpp
Line: 4025
Expression: iExisting==-1 || iExisting==idx || strcmp(szType, "ERROR")==0
Message: xml info type entry RELIGION_IMPERIAL already exists
-------------------
Assert Failed

File: CvGlobals.cpp
Line: 4025
Expression: iExisting==-1 || iExisting==idx || strcmp(szType, "ERROR")==0
Message: xml info type entry RELIGION_TLEILAXU already exists
------------------
Assert Failed

File: CvGlobals.cpp
Line: 3654
Expression: strcmp(szType, "NONE")==0 || strcmp(szType, "")==0
Message: type FLAVOR_IMERIAL not found
-----------------
Assert Failed

File: CvGlobals.cpp
Line: 3654
Expression: strcmp(szType, "NONE")==0 || strcmp(szType, "")==0
Message: type FLAVOR_IMERIAL not found

I then launched a game and these errors came up before CTD -

Spoiler :

Assert Failed

File: CvGlobals.cpp
Line: 3993
Expression: strcmp(szType, "NONE")==0 || strcmp(szType, "")==0
Message: info type TERRAIN_TUNDRA not found, Current XML file is: xml\GameInfo/CIV4EspionageMissionInfo.xml
-------------------
Assert Failed

File: CvGlobals.cpp
Line: 3993
Expression: strcmp(szType, "NONE")==0 || strcmp(szType, "")==0
Message: info type FEATURE_ICE not found, Current XML file is: xml\GameInfo/CIV4EspionageMissionInfo.xml
--------------------
Assert Failed

File: CvGlobals.cpp
Line: 3993
Expression: strcmp(szType, "NONE")==0 || strcmp(szType, "")==0
Message: info type FEATURE_JUNGLE not found, Current XML file is: xml\GameInfo/CIV4EspionageMissionInfo.xml
------------------
Assert Failed

File: CvGlobals.cpp
Line: 2003
Expression: eImprovementNum > -1
Message:

Update *** I think I found the problem. Attached to the IDE this error comes up right before CTD -"Unhandled exception at 0x04217dfa (CvGameCoreDLL.dll) in Civ4BeyondSword.exe: 0xC0000005: Access violation reading location 0x88000180." The breakpoint is line 16183 in CvInfos.cpp

int CvImprovementInfo::getAddsFreshWaterInRadius() const
{
return m_iAddsFreshWaterInRadius; <-- this line
}