Better DuneBUG AI

Jester Fool · May 10, 2010

Deliverator said:
I believe so, but no one else has confirmed yet.

Confirmed. The first error shows before the 'run in fullscreen mode?' popup.

Code:

This application has failed to start because MSVCR71D.dll was not found. Re-installing the application may fix this problem.

I have confirmed that this is Dune specific. Vanilla Civ doesn't throw this error. It is non-fatal though.
Next, I get four instances consecutively of the following error:

Code:

There are more siblings than memory allocated for them in CvXMLLoadUtility::SetVariableListTagPair
Current XML file is: xml\Civilizations/Civ4LeaderHeadInfos.xml

After that, I get the following error followed by CTD {(Dune Wars) Init XML(uncached) is on the screen while this happens}:

Code:

Unhandled exception at 0x7729ac6c in Civ4BeyondSword.exe:0xC0000005:Access violation writing location 0x00007364

Fimbulvetr · May 10, 2010

You can simply download MSVCR71D.dll (just google it) and put it into c:\windows\system32\
(Size: 544 768 bytes, MD5: 40D72771DED1A9B92110A20E65CD15E9)

The other error looks really evil though. Is the modified CIV4CivilizationsSchema.xml file present in the xml\Civilizations folder too? It needs to contain these lines:

Code:

	[COLOR="Green"]<!-- BBAI Victory Strategy -->[/COLOR]
	<ElementType name="iCultureVictoryWeight" content="textOnly" dt:type="int"/>
	<ElementType name="iSpaceVictoryWeight" content="textOnly" dt:type="int"/>
	<ElementType name="iConquestVictoryWeight" content="textOnly" dt:type="int"/>
	<ElementType name="iDominationVictoryWeight" content="textOnly" dt:type="int"/>
	<ElementType name="iDiplomacyVictoryWeight" content="textOnly" dt:type="int"/>
	[COLOR="Green"]<!-- BBAI Victory Strategy -->[/COLOR]

(...)

		[COLOR="Green"]<!-- BBAI Victory Strategy -->[/COLOR]
		<element type="iCultureVictoryWeight"/>
		<element type="iSpaceVictoryWeight"/>
		<element type="iConquestVictoryWeight"/>
		<element type="iDominationVictoryWeight"/>
		<element type="iDiplomacyVictoryWeight"/>
		[COLOR="Green"]<!-- BBAI Victory Strategy -->[/COLOR]

If it's not there, it's these 5 siblings that are causing the crash.
(If that schema file is there yet it keeps crashing because of that then this very likely not introduced here but a fault in Better AI.)

Spoiler :

Jester Fool · May 10, 2010

Fuyu said:
Is the modified CIV4CivilizationsSchema.xml file present in the xml\Civilizations folder too?

Yes.

Fuyu said:
It needs to contain these lines:...

And it does.

Fuyu said:
Every leader in Civ4LeaderHeadInfos.xml needs the corresponding values...

All the BBAI stuff is there.

Deliverator · May 10, 2010

If you compile a Final_Release DLL do you get any further Jester Fool? I have been able to at least get a game underway. With this code I have been having issues in the Debug DLL that I don't seem to get with the Final_Release.

Fimbulvetr · May 10, 2010

For me, the real question now would be: can anyone get a game underway with the debug dll of Better BUG AI, and if not, at least with Better BTS AI?
My games with Better BUG AI seem stable but the debug dlls I created so far gave me a heap error while xml loading, failed to load python and crashed soon afterwards.

Spoiler :

VS 2003 debugger msgs

Code:

'Civ4BeyondSword.exe': Loaded 'T:\CIV4\Civilization 4\Beyond the Sword\Civ4BeyondSword.exe', No symbols loaded.
'Civ4BeyondSword.exe': Loaded 'C:\WINDOWS\system32\ntdll.dll', No symbols loaded.
'Civ4BeyondSword.exe': Loaded 'C:\WINDOWS\system32\kernel32.dll', No symbols loaded.
'Civ4BeyondSword.exe': Loaded 'T:\CIV4\Civilization 4\Beyond the Sword\boost_python-vc71-mt-1_32.dll', No symbols loaded.
'Civ4BeyondSword.exe': Loaded 'T:\CIV4\Civilization 4\Beyond the Sword\python24.dll', No symbols loaded.
'Civ4BeyondSword.exe': Loaded 'C:\WINDOWS\system32\user32.dll', No symbols loaded.
'Civ4BeyondSword.exe': Loaded 'C:\WINDOWS\system32\gdi32.dll', No symbols loaded.
'Civ4BeyondSword.exe': Loaded 'C:\WINDOWS\system32\advapi32.dll', No symbols loaded.
'Civ4BeyondSword.exe': Loaded 'C:\WINDOWS\system32\rpcrt4.dll', No symbols loaded.
'Civ4BeyondSword.exe': Loaded 'C:\WINDOWS\system32\secur32.dll', No symbols loaded.
'Civ4BeyondSword.exe': Loaded 'C:\WINDOWS\system32\shell32.dll', No symbols loaded.
'Civ4BeyondSword.exe': Loaded 'C:\WINDOWS\system32\msvcrt.dll', No symbols loaded.
'Civ4BeyondSword.exe': Loaded 'C:\WINDOWS\system32\shlwapi.dll', No symbols loaded.
'Civ4BeyondSword.exe': Loaded 'T:\CIV4\Civilization 4\Beyond the Sword\msvcr71.dll', No symbols loaded.
'Civ4BeyondSword.exe': Loaded 'T:\CIV4\Civilization 4\Beyond the Sword\msvcp71.dll', No symbols loaded.
'Civ4BeyondSword.exe': Loaded 'T:\CIV4\Civilization 4\Beyond the Sword\zlib1.dll', No symbols loaded.
'Civ4BeyondSword.exe': Loaded 'T:\CIV4\Civilization 4\Beyond the Sword\binkw32.dll', No symbols loaded.
'Civ4BeyondSword.exe': Loaded 'C:\WINDOWS\system32\winmm.dll', No symbols loaded.
'Civ4BeyondSword.exe': Loaded 'C:\WINDOWS\system32\ws2_32.dll', No symbols loaded.
'Civ4BeyondSword.exe': Loaded 'C:\WINDOWS\system32\ws2help.dll', No symbols loaded.
'Civ4BeyondSword.exe': Loaded 'C:\WINDOWS\system32\version.dll', No symbols loaded.
'Civ4BeyondSword.exe': Loaded 'C:\WINDOWS\system32\ole32.dll', No symbols loaded.
'Civ4BeyondSword.exe': Loaded 'C:\WINDOWS\system32\oleaut32.dll', No symbols loaded.
'Civ4BeyondSword.exe': Loaded 'C:\WINDOWS\system32\d3dx9_33.dll', No symbols loaded.
'Civ4BeyondSword.exe': Loaded 'T:\CIV4\Civilization 4\Beyond the Sword\Mss32.dll', No symbols loaded.
'Civ4BeyondSword.exe': Loaded 'T:\CIV4\Civilization 4\Beyond the Sword\hapdbg.dll', No symbols loaded.
'Civ4BeyondSword.exe': Loaded 'C:\WINDOWS\system32\dsound.dll', No symbols loaded.
'Civ4BeyondSword.exe': Loaded 'C:\WINDOWS\system32\imm32.dll', No symbols loaded.
'Civ4BeyondSword.exe': Loaded 'C:\WINDOWS\system32\lpk.dll', No symbols loaded.
'Civ4BeyondSword.exe': Loaded 'C:\WINDOWS\system32\usp10.dll', No symbols loaded.
'Civ4BeyondSword.exe': Loaded 'C:\WINDOWS\WinSxS\x86_Microsoft.Windows.Common-Controls_6595b64144ccf1df_6.0.2600.5512_x-ww_35d4ce83\comctl32.dll', No symbols loaded.
'Civ4BeyondSword.exe': Loaded 'C:\WINDOWS\system32\comctl32.dll', No symbols loaded.
'Civ4BeyondSword.exe': Loaded 'C:\WINDOWS\system32\uxtheme.dll', No symbols loaded.
'Civ4BeyondSword.exe': Loaded 'C:\WINDOWS\system32\msctf.dll', No symbols loaded.
'Civ4BeyondSword.exe': Loaded 'C:\WINDOWS\system32\clbcatq.dll', No symbols loaded.
'Civ4BeyondSword.exe': Loaded 'C:\WINDOWS\system32\comres.dll', No symbols loaded.
'Civ4BeyondSword.exe': Loaded 'C:\WINDOWS\system32\setupapi.dll', No symbols loaded.
'Civ4BeyondSword.exe': Loaded 'C:\WINDOWS\system32\linkinfo.dll', No symbols loaded.
'Civ4BeyondSword.exe': Loaded 'C:\WINDOWS\system32\ntshrui.dll', No symbols loaded.
'Civ4BeyondSword.exe': Loaded 'C:\WINDOWS\system32\atl.dll', No symbols loaded.
'Civ4BeyondSword.exe': Loaded 'C:\WINDOWS\system32\netapi32.dll', No symbols loaded.
'Civ4BeyondSword.exe': Loaded 'C:\WINDOWS\system32\userenv.dll', No symbols loaded.
DLL Load:Mods\Better BUG AI\Assets\CvGameCoreDLL.dll
'Civ4BeyondSword.exe': Loaded 'T:\CIV4\Civilization 4\Beyond the Sword\Mods\Better BUG AI\Assets\CvGameCoreDLL.dll', Symbols loaded.
'Civ4BeyondSword.exe': Loaded 'T:\CIV4\Civilization 4\Beyond the Sword\boost_python-vc71-mt-gd-1_32.dll', No symbols loaded.
'Civ4BeyondSword.exe': Loaded 'C:\WINDOWS\system32\msvcp71d.dll', Symbols loaded.
'Civ4BeyondSword.exe': Loaded 'C:\WINDOWS\system32\msvcr71d.dll', Symbols loaded.
'Civ4BeyondSword.exe': Loaded 'C:\WINDOWS\system32\msctfime.ime', No symbols loaded.
'Civ4BeyondSword.exe': Loaded 'C:\WINDOWS\system32\SSSensor.dll', No symbols loaded.
DLL_PROCESS_ATTACH
'Civ4BeyondSword.exe': Loaded 'C:\WINDOWS\system32\msxml3.dll', No symbols loaded.
LoadCivXml (xml\GameInfo/CIV4PlayerOptionInfos.xml)
LoadCivXml (xml\GameInfo/CIV4GraphicOptionInfos.xml)
[B]HEAP[Civ4BeyondSword.exe]: Invalid Address specified to RtlFreeHeap( 01860000, 01DD6AB0 )[/B]
Unhandled exception at 0x7c91120e in Civ4BeyondSword.exe: User breakpoint.
[COLOR="Green"]etc[/COLOR]

Deliverator · May 10, 2010

Fuyu said:
For me, the real question now would be: can anyone get a game underway with the debug dll of Better BUG AI, and if not, at least with Better BTS AI?
My games with Better BUG AI seem stable but the debug dlls I created so far gave me a heap error while xml loading, failed to load python and crashed soon afterwards.

I can get a game underway with a debug dll of Better BUG AI 1.00. Here's a link to my compiled Debug DLL.

Jester Fool · May 10, 2010

Deliverator said:
If you compile a Final_Release DLL do you get any further Jester Fool? I have been able to at least get a game underway. With this code I have been having issues in the Debug DLL that I don't seem to get with the Final_Release.

The Final_Release DLL will start, although I haven't actually tried to start a game. CvXMLLoadUtilitySet.cpp has the following :

Code:

 if(!(iNumSibs <= iInfoBaseLength))
				{
					char	szMessage[1024];
					sprintf( szMessage, "There are more siblings than memory allocated for them in CvXMLLoadUtility::SetVariableListTagPair \n Current XML file is: %s", GC.getCurrentXMLFile().GetCString());
					gDLL->MessageBox(szMessage, "XML Error");
				}

If I understand this correctly, the XML doesn't match the memory created to store the information from it.

Fimbulvetr · May 10, 2010

So much for that idea. But at least now I can assume it's not my fault

And thank you for for the compile (for actual debugging purposes the much bigger incremental linker file (.ilk) is needed too though, without it there will be "no symbols loaded" for the dll).
How did you do it, just nmake /f Makefile Final_Release (using my Makefile) in a VC++ express edition with 2003 toolkit as described in the tutorials?
It really bugs me that the obvious solution, which is simply clicking "Build..." in Visual Studio 2003, doesn't work for me..

Deliverator · May 10, 2010

I use Visual C++ 2008 Express Edition. I followed Refar's guide to set everything up.

If the issue is to do with XML loading of LeaderheadInfos then perhaps it is just a question of looking at the Dune Wars changes to that file. I did have a issue at one point when doing the merge with the additional MemoryInfos that David added.

davidlallen · May 10, 2010

Jester Fool said:
If I understand this correctly, the XML doesn't match the memory created to store the information from it.

This usually happens due to a syntax error in the schema file.

Deliverator · May 11, 2010

The changes to the Civilizations Schema between Better BUG AI and DW 1.8X are as shown in this report:

Spoiler :

Code:

Text Compare
Produced: 11/05/2010 08:09:19

Mode:  Differences, Ignoring Unimportant
Left file: E:\Program Files\Firaxis Games\Sid Meier's Civilization 4\Beyond the Sword\Mods\Better BUG AI\Assets\XML\Civilizations\CIV4CivilizationsSchema.xml
Right file: E:\Program Files\Firaxis Games\Sid Meier's Civilization 4\Beyond the Sword\Mods\Dune Wars_X_Debug\Assets\xml\Civilizations\CIV4CivilizationsSchema.xml
                                      -+ 70  	<ElementType name="iHatedCivicAttitudeChange" content="textOnly" dt:type="int"/>
------------------------------------------------------------------------
------------------------------------------------------------------------
                                      -+ 91  	<ElementType name="HatedCivic" content="textOnly"/>
------------------------------------------------------------------------
------------------------------------------------------------------------
                                      -+ 258 		<element type="iHatedCivicAttitudeChange"/>
------------------------------------------------------------------------
------------------------------------------------------------------------
                                      -+ 279 		<element type="HatedCivic"/>
------------------------------------------------------------------------
------------------------------------------------------------------------
                                      -+ 431 	<ElementType name="ReligionName" content="textOnly"/>
                                         432 	<ElementType name="bForbidden" content="textOnly" dt:type="int"/>
                                         433 	<ElementType name="ForbiddenReligion" content="eltOnly">
                                         434 		<element type="ReligionName" minOccurs="1"/>
                                         435 		<element type="bForbidden" minOccurs="1"/>
                                         436 	</ElementType>
                                         437 	<ElementType name="ForbiddenReligions" content="eltOnly">
                                         438 		<element type="ForbiddenReligion" maxOccurs="*"/>
                                         439 	</ElementType>
------------------------------------------------------------------------
------------------------------------------------------------------------
                                      -+ 466 		<element type="Help" minOccurs="0"/>
------------------------------------------------------------------------
------------------------------------------------------------------------
                                      <> 483 		<element type="ForbiddenReligions" minOccurs="0"/>
466 		<element type="InitialCivics"/>    484 		<element type="InitialCivics" minOccurs="0"/>
------------------------------------------------------------------------

Nothing looks obviously wrong. I think the game wouldn't start if there was a syntax error? What doesn't make sense is that I can start a game fine and play many turns with my 1.8X Debug DLL or Final Release DLL whereas for other people it won't even reach the title screen.

Fimbulvetr · May 11, 2010

I don't see any problems in that schema file either, well-formed and all elements declared; but he did say usually - In this case it's the xml file itself.
What I found in lines 12563+ while skimming through it seems to confirm that.

Code:

			<Flavors>
				<[COLOR="Blue"]Flavor>[/COLOR]
					<FlavorType>FLAVOR_GOLD</FlavorType>
					<iFlavor>5</iFlavor>
				[COLOR="Blue"]</Flavor>[/COLOR]
				[COLOR="Blue"]<Flavor>[/COLOR]
					<FlavorType>FLAVOR_MILITARY</FlavorType>
					<iFlavor>2</iFlavor>
					[B][COLOR="Red"]<Flavor>[/COLOR]
						<FlavorType>FLAVOR_CHOAM</FlavorType>
						<iFlavor>6</iFlavor>
					[COLOR="Red"]</Flavor>[/COLOR][/B]					
				[COLOR="Blue"]</Flavor>[/COLOR]
			</Flavors>

which should obviously look like this:

Code:

			<Flavors>
				[COLOR="Blue"]<Flavor>[/COLOR]
					<FlavorType>FLAVOR_GOLD</FlavorType>
					<iFlavor>5</iFlavor>
				[COLOR="Blue"]</Flavor>[/COLOR]
				[COLOR="Blue"]<Flavor>[/COLOR]
					<FlavorType>FLAVOR_MILITARY</FlavorType>
					<iFlavor>2</iFlavor>
				[COLOR="Blue"]</Flavor>[/COLOR]
				[B][COLOR="Blue"]<Flavor>[/COLOR]
					<FlavorType>FLAVOR_CHOAM</FlavorType>
					<iFlavor>6</iFlavor>
				[COLOR="Blue"]</Flavor>[/COLOR][/B]
			</Flavors>

Jester Fool · May 11, 2010

Fuyu said:
I don't see any problems in that schema file either, well-formed and all elements declared; but he did say usually - In this case it's the xml file itself.
What I found in lines 12563+ while skimming through it seems to confirm that.

That was definitely one error. I still get a similar error but at a diffent memory location in the same xml file. Now I know what to look for. I really am a dummy when it comes to xml file structure. :lol:

Deliverator · May 11, 2010

Fuyu said:
I don't see any problems in that schema file either, well-formed and all elements declared; but he did say usually - In this case it's the xml file itself.
What I found in lines 12563+ while skimming through it seems to confirm that.

I found quite a bit of malformed XML in the original Dune Wars 1.8 XML when doing the merge. I cleaned up some of it, but clearly there is some left to weed out. I think the XMLCopy code or something in the DW1.8 RevDCM code base must make it much more forgiving of malformed XML otherwise I don't know how 1.8 works considering the number of errors in it.

Surely there are tools for validating XML against a given schema?

Edit: This tool sounds like it does what we want - I'm going to try it when I get home.

davidlallen · May 11, 2010

This is an unexpected discovery. I know that RevDCM includes "WOC Lite" which included an ambitious attempt to improve "modular loading" of xml. Perhaps it introduced a number of ambiguities into the xml parser, which are not present in the vanilla parser. I would have expected some kind of syntax errors, even misleading ones, but it seems you are finding "silent omissions".

I'm not aware of any civ modders needing to use generic xml syntax checking tools before, so I will be very interested to know how it goes. If it finds useful stuff without too much "noise" it may be a useful third tool for xml checking -- after civchecker and the vanilla xml loader itself.

Deliverator · May 11, 2010

The problem seems to be that the schema is in XDR or XML Data Reduced format which is pretty much universally deprecated in favour of XSD and others. I'm finding it hard to find anything that will do the validation.

keldath · May 11, 2010

hey guys,

found a python error:

davidlallen · May 11, 2010

Deliverator said:
The problem seems to be that the schema is in XDR or XML Data Reduced format which is pretty much universally deprecated in favour of XSD and others. I'm finding it hard to find anything that will do the validation.

For finding these incorrect xml syntax issues, does the file My Games/Beyond The Sword/Logs/xml.log help at all? These issues may be logged there but if they are not fatal, they may not prevent the game from starting. These messages require a little understanding. (a) There are always tons of "info type NONE not found" which you should ignore, (b) Many of the messages are generated after the last file is read, so the "Current XML file is..." information should be taken with a big grain of salt. If there is an undefined symbol listed in the logfile, grep in your xml files for it and don't be surprised if it is in a different file than the one listed in the message.

Deliverator · May 11, 2010

Hi Keldath - yeah I noticed that one. It seems to happen unpredictably when in the pedia. It doesn't seem like really a error in the Python, but an error in the C++ being reported in Python. No idea why that is happening, but it's not as bad as the game crashing since it usually just means a pedia page fails to load but then goes away if you try again.

Anyway, I've found a way to validate the XML.

1. In Visual Studio open the schema file e.g. CIV4CivilizationSchema.xml.
2. In the Menu choose XML - Create Schema. This will convert the XDR schema into a more up-to-date and widely supported XSD schema.
3. You then have a number of options for validating. I used this handy online validator:
http://xmltools.corefiling.com/schemaValidate/

This gave me the following output for CIV4LeaderheadInfos.xml (The headers are bit screwed):

Code:

The following errors were found:
TYPE	LOC	MESSAGE
Validation	12566, 14	cvc-complex-type.2.4.a: Invalid content was found starting with element 'Flavor'. One of '{"x-schema:CIV4CivilizationsSchema.xml":FlavorType, WC[##other:"x-schema:CIV4CivilizationsSchema.xml"], WC[""]}' is expected.

The problem/line number highlighted is the issue with the Flavor tags already noticed by Fuyu. After that is fixed the file passes as valid, so if Jester Fool is still getting a crash on the same file I'm stumped.

Deliverator · May 11, 2010

davidlallen said:
For finding these incorrect xml syntax issues, does the file My Games/Beyond The Sword/Logs/xml.log help at all?

My xml.log seems completely error free. But I can't recreate Jester Fool's problem on my PC anyway. With my Debug DLL version of 1.8X I load the mod and start a game fine on one PC, but on another one that doesn't have Visual C++ installed it crashes on startup.

Better DuneBUG AI

Emperor

Emperor

Emperor

Graphical Hackificator

Emperor

Graphical Hackificator

Emperor

Emperor

Graphical Hackificator

Deity

Graphical Hackificator

Emperor

Emperor

Graphical Hackificator

Deity

Graphical Hackificator

LivE LonG AnD PrOsPeR

Attachments

Deity

Graphical Hackificator

Graphical Hackificator

Similar threads