Debugging Help (mod mergers)

[phungus420]

I just received this report today:

http://forums.civfanatics.com/showpost.php?p=8750379&postcount=532

It's the first critical bug report for Legends of Revolution in about 2 months. The user plays the GotM frequently and as such uses BUFFY without error. So I can't figure out what's going wrong, as it can't be the BUG mod (though I suspect it's something related to BUG). Also this has to be some user specific issue, but again I'm clueless as to what it could be. They enabled python exceptions, and error logging and have uploaded the python logging files (which I am uploading here). I took a look at them, but can't see anything that can help me debug the issue. Any help would be greatly appreciated (before this report there were no critical bugs known to exist, and I'd like to get things that way again).

 

[Mesix]

Hey...glad that I could find a new bug.

Here is some of my system info incase it helps:

T7700 2.4 GHz
2 GB RAM
8600M GT 256 MB
Running Vista 32 bit

Here are the other mods that are installed (all working):

BUFFY (HOF Mod)
FfH2
TAM
History of the Three Kingdoms

 

[EmperorFool]

The error is right at the very top of PythonErr.log:

  File "CvMainInterface", line 21, in ?
  File "<string>", line 52, in load_module
  File "CvConfigParser", line 26, in ?
  File "<string>", line 52, in load_module
  File "CvPath", line 135, in ?
  File "CvPath", line 112, in _getUserDir
  File "CvPath", line 104, in __getRegValue
[B]WindowsError: [Errno 2] The system cannot find the file specified[/B]

This says to me that you have modified CvMainInterface to import CvConfigParser which imports CvPath. CvPath has had issues for a long while, and that's why I wrote BugPath. I suspect CvPath is failing to find the user's My Documents folder, but I cannot be sure as I don't have your version of CvPath (there are many).

You're free to try to fix CvPath, but I highly recommend you either port over CvConfigParser's use of CvPath to BugPath (it probably just needs a path to the INI files) or switch whatever is using CvConfigParser to use BUG's options framework.

 

[phungus420]

Well I tried to compare with BUG, and then BtS, and then the default Civ4 vanilla files, but could not find them. These are RevDCM's CvConfigParse, and CvPath. Also CvMainInterface is in there too, but it's pretty similar to BUG's, the only difference I could see was the position of scores mod, and some added buttons dealing with Revolutions.

Any particular direction I can go to fix this for users that experience this problem? Do you have any idea what could be happening on certain systems that can destroy the interface? It's weird because, as I said this is the first critical bug report in almost a couple of months for LoR specifically, but also I haven't seen anything similar for the other RevDCM mergers (though it's hard to tell if a bug report for a large RevDCM merger is due to the merge or the RevDCM core).

 

[EmperorFool]

Well as the stack trace shows, CvMainInterface imports CvConfigParser (BUG's does not) which imports CvPath. BUG uses a slightly different config parser IIRC, but it doesn't reference CvPath or BugPath.

To fix this I would find all references to CvPath and change them to use BugPath. BugPath should have an equivalent function to do whatever you need done by CvPath. As I said, this is typically "find me an INI file". BUG has plenty of examples of this in action if you want to do it all manually. Far better would be to BUGify whatever is using that so that the options are in a configuration XML file and accessible via BugCore.game.

 

[phungus420]

Ah, I bet it's the Revolution.ini file

I'm apprehensious about messing with this, and should probably wait on glider/jdog then. I'm completely unfamiliar with the code here. I'll take a look though, thanks for pointing out the direction and explaining the problem.

 

[phungus420]

Well it didn't like me trying to just replace all instances of CvPath with BugPath in CvConfigParser. I get those python failure message, directing me to the fact assetsPath doesn't exist in BugPath.py. I tried looking at things, and can't make sense of it. In CvPath looking up assetsPath yields this:

userAssetsDir = os.path.join(userDir, "CustomAssets")
...
installAssetsDir = os.path.join(installDir, "Assets")
...

assetsPath = [userAssetsDir, installAssetsDir]

if (activeModName != None):
	userActiveModDir = os.path.join(userModsDir, activeModName)
	userActiveModAssetsDir = os.path.join(userActiveModDir, "Assets")
	installActiveModDir = os.path.join(installModsDir, activeModName)
	installActiveModAssetsDir = os.path.join(installActiveModDir, "Assets")
	assetsPath.insert(0, userActiveModAssetsDir)
# YAUGM changes begin
#   assetsPath.insert(1, installActiveModAssetsDir)
	assetsPath.insert(2, installActiveModAssetsDir) # now goes after CustomAssets!
# YAUGM changes end

pythonPath = []
for dir in [os.path.join(d, "Python") for d in assetsPath]:
	for root, subdirs, files in os.walk(dir):
		if (len(files) > 0):
			pythonPath.append(root)
...
def getPath(relPath, fileName):
	for filePath in [os.path.join(d, relPath) for d in assetsPath]:
		if os.path.isfile(os.path.join(filePath, fileName)):
			return filePath
	return ""
...
def getINIPathForCvConfigParser(fileName):
	if (fileName != None):
		filenames = [os.path.join(os.path.dirname(dir), fileName)
					 for dir in assetsPath]
		filenames.reverse()
		return filenames
	return None
...
def get_INI_File(szINIFileName):
	filepaths = [os.path.join(os.path.dirname(dir), szINIFileName) 
		for dir in assetsPath]
	for filepath in filepaths:
		if os.path.isfile(filepath):
			return filepath
	return ""
...
def _test():
	print "activeModName = " + str(activeModName)
	print "userDir = " + userDir
	print "userAssetsDir = " + userAssetsDir
	print "userModsDir = " + userModsDir
	print "userActiveModDir = " + str(userActiveModDir)
	print "userActiveModAssetsDir = " + str(userActiveModAssetsDir)
	print "installDir = " + installDir
	print "installAssetsDir = " + installAssetsDir
	print "installModsDir = " + installModsDir
	print "installActiveModDir = " + str(installActiveModDir)
	print "installActiveModAssetsDir = " + str(installActiveModAssetsDir)
	print "assetsPath = " 
	for dir in assetsPath:
		print "  " + dir
	print "pythonPath = "
	for dir in pythonPath:
		print "  " + dir

None of these things look anything like stuff in BugPath.py, at least I search finds nothing, and I can't see anything myself. I can't makes sense of what I should be porting over to BugPath, what can be ignored, and what it is I'm doing actually.

 

[EmperorFool]

RevDCM includes BUG, so I see no reason not to port over the INI stuff to use BUG's Options Core. Post CvMainInterface from RevDCM and any other module that references CvConfigParser, and I'll take a look.

 

[phungus420]

Here you go, and thanks for taking a look. By module I'm assuming you mean any python file that references it.

 

[EmperorFool]

First some error message output needs to change in RevolutionInit.py:

import [B][COLOR="Orange"]BugPath[/COLOR][/B]
...
			bodStr += "  Check mod installation directory, should be:\n\n" + [B][COLOR="Orange"]BugPath.getModDir()[/COLOR][/B]
			[B][COLOR="Red"][s]bodStr += "\n\nOr:\n\n" + CvPath.userActiveModDir[/s][/COLOR][/B]

Next you need to modify CvMainInterface.py. This is trickier, but there aren't that many places to touch. If you can use those docs (BUG Core Options) that I linked, I bet a smart guy like you could get it working in half an hour. Remove CvConfigParser and use BugCore.game. As I said, tons of examples in BUG plus docs.

revConfig isn't even used--only the first INI file is checked though two are loaded. You'll replace lines like this

iMaxScoreLines = config.getint("Score Board", "MaxScoreLines", -1)

with some XML (bold line once per option)

<!-- Revolution.xml -->

<mod id="Revolution">
    <options file="Revolution.ini">
        [B]<option id="MaxScoreLines" type="init" default="-1"/>[/B]
    </options>
</mod>

and some Python

# once only at top of file
import BugCore
RevOpt = BugCore.game.Revolution

...

# any place you see config.getXXX()
iMaxScoreLines = RevOpt.getMaxScoreLines()

 

[phungus420]

I'm a little confused. What exactly should I be doing here? Am I moving the stuff that's stored in Revolution.ini into a config XML file (that seems like a lot of work)? Could you point me to an example of what I should be doing, if I see something done I can usually pick it up. But right now I'm a bit confused as to what conceptually I'm trying to replace, and why.

 

[EmperorFool]

You're replacing how Revolution.ini is accessed. You'll move some of what's in Python (how to find the file and for each option its type, key in the INI file, and default value) to an XML configuration file.

Look at Config/AutoSave.xml:

<mod id="AutoSave">
	<options id="AutoSave" file="AutoSave.ini">
		<section id="AutoSave">
			<option id="CreateStartSave" key="Create Start Save" 
					type="boolean" default="False"/>

			<option id="CreateEndSave" key="Create End Save" 
					type="boolean" default="False"/>

			<option id="CreateExitSave" key="Create Exit Save" 
					type="boolean" default="False"/>

			<option id="UsePlayerName" key="Use Player Name" 
					type="boolean" default="False"/>
		</section>
	</options>
</mod>

The <mod> element's "id" attribute should be Revolution. The <options> tag tells you the name of the INI file; its "id" attribute should match the one in <mod> for your case. And the <section> says which section in the INI holds the options ("Scoreboard" I think--check the iNI file and CvMainInterface). You can have multiple sections if necessary.

Within that you need one <option> for each option--every line that says "config.getXXX()" in CvMainInterface (there are about four IIRC). Each line has an "id" attribute that you'll use to access the value in Python, a "key" that tells it the name in the INI file (use what's in CvMainInterface), a type (boolean, int, float, string), and a default value.

In CvMainInterface you'll replace all those config.getXXX() lines with what I posted above. And you'll ditch CvConfigParser and CvPath. Finally, put

<load mod="Revolution"/>

near the end of Config/init.xml to tell BUG to load your new configuration file.

Or you can figure out how to fix CvPath and get it to play nice with Vista. And then Windows7. And then . . .

 

[phungus420]

Tried the above. I have no idea if it's working or not, as Python fails to load. What do I do about RevolutionInit.py

It as well is full of config references, like this:

class RevolutionInit :

	def __init__( self, customEM, configFileName = None ) :
		
		CvUtil.pyPrint("Initializing Revolution Mod")

		self.EventKeyDown = 6
		self.customEM = customEM
		self.configFileName = configFileName

		config = CvConfigParser.CvConfigParser(configFileName)
		self.config = config

		self.bFoundConfig = config.getboolean("RevConfig", "FoundConfig", False)
		self.bShowActivePopup = config.getboolean("RevConfig", "ActivePopup", True)

 

[EmperorFool]

Weird, I thought I had searched that file for CvConfigParser and got no hits, but now I think I actually searched it for CvPath. Yes, you need to make the same changes as you did to CvMainInterface.py. Hopefully there aren't 50 more options in that module.

 

[phungus420]

There are unfortunately. But OK, I think I got it. Will ask when I hit the next problem, but for now this is just alot of cutting and pasting.

 

[Mesix]

Thanks for working diligently to fix a problem that only seems to be affecting me. Your programming skills are far beyond mine. I will be happy to test your "patch" when you are finished with it.

Merry Christmas!

 

[phungus420]

Well I have succeeded in completely destroying Revolutions.

I think I am missing a few things here. For one creating XML entries for all the stuff in the Revolution config file doesn't seem right, as the Revolution config file has varying sections that use the same names, and I don't see where in the python I'm differentiating between the MaxCiv value for Barbarian Civs, and the MaxCiv value for Revolutions. The only python exception I'm getting is "Barbarian Civ has no attribute 'OfferControl', line number 147 in the modified code (can't make sense of it, it looks wrong, but I don't know what it should be). Other then that it's simply behaving like revolutions (and I suppose every other component in Revolution.ini) is turned off.

Here are the changed files (originals in the attachment above)

 

[phungus420]

Actually fixed that exception. Now it there are no exceptions. It just says "Waring Revolution.ini not found" when you load the game in one of RevDCM's custom pop ups. Then it loads all the revolution modules under "default" setup as though the config settings aren't there. Attached is the modified files, or other files of interest (original RevDCM files in the python attachement above).

 

[EmperorFool]

I'll be home Friday night and take a look at those files. For now I can give you a few pointers.

You should have in your new configuration XML file one

<section id="section-name">

element for each

[section-name]

in the INI file. This is how you can have two properties both named MaxCiv.

You should have one

<option>

element for each property in the INI file.

The <option> in the XML configuration file tells BUG, "hey, there's this option value that I want to store in an INI file. It is in section S with name N, and it has type T with default value D."

Each option for a single mod must have a unique ID attribute. You can have duplicate KEY attributes as long as they are in different sections (rules of INI). If you don't specify a KEY attribute for an <option>, BUG uses the ID attribute for the KEY. This means where you have duplicate KEYs you need to choose and specify a different ID.

<section id="X">
    <option id="MaxCiv" type="int" default="12"/>
</section>
<section id="Y">
    <option id="[B][COLOR="Orange"]OtherMaxCiv[/COLOR][/B]" key="MaxCiv" type="int" default="21"/>
</section>

 

[phungus420]

OK. That's helpful. Is there some reason why it says it can't find the Revolution.ini file off the top of your head? Is there something I need to add to the CustomRevAdv.txt file in UserSettings and move the ini over to the UserSettings folder?