BUG and Mac

alerum68 · Aug 8, 2009

With the release of BTS for Macintosh, I was wondering if anyone has tried BUG with the Mac. I know very little about Mac, so need to ask some questions... can you use the installer, or will released need to be packaged in an archive format such as ZIP. I believe EF made changes to make it work with Mac before BTS was released for Mac, but I haven't heard anything from Mac Users if it works...

I just noticed that FFH has their own Icon to add to the message titles... what are we gonna have to do to get that?

Flintlock1415 · Aug 11, 2009

Basically the two things that stop mods from running on a mac are exe installers and custom dll files. There are reports that Aspyr is working on getting those to work, but that may be a ways off. I don't know if BUG uses a custom dll, but I'd suspect it does.

EDIT: actually, it seems that BUG doesn't have a dll, am I correct? I'll probably check it out later and post back here. I also want to make sure that this works on 3.17, which it says it does. The Mac developers are working on the 3.19 patch, but for now we have only 3.17.

Flintlock1415 · Aug 11, 2009

Okay, I can't get it to work. I'm trying to run BUG 3.5.1, following the install instructions I found on the Sourceforge project area.

I don't know which logs exactly that you'd like, but there is something in my PythonErr.log.

Spoiler :

I don't know what 'set' does, but maybe there is a compatibility issue with Python 2.3 (Mac) and 2.4 (Windows)

EmperorFool · Aug 12, 2009

Ah okay, I think I know what to do here. Basically they used two different Python versions as you said (2.3 versus 2.4). Python 2.3 has a Sets module that got pulled into the language in 2.4 as the set built-in class. They are apparently API-compatible so I just need to do some Python trickery to get it to use one or the other as appropriate.

If you are feeling adventurous, would you mind giving this little test a shot. Open the file Assets/Python/Screens/CvVictoryScreen.py in a text editor and insert this at the top:

Code:

try:
    set()
except NameError:
    import sets
    set = sets.Set
    frozenset = sets.Set

There are no doubt other modules that use set(), and this will need to be added to each. However, you can just do this first one and then check PythonErr.log after running it again. If you get the same error in a different module, it looks like this solution will work and I can apply it to all modules using set().

Flintlock1415 · Aug 12, 2009

I got the error in Bugeventmanager.py as well, and dropped the code in there as well. I'm still getting an error, but I can't understand it.

Here are my logs:

PythonErr2.log

Spoiler :

PythonDbg.log

Spoiler :

EmperorFool · Aug 12, 2009

I don't see any errors there. You didn't post PythonErr.log this time--is it empty or not exist? If so, there were no errors. Do you have a functional interface?

AlanH · Aug 12, 2009

I've been trying this today. I got to the stage where the game would load up as far as the main menu. This is BUG, by the way. BUFFY stands no chance on a Mac, as it makes intensive use of a DLL.

@Flintlock: Once you have Python showing that it has reached load_module xmllib you need to look for errors in the xml.log file. I "zapped gremlins" in a couple of XML files (a BBEdit technique for getting rid of badly encoded characters in text files), but eventually had to remove xml/text/Strategy_CIV4GameText.xml, as I couldn't work out why it was causing a crash.

I also had to fix an instance of _winreg that appears in Smart_Map.py, as I had installed the PublicMaps folder.

I have now reached the point where, if I start a game, I get to the good old No Interface state, because Python throws runtime errors. The one I've stalled on now is:

Code:

Traceback (most recent call last):

  File "CvAppInterface", line 88, in preGameStart

  File "CvScreensInterface", line 73, in showMainInterface

  File "CvMainInterface", line 2414, in interfaceScreen

  File "CvMainInterface", line 740, in getMaxRow

ZeroDivisionError: integer division or modulo by zero
ERR: Python function preGameStart failed, module CvAppInterface

Since the relevant line is dividing by PleOpt.getVerticalSpacing(), I'm guessing this is because there's no Options file for some reason, and I'm following that trail of breadcrumbs.

EmperorFool · Aug 12, 2009

This last error occurs because the BugInit process has crapped out at some point, so the options aren't loaded, leaving that value as 0. I should fix CvMainInterface.py to at least use a reasonable default in that case, but that's a different matter.

Can you change the File Level in "BUG Core.ini" to 0 (DEBUG) and post PythonDbg.log?

AlanH · Aug 12, 2009

Further update.

The problem with access to options related to the way the search paths are evaluated in the BugPath,py script. It uses sys.executable to get the Civ4 application path. On OS X, that returns the path to the Python interpreter - usr/bin/. I've hard wired the appName as "Civilization IV Beyond the Sword", and used os.cwd() to get rootDir. It now finds init.xml correctly in CustomAssets. I haven't tried it installed in /Mods/ yet.

We then trip over the set() problem again. BugConfig fails to parse the first type="set" argument in the init.xml file:

In PythonDebug.log:

Code:

23:55:27 BugConfig - failure parsing /Users/alan/Documents/Civilization IV Beyond the Sword/CustomAssets/Config/init.xml at line 25
23:55:27   invalid syntax (line 1)

... and in PythonErr.log:

Code:

Traceback (most recent call last):
  File "BugConfig", line 364, in parse
  File "/System/Library/Frameworks/Python.framework/Versions/2.3/lib/python2.3/xmllib.py", line 171, in feed
    self.goahead(0)
  File "/System/Library/Frameworks/Python.framework/Versions/2.3/lib/python2.3/xmllib.py", line 287, in goahead
    k = self.parse_endtag(i)
  File "/System/Library/Frameworks/Python.framework/Versions/2.3/lib/python2.3/xmllib.py", line 697, in parse_endtag
    self.finish_endtag(tag)
  File "/System/Library/Frameworks/Python.framework/Versions/2.3/lib/python2.3/xmllib.py", line 730, in finish_endtag
    self.handle_endtag(nstag, method)
  File "/System/Library/Frameworks/Python.framework/Versions/2.3/lib/python2.3/xmllib.py", line 751, in handle_endtag
    method()
  File "BugConfig", line 582, in end_arg
  File "BugConfig", line 248, in createArgument
  File "BugConfig", line 74, in <lambda>
  File "<string>", line 1
 
    set([ 			"gameUpdate", 			"mouseEvent", 			"kbdEvent" 		])
 
         ^
 
SyntaxError: invalid syntax

What fun!

The relevant BugConfig.py code is at line 68. I'm some way past my comfort zone in Python at this point:

Code:

TYPE_EVAL = { "boolean": lambda x: x.lower() in TRUE_STRINGS,
			  "string": lambda x: x,
			  "int": lambda x: int(x),
			  "float": lambda x: float(x),
			  "tuple": lambda x: eval("(%s,)" % x),
			  "list": lambda x: eval("[%s]" % x),
			 [b] "set": lambda x: eval("set([%s])" % x),[/b]
			  "dict": lambda x: eval("{%s}" % x),
			  "key": lambda x: InputUtil.stringToKeystrokes(x) }

Needless t say, I have put the set = sets.Set fix in this file. I'm guessing this substitution is not being applied during the eval() function?

AlanH · Aug 12, 2009

Hmm. I've tried substituting 'sets.Set' for 'set' in the above eval() command, and it still fails. It looks as if it doesn't like the tabs in between the list items. I've edited them out of the init.xml file in the module="BugEventManager" element, and it then fails on the tuple type at line 47.

Removing all returns and tabs in those lists gets them past the BugConfig.py parser. Now on to more modules that need the set fix - Civ4lerts.py, MoreCv4lerts.py - and we have an interface! UnitUtil.py throws the set error ...

BUG Options opens using Command+Option+O. Help doesn't open - Python error indicates that os.startfile() is probably not available in v2.3

[UPDATE] It's now loading error-free and I can open all the adviser screens and Options screens with no errors thrown.

Only the above Help file issue is visible currently - not sure what that's about. os.startfile() is supposed to be available from 2.0, but maybe it's disabled in OS X for platform-specific reasons. Error message is:

Code:

Traceback (most recent call last):

  File "CvOptionsScreenCallbackInterface", line 521, in handleBugHelpButtonInput

  File "BugHelp", line 24, in launch

AttributeError: 'module' object has no attribute 'startfile'
ERR: Python function handleBugHelpButtonInput failed, module CvOptionsScreenCallbackInterface

@EF: I'll tidy it up and check that it works in the Mods folder as well - all my testing as in CustomAssets. I've tried to ensure that I didn't change default behaviour anywhere, but I can test this file set in Windows as well just to make sure. Then I can forward the modified files so that you can merge the changes.

EmperorFool · Aug 12, 2009

Yes, if it had been the problem with sets you would have seen a NameError. Can you be more specific about the problem with tuples?

AlanH · Aug 12, 2009

It looked as if it was the same as for the sets. The error message was the same - SyntaxError: invalid syntax, with a pointer to the first tab character in the eval function argument list. I had to remove all the returns and tabs from that parameter list in init.xml to get it to run. I guess you could remove the tabs and returns from the list before parsing it, to avoid having to destroy the formatting in the init.xml file.

I've played some turns to see if that raised any new errors. There are some more modules that throw runtime errors and need the set fix. I should do a global search for 'set' across the mod and change them all, rather than wait for the errors. Or find a single global change that will solve it across the whole mod.

EmperorFool · Aug 13, 2009

Put this near the very top of CvEventInterface.py (it looks like this is the first module that gets loaded):

Code:

try:
    set()
except NameError:
    import sets
    __builtins__.set = sets.Set
    __builtins__.frozenset = sets.Set

As for the tabs causing syntax errors, that must be a 2.3 vs. 2.4 thing. I remove the carriage returns before calling eval(), but not tabs. I can remove tabs as well.

AlanH · Aug 13, 2009

I've tried the change in CvEventInterface.py. It throws an error:

Code:

Traceback (most recent call last):
  File "<string>", line 1, in ?
  File "<string>", line 52, in load_module
  File "CvEventInterface", line 20, in ?
AttributeError
: 
'dict' object has no attribute 'set'

Failed to load python module CvEventInterface.
ERR: Call function onEvent failed. Can't find module CvEventInterface
ERR: Call function onEvent failed. Can't find module CvEventInterface

From my brief googling, I don't think you can create new attributes in __builtins__ (aka 'dict')

I wondered why the returns didn't show up in the error messages. Should have guessed you had removed them. But, I think you need to remove \r as well, so I have just added line 259 in BugConfig.py to delete \r, and that seems to work. It wasn't the tabs at all.

I'll do the rest of the testing and tidying tonight.

DoctorDon · Aug 13, 2009

I think you want to try the same trick with "__builtin__" (no 's'). Both "__builtin__" and "__builtins__" exist and differ in ways I don't fully understand. They both seem to have the same objects. In a quick test from the interpreter, I could patch a class into __builtin__ and it was accessible in an imported module.

For sanity's sake, though, it's probably a better idea to add the try: import sets... code in every file that uses sets. I've seen production code that hacked global namespaces even worse than this and it was highly frustrating to debug!

AlanH · Aug 13, 2009

I've tried a few tests at the command line within Civ4.

A reference to __builtin__ with no 's' fails: NameError: name '__builtin__' is not defined.

A reference to set fails: NameError: name 'set' is not defined.

If I import sets and enter __builtins__.set=sets.Set, then a reference to set is OK.

So it looks as if the __builtins__ trick works at the command line, but it doesn't work when used in running program.

I think I'll just go quietly and make sure the set=sets.Set assignment is included at the top of each module that uses set.

EmperorFool · Aug 13, 2009

I went for __builtin__ because you got an error in the eval() command, but that was in reference to the leftover '\r's. Adding the trick to each file (remove both __builtin__ from the last two lines) containing "set(" is good enough.

A cleaner option is to put this into a function in BugUtil.

Code:

def defineSets(vars):
    if "set" not in __builtin__:
        import sets
        vars["set"] = sets.Set
        vars["frozenset"] = sets.ImmutableSet

You'd call it from each module using sets like this:

Code:

import BugUtil
BugUtil.defineSets(globals())

That was my original plan, but then I thought of __builtin__ and hoped to just apply the fix once. Cést la vie. I tried it in the IDLE interpreter with no problems. Civ4 must lock it down.

AlanH · Aug 13, 2009

Whatever. It still requires a change to every file using set. I'll find them all and add the code.

EmperorFool · Aug 13, 2009

Can you keep a list as you go? I'll be doing the same to the repository once you've tested that this works. I'll also need to apply the same fix to BUgConfig to remove \r separate from \n. The problem is that I remove the OS-specific line-ending which is different on each OS.

AlanH · Aug 13, 2009

I can give you a diff against the current svn, or a zip containing all the files that I changed, or both.

The line endings are not an issue. Just delete \n and \r if they appear anywhere. If they aren't in the file they won't be deleted.

BUG and Mac

Priest of Hiroshima

Emperor

Emperor

Deity

Emperor

Deity

Mac addict, php monkey

Deity

Mac addict, php monkey

Mac addict, php monkey

Deity

Mac addict, php monkey

Deity

Mac addict, php monkey

Chieftain

Mac addict, php monkey

Deity

Mac addict, php monkey

Deity

Mac addict, php monkey

Similar threads