Better DuneBUG AI

[Deliverator]

For a long time we have discussed whether it is possible to rebuild Dune Wars without using RevDCM as a base.

I have now made a new version of the mod built on Better BUG AI - this version is called Dune Wars 1.8X.

The changes in version numbers from 1.8 is as follows:

Spoiler :

Dune Wars 1.8 base:
RevDCM - version 2.61
Better BTS AI - version 0.82
BUG - version 4.2
BULL - version 1.0

Better DuneBUG AI base:
RevDCM - not included
Better BTS AI - version 0.90
BUG - version 4.3
BULL - version 1.1

The mod is playable and at a stage where play-test feedback would be useful even though there are a few issues.

Download the Dune Wars 1.8X installer HERE

Notes/Remaining Issues:

1. I *think* I have merged over all custom Dune Wars functionality from 1.8 including:
- Dales Combat Mod
- The latest Orion's Inquistion Mod implementation of Inquisitors

2. The Gamefont small icons for bonus resources are messed up and the water drop gamefonts have reverted to bread. This is easy to fix later.

3. The version of Better AI that is included allows different Leaders to prioritize different victories and has new code that focuses on achieving them. I have disabled the Space race victory, but I guess this code could cause issues.

4. I have included updated versions of the mapscripts. I'm fairly sure that the intermittent crashing of Arrakis.py has been due to the mapscript deciding that an impassable Peak feature is great spot for a city location. I have moved the "chrome" code that adds the Outcrop and Peak features inside the mapscript so that it executes before starting plots are decided. Arrakis.py now seems more stable.

5. I have used a Debug DLL to fix a number of Assert errors. There are still some strange ones which perhaps need to be addressed to get the mod stable.

6. I have generally operated a bare-minimum policy in bringing across code so potentially I missed some bits of code that I didn't appreciate were needed.

Any help with testing and debugging this version of Dune Wars would be appreciated. It feels like I am more than 95% of the way to a new stable base for Dune Wars.

 

[davidlallen]

For a long time we have discussed whether it is possible to rebuild Dune Wars without using RevDCM as a base.

Excellent! This was a challenging project.

2. You will need to manually create the folder Better DuneBUG AI within My Documents/My Games/Beyond the Sword. Within that folder create another folder called UserSettings. This is so that Bug does not complain about being unable to locate the settings folder.

If players have trouble with this it should be possible to modify the "main release" installer script I posted on the release howto thread. All that should be needed is changing one "Dune Wars" to "BDBAI" or whatever you like.

1. I *think* I have merged over all custom Dune Wars functionality from 1.8 with a few exeptions:

Have you tried the various spy missions like arranged marriage and reassign city? This is based on Super Spies, which is part of RevDCM.

4. The thing I've had most trouble with is the Homeworld screen/Mercenaries Mod. [...] KeyError: 31[...]

			[COLOR="Red"][B]del mercenaries[iPlayer][id][/B][/COLOR]

Perhaps it is obvious and not sufficient, but have you added a print statement to make sure that iPlayer is valid, and id is less than the length of the list?

5. For some reason I haven't been able to work out yet, civs that are not running Arrakis Paradise civic are able to build Catchbasins and Reservoirs of Liet.

This is controlled by cannotConstruct. Is it possible that in your xml, you did not turn on this callback? See assets/xml/PythonCallbacks.xml. Is it possible that in your init code for BUG, you did not register this callback? Temporarily add a print statement inside cannotConstruct to verify it is being called. Warning, if it is called, it is called a *lot*, don't spam your screen.

8. I have used a Debug DLL to fix a number of Assert errors. There are still some strange ones which perhaps need to be addressed to get the mod stable.

Cleaning out asserts is moderately interesting. In my experience, at least with RevDCM, there are dozens of asserts which happen even without any modifications to RevDCM. I am not sure exactly how to do it, but if you can verify that an assert happens without the DW modifications, then do not worry about it.

 

[phungus420]

The latest version of RevDCM, 2.8 on the svn, has a lot of functionality in it you guys could use; it would be really easy to simply extricate Revolutions, and have everything else work (just remove the gameoption outright, then find the dozen or so places where it is called, and set it up so that those control statements are always handled as though Revolutions is off). Here is a list of the 2.8 modding stuff, along with SuperSpies, DCM, and such:

from RevolutionDCM_ModderInfo.txt:

---------------------------------------------
Added XML tags:

In general you can easily reference the added XML tags inclued in RevDCM by checking the first item in the list.
For instance if you examine the Unit Lion (the first unit in UnitInfos), you will see this:
<!-- RevDCM Inquisitions -->
<bInquisitor>0</bInquisitor>
<!-- RevDCM Inquisitions -->
Showing where in UnitInfos the added tag is, and how to use it (the name should, in general be self explanitory).
So reference the first object in the XML to see the structure of the tags added by RevDCM, which are listed below:

BuildingInfos:

• PrereqGameOption -If defined the building will require the spedified Game Option to be built
• NotGameOption -If defined the building may not be constructed if the specified game option has been selected
• iUnitUpgradePriceModifier -Spefies a % modifier to UnitUpgrade gold cost (Leonardo's Workshop)
• iDCMAirbombMission -Allows and sets odds of air bombing building (should be applied to all new buildings you wish to be bombable)

TraitInfos:

• bNonStateReligionCommerce -If enabled non state religions still apply culture
• bUpgradeAnywhere -If enabled units can be upgraded outside national borders

CivicInfos:

• bUpgradeAnywhere -Allows units of civ running this civic to upgrade units outside cultural borders
• bAllowInquisitions -Allows uits defined as inquisitors to run the inquisitions mission
• bDisallowInquisitions -Dissallows conducting the inquisitions mission (overides all other considerations)

EraInfos:

• iTechCostModifier -Applies general modifier to TechCost by Era

TechInfos:

• bDCMAirBombTech1 -If enabled DCM air bombing chances increased
• bDCMAirBombTech2 -If enabled DCM air bombing chances increased

FeatureInfos:

• iWarmingDefense -gives defense against global warming
• GrowthSound -Sets a growth sound

UnitInfos:

• bInquisitor -If set the unit may conduct the Inquistion mission
• MaxStartEra -If set the unit can not be trained on starts later then defined era
• ForceObsoleteTech -If set the unit can not be trained once the defined tech has been researched
• bStateReligion -If set the unit may only be trained in cities with the state religion
• PrereqGameOption -If set the unit may only be trained if the defined game option has been selected
• NotGameOption -If set the unit may not be trained if the defined game option has been selected
• PrereqOrCivics -(array) If set one of the defined civics must be run by the player to train the unit
• PrereqBuildingClasses -(array) If set the unit may only be trained if the defined buildingclasses have been built in the city
• ForceObsoleteUnitClasses -(array) If Any unit set in the array is trainable in the city, the unit may not be trained
• iDCMBombRange -If set the unit may bombard tiles up to the range defined
• iDCMBombAccuracy -Accuracy of DCM bombarding
• bDCMAirBomb1 -Allows fighter support
• bDCMAirBomb2 -Allows bombing of buildings
• bDCMAirBomb3 -Allows bombing of factories
• bDCMAirBomb4 -Allows bombing of port facilities to attach ships directly in cities
• bDCMAirBomb5 -Allows bombing (destroying hammers) invested in city's current production
• bDCMFighterEngage -Allows unit to engage enemy air units directly

------------------------------------
API

In general the python API for exposed functions follows the same conventions as standard Civ4.
So for instance to get the UnitUpgradePrice modifier for a building you can use:
kBuilding.getUnitUpgradePriceModifie() to pull the value for a building from BuildingInfos
and you can see how this is effecting a player with:
pPlayer.getUnitUpgradePriceModifie()

There may be some slight differences as well. For instance if you call getRevIdxNational(),
on a specified object that applies an effect, such as a building, civic, or trait,
the call will return the value for that specific object, defined in that object's infos;
however if you pull it from a specified player using pPlayer.getRevIdxNational()
the function will return the cumultive effects of all of these currently being applied to the specified player.

Nearly all XML tags, and any derivative functions are exposed to python (if you find one that is not, let us know and we'll expose it)


There are a few noteworthy non XML api functions added to the SDK, most have been exposed to python:

CvPlayer & CyPlayer:
isInquisitionConditions() -boolean, returns true if inquisition conditions are valid for a player, false if not
isHasHolyCity() -boolean, returns true if a Player has a holy city
isHasStateReligionHolyCity() -boolean, returns true if the player has their state religion holy city
isHasStateReligionShrine() -boolean, returns true if a player has their state religion shrine
canFoundReligion() -boolean, returns true if the player can found a religion (used mainly for limited Religions, but will work if this option is off, it'll just always return true)

CvGame & CyGame:
countNumReligionsFounded: -integer, returns the number of religions that have been founded in a game
countNumReligionTechsDiscovered -integer, returns the number of Techs that have been discovered which can found a religion (should only differ from above function in Limited Religions games)
isTechCanFoundReligion(TechTypes eIndex) -function returns true if the passed tech can still found a religion (similar to isReligionSlotTaken, but checked against the tech instead of a religion)

-------------------------------------
Python Console Mod Tools:
See thread for information on how to use these:
http://forums.civfanatics.com/showthread.php?t=361879

Also Multiplayer is almost entirely fixed. I just dont' really think it would be worth it to drop it, updating to 2.8, or at least the version that's currently in the SVN would make the most sense. 2.8 likely wol't be released for a month or two, but the current SVN version already has the above functionality with no major bugs known in it; and multiplayer is even working.

 

[Deliverator]

Excellent! This was a challenging project.

Yes indeed, it's been a fair amount of work getting this far.

Have you tried the various spy missions like arranged marriage and reassign city? This is based on Super Spies, which is part of RevDCM.

Not tested yet, but I was careful to bring across all the TSHEEP and Super Spies related code I could find.

Perhaps it is obvious and not sufficient, but have you added a print statement to make sure that iPlayer is valid, and id is less than the length of the list?

I think it's because the pickling stuff for the Mercenaries is not actually working properly. I thought I had it working, but it not seems broken again in the version I've posted. I'm a bit stuck really, so if fancy applying your greater Python experience to it I'd be happy.

Edit: Looking in the PythonDbg.log it looks like the table that stores the Mercanaries Mod data is not being properly stored for some reason. There are repeated entries like:
17:18:27 DEBUG: BugTable - Mercenaries Mod not found in root
17:18:27 DEBUG: BugData - returning open table root.Mercenaries Mod
17:18:27 DEBUG: BugData - returning open table root.Mercenaries Mod.MercenaryData
17:18:27 DEBUG: BugData - new data for root.Mercenaries Mod.MercenaryData: {'Sequence': 'on', 'UniqueID': 0, 'AvailableMercenaries': {}}

Edit Edit: Actually, now I'm getting an error:
Traceback (most recent call last):
File "BugEventManager", line 366, in _handleDefaultEvent
File "DuneWars", line 213, in onModNetMessage
File "MercenaryUtils", line 328, in placeMercenaries
KeyError: 2

Looks like a problem with index out of bounds again.

This is controlled by cannotConstruct. Is it possible that in your xml, you did not turn on this callback? See assets/xml/PythonCallbacks.xml. Is it possible that in your init code for BUG, you did not register this callback? Temporarily add a print statement inside cannotConstruct to verify it is being called. Warning, if it is called, it is called a *lot*, don't spam your screen.

Thanks, a simple fix. I hadn't brought PythonCallbacks.xml over at all.

 

[davidlallen]

Also Multiplayer is almost entirely fixed.

I am always reluctant to use an intermediate version in svn. It can change daily. It is true that one motivation for removing revdcm was lack of multiplayer. Perhaps if other players / mods find multiplayer completely working, we might add it back.

But, this raises a good point. Would anybody be able to test multiplayer with this modified BDBAI version? If it works, that will be awesome. If not, at least there is less to investigate about why it may still be broken.

 

[Deliverator]

My brain is fried from trying to fix this Homeworld/Mercenaries Mod issue. I can't even seem to get debug messages to appear. I'll revisit tomorrow unless someone has figured it out by then.

 

[phungus420]

I am always reluctant to use an intermediate version in svn. It can change daily.

And for good reason, I just found a critical bug in it, lol. First one in a while though, and it's in a function I added literally yesterday. Anyway I've isolated the cause at least, and should have a fix up in a couple of hours.

 

[Deliverator]

Well, I'm pretty much defeated by the Homeworld screen issue.

If I use the DW 1.8 SdToolkitCustom then everything works, but the Mercenary Mod data re-initializes every time the mod is loaded or saved meaning the list of available units in the HW screen changes.

If I use BUG version of SdToolkit then its even worse. It seems the data is never even stored and every time I enter the HW screen the Mercenary Mod data is re-initialized and I get a different list.

I've tried debugging this, but I can't figure out the reason at all. Might have to give up on this project as I'm not sure it's worth the effort...

 

[davidlallen]

In the 1.8 release, is everything working for the mercenary save/restore? A "known, minor bug" where the mercenary list is not preserved after a save/restore does not seem too bad to me. I don't suppose people very often see a cool unit in the merc list, but can't afford it, and go back later expecting to still find it.

 

[davidlallen]

I haven't investigated mercenaryutils very carefully, and I've never used sdtoolkit. I see sdtoolkit has some "sdEcho" calls commented out which print some trace information; maybe you could remove the comments, try a small game, and see if the print statements shed any light on anything. Also, I notice that sdModLoad and sdMlodSave use try/except to catch any exception. That is nice, but if something is wrong, these calls will hide it from you. You might add an sdEcho or similar statement inside the "except" part to see if this is triggering. Not sure what to do if it is, or why it would just start happening.

 

[keldath]

deliverator,

i just freaking love you (in a manly way off course.. )

im excited over my head that you built a new betterduneai dll.

supreme work .

im sure you put many hours on this and much frustration!


so,
im trying to dl - but the link is dead,
plz reupload the file, and ill help you - ill go over the entire files and ill complare it to the old dune code to help you find missing dune codes iff there are any.

also about dale,
if you decide to add it, i think getting it from revdcm would be a harder work for you,
i have a code from the merged mod by ninja, and that code have dale code in it without revdcm, and it might be easier to add, also ninja have improved some stuff on the range bombardment and accuracy i think along with air combat.

anyway,
supreme work, i wanna help you test this, so plz upload the code (with the defeated homeworld code ), im going to be unemployed in 3 weeks so ill have some time on my hand to play dune! yeah!

 

[davidlallen]

Mediafire appears to be dead. Hopefully it is only temporary. I cannot even view my own files, there is a "database error". It may be worthwhile to put a temporary note into the DW download page. I am sure you have the files locally; do you have a preferred mirror site?

 

[keldath]

yeah david your right, i cant get into my files either...crap...

 

[Deliverator]

Mediafile seems to be back up. From here at least.

A "known, minor bug" where the mercenary list is not preserved after a save/restore does not seem too bad to me. I don't suppose people very often see a cool unit in the merc list, but can't afford it, and go back later expecting to still find it.

Personally, I think it's pretty much a shopstopper for the homeworld screen. If you have autosave set every 5 turns the list of units rebuilds every 5 turns. You can exploit by repeatedly saving your game until the unit you want shows up. Saving the game shouldn't affect the game.

I haven't investigated mercenaryutils very carefully, and I've never used sdtoolkit. I see sdtoolkit has some "sdEcho" calls commented out which print some trace information ... Not sure what to do if it is, or why it would just start happening.

The problem is that adding the debugging just tells me that the storage table Mercenary Mod is not being found, but I don't know why the storage is failing. Perhaps more logging around exceptions, as you say, might help.

im excited over my head that you built a new betterduneai dll.

Cheers, keldath. In terms of check the Python files - I don't want to bring over any more files unless they are really needed - I want to keep this new version as clean and light as possible.

It would be nice to figure out this Mercenary Mod data storage issue. At the moment, I've decided it is much more fun to add new features to the SDK rather than debug black-box Python that I don't really understand.

also about dale,
if you decide to add it, i think getting it from revdcm would be a harder work for you

I was planning to use the 'vanilla' version of DCM to keep it nice and simple.

If you want to try out Better DuneBUG AI:

1. Install folder in Mods.
2. Copy PythonCallbacks.xml from Dune Wars 1.8 to Better DuneBUG AI/assets/Python. This file was missed by me.

You can switch MercenaryMod.py and HomeworldAdvisor.py between importing SdToolkit and SdToolkitCustom if you want to see the issues I've described.

 

[davidlallen]

Mediafile seems to be back up. From here at least.

I agree, I was able to start downloading the BDBAI file.

Personally, I think it's pretty much a shopstopper for the homeworld screen. If you have autosave set every 5 turns the list of units rebuilds every 5 turns. You can exploit by repeatedly saving your game until the unit you want shows up.

You can always exploit the game with save/restore with random seeds. Having the list randomly change every five turns is still not fatal, in my opinion, but I don't buy mercenaries that much.

For the other mod I am working on, I have a similar need to store arbitrary data inside the game scriptdata. I have never used sdtoolkit and it seems rather complicated for my needs. So, no commitment, but I will try to design something that can be used for both, instead of relying on the black box behavior of other components.

 

[dammitbiscuit]

corrupting files again

Mediafire appears to be dead. Hopefully it is only temporary. I cannot even view my own files, there is a "database error". It may be worthwhile to put a temporary note into the DW download page. I am sure you have the files locally; do you have a preferred mirror site?

Rapidshare? Megaupload?

Rapidshare is the easiest to upload to, but it has the slowest download speeds. Megaupload is a little more work for the uploader, but it provides faster downloads and doesn't force you to wait 15 minutes between downloads if you're downloading multiple files.

I know when I'm downloading something from , rapidshare links are the most common, mediafire links are a last resort, and I try to find megaupload links when I can.

A program called JDownloader will, if you Ctrl C a download link, take the download link from your clipboard, navigate to the download in question, beat the CAPTCHA if necessary, wait the required number of seconds if you're an unpaid user, and then download and unzip the file for you.

 

[Fimbulvetr]

8. I have used a Debug DLL to fix a number of Assert errors. There are still some strange ones which perhaps need to be addressed to get the mod stable

While you're at it, could you build a pure Better BUG AI Debug DLL r71 for me? I tried it myself but obviously I'm doing it wrong. Infos about what Asset failures/bugs you ran into and how you fixed them would be appreciated too.

I am always reluctant to use an intermediate version in svn. It can change daily.

Better BUG AI is a merge of those daily changing svn versions of Better AI, BULL and BUG, so that shouldn't bother you.

It is true that one motivation for removing revdcm was lack of multiplayer. Perhaps if other players / mods find multiplayer completely working, we might add it back.

But, this raises a good point. Would anybody be able to test multiplayer with this modified BDBAI version? If it works, that will be awesome. If not, at least there is less to investigate about why it may still be broken.

Multiplayer completely working? That would be too good to be true but who knows. Both newest RevDCM svn and Better BUG AI should be reasonably stable in MP though.

 

[Deliverator]

While you're at it, could you build a pure Better BUG AI Debug DLL r71 for me? I tried it myself but obviously I'm doing it wrong. Infos about what Asset failures/bugs you ran into and how you fixed them would be appreciated too.

I think all of the Assert errors I fixed were specific to Dune Wars content. I'll try and compile a Debug DLL this weekend.

 

[Deliverator]

I've managed to fix the Homeworld screen issue by reworking it to use the BugData class rather than SdToolkit. BugData seems altogether nicer and easier to use for persistent data. Better logging too.

Better DuneBug AI now seems to be stable. I may as well fix the pedia python and gamefonts before posting a patch - then this should be pretty much the same as Dune Wars 1.8 except with more up-to-date versions of BUG and Better AI.

 

[keldath]

sweet deliverator,

glad to now you got the hw fixed,
dl the file, gonna try it now, and walk through the sdk, for fun

so - is this gonna be in 1.8.1?
and the gamefonts?