Dune Wars Multiplayer - Getting it working

Deliverator · Dec 30, 2010

Latest: Dune Wars now *seems* to be fully working in multiplayer. You need to install Dune Wars 1.9.1 and then the 1.9.1.2 patch. Would love any feedback on your multiplayer experiences/testing, good or bad.

--------------

I abandoned rebuilding Dune Wars using RevDCM 2.83 as base in favour of using Better BUG AI. In the process of working on the RevDCM merge I had a good look into the changes that were made to get multiplayer working in that mod. There seem to be a few areas that might need attention to get Dune Wars working in multiplayer.

1. In RevDCM 2.83, all persistent Python data seems to be stored inside a BugCore object called RevData. A number of Python files refer to this including a MultiplayerEnforcer that ensures RevData is sync'ed at the beginning of a game (as far as I can tell). It makes sense that you need to be careful how you handle persistent data in order have everything in sync.

2. I asked Lemmy101, what he did to get Multiplayer working in RevDCM 2.83 and he said the main issue was with using modNetMessage and dialogs (which Dune Wars has e.g. landing stages and homeworld screen):

From PM:
"Well the main things I fixed in RevDCM were involving when there was some kind of player choice in a dialog that needed to be sent via a mod net message in the python script to be applied on the other player's PC's, it didn't bear in mind the fact that the player who sent the message also gets the message. As I remember there was some kind of auto UI based python script that handled sending the net messages when the player clicked a choice or clicked OK. This system was ALSO calling the thing the netmessage does on the otherside, meaning it got called twice, once by the direct call, and the other by the netmessage.

OR it could be the other way around. They may have ASSUMED they got the message they sent, when they didn't. And perhaps I made it call them directly.

I can't remember which way around, but the main issues revolved around this."

If the same approach can be taken to net messages in Dune Wars perhaps that would help.

3. Lemmy101 also did some work to ensure AI Autoplay works in Multiplayer - this might not be required for Dune Wars.

From PM:
"The only actual DLL changes I made, IIRC, was fixing AI Autoplay to work multiplayer, so really if you look in Revolutions.py you should see most of the changes I made in comment tags, it should be quite trivial to transfer those fixes to yours if you search for lemmy101."

4. Bringing code out of Python into the DLL may help with some issues for example Afforess brought the Inquisition code into the DLL reducing the Python functionality before RevDCM 2.83.

5. There is a reported issue with the 1.8 version of Dune Wars that unit combat causes OOS errors which might need to be addressed. See the end of this old multiplayer thread.

Update: This was caused by an issue with BULL described in this thread.

Moving to the Better BUG AI base has simplified Dune Wars code more, but multiplayer still has issues even in Better BUG AI apparently. Anyway, I thought I'd share the information I have found so far.

Deliverator · Dec 31, 2010

This is the thread that describes how multiplayer was fixed in the RevolutionsMP mod.

cyberchris · Jan 3, 2011

Hello,
I really like your mod and would love to play it with my wife in multiplayer.
But whenever we try to start a game, the game of the hoster breaks down.

In singleplayer everything works fine. And without the mod, multiplayer works, too.

The Colonization TAC-Mod we have to ply with direct ip-Connect. But that did not work, too. Hot- Seat also did not work. Do you have an idea or is it too early to try playing the mod in multiplayer?

Thank you,
Cyberchris

Deliverator · Jan 3, 2011

Lots of people have said they would love Dune Wars to work in multiplayer, but unfortanately I really lack the know-how to do much about it. They are probably only a handful of people with experience of successfully debugging and fixing multiplayer for mods and unless one of those people decides to help out I may not be able to do much about it.

davidlallen · Jan 3, 2011

I agree this is very painful. At present, the version of RevDCM which DW is built on top of, is broken for multiplayer. Have you completely discarded the idea of merging 2.83 with DW? That seems to be the fundamental problem. That may, actually be the only problem. I have written successful multiplayer python code in previous mods, so I claim (we cannot prove

that the DW python is multiplayer safe. It is possible that koma's homeworld code may not be multiplayer safe. So, if 2.83 was magically ported, and the homeworld code were temporarily disabled, it *might* work. Unfortunately it requires an expert programmer to successfully merge the current DW code with 2.83.

Deliverator · Jan 4, 2011

davidlallen said:
I agree this is very painful. At present, the version of RevDCM which DW is built on top of, is broken for multiplayer. Have you completely discarded the idea of merging 2.83 with DW? That seems to be the fundamental problem. That may, actually be the only problem. I have written successful multiplayer python code in previous mods, so I claim (we cannot prove that the DW python is multiplayer safe. It is possible that koma's homeworld code may not be multiplayer safe. So, if 2.83 was magically ported, and the homeworld code were temporarily disabled, it *might* work. Unfortunately it requires an expert programmer to successfully merge the current DW code with 2.83.

I abandoned the merge of RevDCM with Dune Wars because I could get the plain RevDCM 2.83 DLL to work with Dune Wars XML. On a total conversion mod like this picking through the XML to find the bit that RevDCM doesn't like is a needle in a haystack problem.

It is cleaner to use Better BUG AI as a base for Dune Wars because Better BTS AI and BUG are the main things Dune Wars uses from RevDCM anyway. The entirety of Revolutions mod and almost all of DCM are not used. The new version made on the Better BUG AI code base has a longer less superfluous code which would be a good thing in ordinary circumstances.

The multiplayer fix by Lemmy101 was actually done to Revolutions not RevDCM. Revolutions doesn't include BUG or Better BTS AI (as far as I know). It's weird that Better BUG AI has issues with multiplayer - I presume it is more likely to be BUG than Better BTS AI that's causing an issue. Perhaps there is a buggy BUG component that was disabled in RevDCM 2.83.

The stable version of Dune Wars made using Better BUG AI is not too different from RevDCM 2.83 in that they are both recent Better BTS AI + recent BUG + other stuff. It is possible to merge in the RevDCM 2.83 stuff into the Better BUG AI version, but whether that will yield a version that works in single player let alone multiplayer is anyone's guess.

To be honest, we're probably unlikely to get multiplayer working without someone actually debugging the errors.

Deliverator · Jan 6, 2011

I've finally successfully merged in all of lemmy101 and jdog5000's changes from RevDCM 2.83 into the Better BUG AI edition. All seems stable in single player. Hopefully, this will make Dune Wars multiplayer much more stable but I can't guarantee anything as no MP testing has been done.

I'll call this Dune Wars 1.9.1 even though it is a complete new version of the mod and not a patch.

I don't have time to write a full patch note now, but I thought it would be worth posting a link anyway, in particular to see if multiplayer is now more stable.

Here's the link to Dune Wars 1.9.1. This is an executable installer just like the 1.9 release. If you get any issues, then try removing your Mods/Dune Wars folder and run the installer again.

As well as the multiplayer fixes from RevDCM 2.83 there are some other changes to gameplay, most noteably I have changed the polar region in the Arrakis to be a proper polar sink. I'll write a full patch release note when I get a chance.

davidlallen · Jan 6, 2011

That is awesome! I think this will be a big help in getting multiplayer to work. I have "claimed" before that most of the DW-specific features should be multiplayer friendly, because a lot of it is built on top of the vanilla random event dialog, and this works for multiplayer. The selection of offworld goods and mentats work this way.

I just looked into the code for the homeworld screen, and there may be a problem. When any human player makes a choice, such as purchasing a new unit, this decision needs to be communicated to all the multiplayer clients using a python function sendModNetMessage and then processed with another function onModNetMessage. There was a recent thread about this in the sdk forum, and I pointed the fundamental guide for programming multiplayer. Sadly there are no calls to either of these functions in the homeworld code written by koma13.

I recommend people try out the multiplayer version. If OOS is found, please pay attention to whether any player has used the homeworld screen. It may be that everything works; great! It may be that the games stay in sync only when players agree not to use the homeworld screen. In that case deliverator and I can study out the homeworld code; it may be that only a few lines of change are needed to make it work. (This would be around the call to initUnit which gives the unit to the human player.)

Ahriman · Jan 6, 2011

If OOS is found, please pay attention to whether any player has used the homeworld screen. It may be that everything works; great! It may be that the games stay in sync only when players agree not to use the homeworld screen.

The AI uses the homeworld purchase, doesn't it?
So there would also need to be some kind of game option that disables AI access.
"No offworld reinforcements" seems like a reasonable game option anyway that players might want to impose.

davidlallen · Jan 6, 2011

Ahriman said:
The AI uses the homeworld purchase, doesn't it?
So there would also need to be some kind of game option that disables AI access.
"No offworld reinforcements" seems like a reasonable game option anyway that players might want to impose.

Unfortunately(?), the AI part will work fine. The issue is when the game presents a dialog to the human player, and the player makes a choice. Changing the database only on that one player's machine is the wrong answer. The right answer is to broadcast the information about what this human did, to all the client machines. Then all the machines will perform the database change, and the games will stay in sync.

I'm not sure it will help everybody, but I think the multiplayer tutorial link I gave does an excellent job of explaining the flow of data for multiplayer.

Deliverator · Jan 7, 2011

davidlallen said:
I just looked into the code for the homeworld screen, and there may be a problem. When any human player makes a choice, such as purchasing a new unit, this decision needs to be communicated to all the multiplayer clients using a python function sendModNetMessage and then processed with another function onModNetMessage. There was a recent thread
about this in the sdk forum, and I pointed the fundamental guide for programming multiplayer. Sadly there are no calls to either of these
functions in the homeworld code written by koma13.

Koma's homeworld screen *does* use sendModNetMessage and onModNetMessage:

CvHomeworldAdvisor.py:

Code:

                                  lPurchasedUnits = []
                                       for id in self.UnitList:
                                               lPurchasedUnits.append(self.mercenaries[iPlayer][id])
                                               del self.mercenaries[iPlayer][id]

[B]                                     CyMessageControl().sendModNetMessage(690, iPlayer,
self.iSelectedCity, -1, -1)[/B]
                                       mercData[PURCHASED_UNIT] = lPurchasedUnits
                                       self.UnitList = []

DuneWars.py:

Code:

  # Receive modnetmessage for various button presses
       def onModNetMessage(self, argsList):
               iMessageID, iData2, iData3, iData4, iData5 = argsList
               if iMessageID == 690:
                       # Homeworld Screen: koma13
                       pCity, pUnit = objMercenaryUtils.placeMercenaries(argsList)
                       self.onUnitBuilt([pCity, pUnit])

But I could see that the stuff that is stored using BugData, the unit lists, etc, could get out of sync. I may try and do some testing this weekend otherwise we are just speculating.

Also, lemmy101's code in AIAutoPlay.py adds an event handler to onModNetMessage which does something I don't understand but I guess is related to the PM info I quoted in the opening post of this thread.

davidlallen · Jan 7, 2011

Sorry for the false alarm. I had only looked in mercenaryutils.py.

Deliverator · Jan 7, 2011

OK, I've done some testing and it's mostly good news.

Using my laptop and my desktop PC I set up a multiplayer game using 1.9.1b over Direct IP. Lemmy101's genius code lets you set both machines to Autoplay meaning I can do automated testing of multiplayer. Very cool. Everything seems rock solid - no issues with combat causing OOS or any issues that I could see.

The only OOS I saw happens when purchasing a unit from the homeworld screen which we anticipated might be an issue. I saw an OOS when the Autoplay was running after Offworld Trade so I don't think the problem is restricted to human controlled players. It must be possible to fix this somehow I'm sure. Perhaps we need to change the way the homeworld data is stored or something to do with the modnetmessage handling.

There is a single Python error in the logs:

Code:

Traceback (most recent call last):
  File "BugEventManager", line 371, in _handleDefaultEvent
  File "DuneWars", line 252, in onModNetMessage
  File "DuneWars", line 412, in onUnitBuilt
AttributeError: 'int' object has no attribute 'getOwner'

...which may be part of the issue.

Also a section of Python debug log:

Code:

Modder's net message!

PY:onModNetMessage
23:14:52 DEBUG: BugData - returning open table root.MercenaryData
23:14:52 DEBUG: BugData - returning open table root.MercenaryData
23:14:52 DEBUG: BugData - root.MercenaryData.AvailableMercenaries = {0: {0: {'Promotions': ['PROMOTION_MERCENARY'], 'UnitType': 99, 'Cost': 210, 'Elite': 1}, 1: {'Promotions': ['PROMOTION_MERCENARY'], 'UnitType': 76, 'Cost': 100, 'Elite': 0}, 2: {'Promotions': ['PROMOTION_MERCENARY'], 'UnitType': 27, 'Cost': 140, 'Elite': 0}, 3: {'Promotions': ['PROMOTION_MERCENARY'], 'UnitType': 11, 'Cost': 90, 'Elite': 0}, 4: {'Promotions': ['PROMOTION_MERCENARY'], 'UnitType': 99, 'Cost': 140, 'Elite': 0}, 5: {'Promotions': ['PROMOTION_MERCENARY'], 'UnitType': 27, 'Cost': 210, 'Elite': 1}, 6: {'Promotions': ['PROMOTION_MERCENARY'], 'UnitType': 27, 'Cost': 140, 'Elite': 0}, 7: {'Promotions': ['PROMOTION_MERCENARY'], 'UnitType': 76, 'Cost': 100, 'Elite': 0}}}
23:14:52 DEBUG: BugData - root.MercenaryData.UniqueID = 8
23:14:52 DEBUG: BugData - returning open table root.MercenaryData
23:14:52 DEBUG: BugData - root.MercenaryData.PurchasedUnit = {}
23:14:52 DEBUG: BugData - root.MercenaryData.PurchasedUnit -> {}
23:14:52 TRACE: Error in ModNetMessage event handler <bound method DuneWars.onModNetMessage of <DuneWars.DuneWars instance at 0x1713A6E8>>
23:14:52 TRACE: 'int' object has no attribute 'getOwner'
23:14:52 DEBUG: Timer - scores took 63 ms

Anyway, overall I'm pretty pleased - if purchasing units from the homeworld screen is the only issue then we've made a big step towards fully stable Dune Wars multiplayer.

Plus since I can run multiplayer autoplays we have a way of testing any fixes we might come up with.

Deliverator · Jan 7, 2011

I think potentially Alia's Abomination trait might be causing an OOS as well.

Also, this looks very handy for locating what exactly is out of sync.

Alexej · Jan 8, 2011

Now everything looks fine except we cant load any savegame to continue, i never experienced MAFs now they prevent us from continue playing saved game. We went into turn 50 or so, and saved because of other activity and returned to play 2 hours ago to find out, we must start new game.

Deliverator · Jan 8, 2011

Alexej said:
Now everything looks fine except we cant load any savegame to continue, i never experienced MAFs now they prevent us from continue playing saved game. We went into turn 50 or so, and saved because of other activity and returned to play 2 hours ago to find out, we must start new game.

I've noticed that too. No idea what the problem there is. Could be tough to fix.

I always get a version mismatch error when connecting - not sure if that is related - I've confirmed that all the mod files are exactly the same on both machines so that's strange too.

Deliverator · Jan 8, 2011

Update: I have reworked the Homeworld code so that it is now multiplayer friendly - every player now has a copy of every other players homeworld list and all is kept nicely in sync. I've run a 500 turn autoplay multiplayer game and the were no OOS - the AI was purchasing offworld units fine - I also stopped the autoplay and bought some offworld units manually to check that also works. So the multiplayer side of things looks pretty good.

Unfortunately, there's an issue with savegames in my new version - on reloading - the game CTDs with the message "failed to decompress game data". This is not specific to multiplayer, but all savegames. Normally this happens if you've modded something since saving - but it's happening all the time even when nothing has changed. If I can figure that one out we'll be in good place.

Deliverator · Jan 8, 2011

Alexej said:
Now everything looks fine except we cant load any savegame to continue, i never experienced MAFs now they prevent us from continue playing saved game. We went into turn 50 or so, and saved because of other activity and returned to play 2 hours ago to find out, we must start new game.

This issue should be fixed by applying the 1.9.1.1 patch - let me know how you get on.

AbsintheRed · Jan 9, 2011

So everything is working and absolutely stable?
Wow, huge kudos to you Deliverator! Thanks

Deliverator · Jan 9, 2011

AbsintheRed said:
So everything is working and absolutely stable?
Wow, huge kudos to you Deliverator! Thanks

Well, I'm not going to say definitely until I get feedback from other people who've tried it - but it looks good so far.

Dune Wars Multiplayer - Getting it working

Graphical Hackificator

Graphical Hackificator

Chieftain

Graphical Hackificator

Deity

Graphical Hackificator

Graphical Hackificator

Deity

Tyrant

Deity

Graphical Hackificator

Deity

Graphical Hackificator

Graphical Hackificator

Prince

Graphical Hackificator

Graphical Hackificator

Graphical Hackificator

Deity

Graphical Hackificator

Similar threads