BUG/BAT Multiplayer - Help!

[HeroFromHyrule]

Ok, so I along with a few friends have gotten back into Civ and have been playing nearly every night. I was poking around on here and found info on the BAT, BUG, and BUFFY mods so I checked them out. One friend and I decided to start up some games using the BUFFY mod, so we did. Everything worked great until we attack something, then we are instantly given the out of sync error. We searched on the forums and found others with the same issue but no solution, so we decided to try the BAT mod. We ran into the same problem with that. We were doing a direct IP game which I read has issues, so we tried alternative ways to play. We have tried playing over Gamespy, but with the BAT mod loaded no matter who hosts the other cannot see the game to join it. We opened all of the ports that were recommended in this post, but it has not helped. We tried playing a LAN game over Hamachi, but the game does not show up when trying over a LAN (even with no mod loaded LAN does not work for us at all).

We downloaded the zipped version of the BUG mod and installed it directly into the BTS folder as instructed in the readme file. If I launch a single player game the BUG mod is functional. If we try playing over Gamespy you can tell the mod is working because when attempting to connect to a game you are given a BUG loading screen hint, but the game does not connect. A direct IP game connects just fine, but for some reason the BUG mod is not loaded, the game plays just like vanilla BTS.

We are at our wit's end. We have started a couple of games with BUFFY and BAT but have really been unable to finish them because of the out of sync issues. However, once you've played even a fraction of a game using any of these mods it is really hard to go back to playing regular BTS, so we're trying to find a way to get this to work so we can still enjoy the benefits of at least the BUG mod.

Anybody have any helpful advice?

[EmperorFool]

First, thank you for a great report and writeup of your actions. This is very helpful. I've got some further questions:

1. Have you tried installing BUG using the EXE?
2. When you said you installed the ZIP version, did you place it into the Mods folder so you have "Mods/BUG Mod 4.2"? The way you describe it (loads up but features disappear) sounds like what would happen if you loaded it into the CustomAssets folder which of course is ignored for all multiplayer games.

I am pretty certain these latest OOS errors are caused by BULL, but I cannot think of what would cause them. There was a crash when you had Stack Attack turned on (a normal BTS feature) which is broken in the BUFFY 3.19.001 release (fixed in v002 which is in public beta right now and in BULL 1.0 which is released). If BULL is the problom, BUG by itself installed from the EXE should work for you without OOS errors.

I can confirm that for some inexplicable reason BAT will not let you see games in the Gamespy lobby. Alerum and I tried many times to no avail. Both BUG and BULL worked, but we didn't actually play a game--just connected.

So try BUG using the EXE's multiplayer install method, and it couldn't hurt to rename your CustomAssets folder just to be sure.

[phungus420]

Originally Posted by EmperorFool

I am pretty certain these latest OOS errors are caused by BULL, but I cannot think of what would cause them.

This same OOS error was introduced into RevolutionDCM when BULL was merged into it. So it's a safe assumption.

[EmperorFool]

Was that BULL 1.0 or a prerelease version (taken from a ZIP or SVN)? The answer to this question will help me track the problem down.

Does anyone know is the code to check for OOS is in the SDK? Is there a function that I can read to see what might be tripping it up?

[phungus420]

Actually now that you mention it. RevDCM 2.5 and 2.51 had BULL in it, but it was the prerelease version glider got from the SVN quite a few months ago. It was RevDCM 2.6 that really broke MP (the OOS error on attacking), and when he integrated BULL in the 2.6 build it was the Official BULL 1.0 release he used. So it was something in a later version of BULL that caused this. I can't be sure, if you want I can try to track down a copy of RevDCM 2.5 and the current version, and you could run a diff. I don't know how usefull that would be though...

[EmperorFool]

The main difference between the pre-release version of BULL and the 1.0 release is the fix for the Stack Attack bug. That gives me a good idea of where to start looking. Unfortunately, the fix I applied seemed pretty straight-forward, and since SA is an interface-only feature I had assumed that each individual unit attack was sent as a command across the network.

[phungus420]

I want to make sure I'm being accurate here, I'm pretty sure I'm correct on that, but can't be positive, and I can't find any reference to BULL in the Rev 2.5 or 2.51 changelog. I've linked this thread from a RevDCM MP thread, and hopefully glider will be along to put in a word or two. He's currently working on a RevDCM MP build, and getting BULL to work right is an important step in that.

[glider1]

@Phungus
Yeah your info is good Phungus. RevDCM 2.51 was BULL 1.0 pre-release and RevDCM 2.60 was BULL 1.0 post release.

@EmperorFool
Here's the scary bit. I'm a complete newbie on multiplayer. When I started work on it, I assumed that there was some sort of underlying mechanism that kept the game in sync at a high level....Then slowly more truth appeared to be revealing itself about multiplayer after confirming with Jdog. There is no underlying mechanism except for the user interface. Anything else that happens on the network but for the user interface, happens because of an assumption about determinism. That is if the sorenRandomNumber is initialised and synchronised across all machines at hook up time, from that moment on, the same sorenRandomNumbers get reproduced on all machines in a deterministic sequence, and each machine on the network runs as if it were in single player. The idea is that only the non-deterministic elements of code are transmitted across the network to keep the game synced. At first I thought that this includes random numbers, but it does not. The only non-deterministic code element that needs to be transmitted across the network are the human interface choices. This is my understanding of how MP works, and we believe that it was an engineering decision done to preserve very small bandwidths for very slow networks that were around when CIV4 was being built.

Unless I am wrong, I shake my head to realise that the multiplayer implementation is absolutely minimalistic. No AI unit behaviour information is sent. No automated production information is sent. All that is ever sent is:
1) That local user interface events, trigger corresponding handlers on all network machines, and all machines execute the handlers in exactly the same way.
2) When local user interface code is running uniquely on a single machine, the game state must not be changed, and no calls to getSorenRandomNumber made either. At this point of execution, events are triggered and for modders, local mod information can be be stored so long as that information is not directly affecting the game state during this time.
3) When any other code is executing on either the local machine or the network, then the game state may be changed. This is because other than at local user interface event moments, all machines are running the same code, as a deterministic assumption.
4) A mathematical OOS algorithm simply checks the coherency of the game state in a very efficient algorithm, and if it detects a difference, multiplayer goes down with an OOS.

This is my understanding, but I am only a beginner on multiplayer truly and respectfully and I could very well be a little bit, somewhat or completely deluded about what Firaxis did with multiplayer.
Cheers

[EmperorFool]

Your understanding matches mine.

BTW, the random number generators are actually called "psuedo-RNGs" because they are actually just a complicated function that takes the previous value as input to produce the next value. One simple example would be "N + 1" which would produce die rolls of 1, 2, 3, 4, 5, 6, 1, 2, 3, 4 ... That would be pretty boring and not very random-looking, so much more complicated formulas are designed to spread the numbers, attempt to hit all values, and "look" random.

Of course, if you roll a die over and over and over, eventually you'll get the same number. PRNGs can never produce the same number back-to-back (or they'd repeat that number forever). This is one way you can tell they are definitely not random, but I digress.

Originally Posted by glider1

1) That local user interface events, trigger corresponding handlers on all network machines, and all machines execute the handlers in exactly the same way.

Yes, but what's transmitted isn't mouse-clicks and key-presses but game commands like "Add an Axeman to London's production queue" and "Change player 3's research rate to 40%". See below why this is important.

2) . . . no calls to getSorenRandomNumber made either.

This was the cause BUG's first OOS error. I saw that there were two RNGs in BTS, so I assumed one was used for game events that had to be kept in sync and the other was for interface randomness (sounds playing, birds flying out of jungle when a unit walks into it, etc). So we used the latter to produce a value as BUG's voting "poll" results fake error factor only to find later that both must be kept in sync. Looking back, I should have just used Python's rand() and been done with it. I don't know what Ruff ended up switching to.

[Note to self: go check!]

3) When any other code is executing on either the local machine or the network, then the game state may be changed. This is because other than at local user interface event moments, all machines are running the same code, as a deterministic assumption.

It can be tough to realize what code qualifies. You must ensure that it is code that is called in response to a mod network event or something similar. But since a function can be called from any number of places, this is tricky. Your IDE can help here.

4) A mathematical OOS algorithm simply checks the coherency of the game state in a very efficient algorithm, and if it detects a difference, multiplayer goes down with an OOS.

Right, my question is "Is this algorithm in the SDK so I can see exactly what it scans? If I can see that it ignores unit names, e.g., I won't bother to worry about the Unit Naming mod causing OOS errors (though the Random Unit Names feature does).

Here's where the above applies to the OOS error in BULL. When you use a stack to attack a plot, does the game send a single "this stack attacks that plot" command to all machines, or does it send one "this unit attacks that plot" command for each unit in the stack until the combat is resolved? I can find this out easily enough when I get home this weekend; I was just wondering if anyone had looked at that code much. In that code is a check for the player's Stack Attack setting.

In any case, I suspect my fix for the null pointer exception I had is what's causing the problem, but I can't think of what problem that could be. Even if the command sent is stack-based, all machines should pick the same units to attack and produce the same combat result. I'm pretty sure that player options are sent over the network, but I suspect that I'm checking the player options in that code, and if I check the "active" player's option, that will be a different player on each machine.

Does anyone know if this problem occurs if both players have Stack Attack turned off? What about if they both have it turned on? In other words, does it only happen if player's have different Stack Attack settings?

[glider1]

@Emporer
I wish I could help but I currently believe the best would be to remove all non-deterministic elements altogether and thus realise a perfect network multiplayer implementation on the network. An algorithm that can determine what a player will select before they select it. Wow that's tricky. It would mean that nothing would have to be transmitted across the network at all. I will now go and contemplate that one
Cheers, thanks for the clarifications and have a peaceful Chrissy.

[HeroFromHyrule]

Originally Posted by EmperorFool

First, thank you for a great report and writeup of your actions. This is very helpful. I've got some further questions:

1. Have you tried installing BUG using the EXE?
2. When you said you installed the ZIP version, did you place it into the Mods folder so you have "Mods/BUG Mod 4.2"? The way you describe it (loads up but features disappear) sounds like what would happen if you loaded it into the CustomAssets folder which of course is ignored for all multiplayer games.

1. Yes. Our initial attempt with BUG was with an install from the EXE
2. No. We installed it directly into the BTS folder so that it was always running

Originally Posted by EmperorFool

I am pretty certain these latest OOS errors are caused by BULL, but I cannot think of what would cause them. There was a crash when you had Stack Attack turned on (a normal BTS feature) which is broken in the BUFFY 3.19.001 release (fixed in v002 which is in public beta right now and in BULL 1.0 which is released). If BULL is the problom, BUG by itself installed from the EXE should work for you without OOS errors.

I can confirm that for some inexplicable reason BAT will not let you see games in the Gamespy lobby. Alerum and I tried many times to no avail. Both BUG and BULL worked, but we didn't actually play a game--just connected.

So try BUG using the EXE's multiplayer install method, and it couldn't hurt to rename your CustomAssets folder just to be sure.

I know I did not have the Stack Attack option turned on as it isn't something I would want on and I'm aware of the issues with it. I'm fairly certain my friend did not have it enabled either. We will attempt an install from the EXE using the multiplayer install method and see how that works. Thanks for the help.

EDIT: Success. We each used a clean install of the game to make sure there would be no issues with the other installations of BUG, BAT, or BUFFY that we had done before, installed BUG from the exe using the Multiplayer install method, and then we were successfully able to play a Direct IP game and attack without the out of sync error. We started a game up as Civs that start with a warrior instead of a Scout and went in search of animals to attack. We ended up finding each other instead so we declared war and attacked each other with no issues. To test out even further, my friend even had a warrior of his attack my capital and there were no problems.

Thanks for the help, I'm grateful that we'll be able to play through a game using the BUG mod without having to worry about sync issues. Also, if you want help testing out things to try and fix sync issues (or any other issues with the mods) just let me know, I have no problems helping out.

[icefire34]

Originally Posted by EmperorFool

Does anyone know if this problem occurs if both players have Stack Attack turned off? What about if they both have it turned on? In other words, does it only happen if player's have different Stack Attack settings?

I tested this and verified that this OOS happens both when I had stack attack turned on as well as off. Both times, I was the only player connected at the time (Pitboss game); does OOS in this case mean my instance of the game is out of sync with the Pitboss game?

[EmperorFool]

Originally Posted by icefire34

Both times, I was the only player connected at the time (Pitboss game); does OOS in this case mean my instance of the game is out of sync with the Pitboss game?

Uh . . . Okay, so I guess PitBoss games run the game on the server as if it were an observing player, so yes, you must be getting OOS with that game. At least this tells me it is not that two players are interacting in some strange way.

Does this happen if you attack with one unit? I had assumed not, but my guess now is that that is the only way you've been attacking. Can you try whichever way you weren't using before? I don't mean Stack Attack, I mean selecting multiple units and attacking.

I'll be home tomorrow night so I can look at this on the weekend.

[ruff_hi]

Originally Posted by EmperorFool

I don't know what Ruff ended up switching to.

[Note to self: go check!]

no need to check ... let me tell you ... nuttin!

Edit: Changed it to the following in 2059 ...

# BUG Poll statistical error
if gc.getGame().isGameMultiPlayer():
	iRandError = 4.0
else:
	iRandError = 3.5 + gc.getGame().getSorenRandNum(10, "Election Results Statistical Error") / 10.0

[EmperorFool]

Aww, let's give the multiplayer gamers some randomness, too! No need to use different methods or Soren's stinky RNG:

import random

...

iRandError = random.uniform(2.5, 4.5)

P.S. You're putting all these changes you make into the changelog, right?

[ruff_hi]

Originally Posted by EmperorFool

P.S. You're putting all these changes you make into the changelog, right?

the what?

Spoiler:

yes I am ... your nagging is finally paying off

[EmperorFool]

You're spoilers are teh awesome!

[Afforess]

I just noticed this thread, and I have a question:

So, any time I call getSorenRand() in python, it will cause OOS's? Do gc.getGame().getMapRand() calls cause OOS too? Should I use the import random, random.uniform() call in place of them?

[EmperorFool]

It depends on what you need the random number for. If your code will run on every human player's computer in a multiplayer game and determine what happens in the game, use Soren's RNG. This will ensure that each computer gets the same random number and thus will take the same action.

If your random number is only used to control some visual non-game-affecting thing for a single player, use that random module I just pointed to. I do not know if the Map RNG must be kept synchronized the same way Soren's must, but I would bet yes.

BTW, take a look at the docs that I linked. There are other random distributions. If you just need a number between 0.0 and 1.0 (you can get 0.0 but cannot get 1.0), use random.random().

[Afforess]

Originally Posted by EmperorFool

It depends on what you need the random number for. If your code will run on every human player's computer in a multiplayer game and determine what happens in the game, use Soren's RNG. This will ensure that each computer gets the same random number and thus will take the same action.

If your random number is only used to control some visual non-game-affecting thing for a single player, use that random module I just pointed to. I do not know if the Map RNG must be kept synchronized the same way Soren's must, but I would bet yes.

BTW, take a look at the docs that I linked. There are other random distributions. If you just need a number between 0.0 and 1.0 (you can get 0.0 but cannot get 1.0), use random.random().

One random number used to generate a number up to 100, and if it is less than 15, in a naval battle, you capture (it really just creates a new ship), the defeated ship. It's the War Prizes modcomp. The other one is to find a city on the map to randomly create a wonder, it's the Circus Hagenbeck code from The J.