[PYTHON] Random number generation OOS error - is there a solution?

[Capt_Blondbeard]

Hi - I've seen a lot of discussion that using the random number generator in python causes MP OOS errors. Has anyone found a solution to this? Is it related to the getSorenRand function, or random numbers in general (i.e. if you created a mathematical function from scratch that generated random numbers does the problem persist?

[xienwolf]

As long as you use getSorenRand, AND you do it in a way that doesn't rely on the activeplayer's condition, you won't have sync issues.

SorenRand ensures that everyone uses the same algorithm for their random numbers, so as long as everyone makes a random number, they all get the same result. If you use a local function to make a rand, then each person's seed is different, and each OS's algorithm is different, so the results won't match.

[killmeplease]

Originally Posted by xienwolf

As long as you use getSorenRand, AND you do it in a way that doesn't rely on the activeplayer's condition ...

can't you describe this more detailed? what's active player condition?
how can i use random numbers and not to cause oos - is it enough just to use sorerand? Is it applicable for python or sdk as well?

[EmperorFool]

An OOS error occurs if the game state on one machine differs from any other machine.

One way this can happen is if one machine calls getSorenRandNum() without any other machine doing it too. The Soren random number generator is part of the game state, and it must be used identically by every machine.

Another way is to have some game-state-altering code run only for the active player without telling the other machines what it did. If you want to add 10 to the active player's treasury every turn, make sure you send a network message to add the gold so every machine does it.

These issues can crop up in the SDK and Python.

[xienwolf]

ActivePlayer refers to the guy behind the computer. So in a multiplayer game, each human is their own ActivePlayer.

If you have a python routine that does something like:
if (getActivePlayer) == CIVILIZATION_ROME
getSorenRand()
...

Then this will cause an OOS, because only one of the humans is Rome, so he grabs an "extra" random number, making him not agree with everyone else.

Same issue if you base it on keypresses or mouse clicks/movements. Like having a random number generated when someone does a mouseover of a building to choose from a variety of flavorful text to supply with the stats. Seems harmless enough, but since not all players will mouse over the building at the same time, they will each be continuously shifting their Randoms so that they can no longer agree with one another.


As long as you avoid these and similar mistakes, using getSorenRand solves all of your issues. Technically you could also use getMapRand, but there is little need to do so, and it'll just confuse people who try to use your work and don't know about the second function.

[Capt_Blondbeard]

OK, I am using the function within a button which I created according to the button tutorial. And it is conditional on the player being active: ie if the unit is active and the unit is in a certain position... But the button has to tie to this unit and conditions - so how would this work? I assume the getSorenRand gets a seed from somewhere - can you set this manually?

Lets say we've gotten to the point where the button has been pushed - can I somehow jump to a global, or grab a global instance? Or maybe just save the fact that the button has been pushed, but wait for the end of the round or the beginning of the next round to do the action?

[EmperorFool]

It sounds like you want a player to be able to initiate a game action--just like attacking or moving a unit--via the interface (button), and all players' machines should perform the same action for that player's unit. For this you want to send a mod net message which is a network message containing a custom command with some data.

For an example, see BUG's CvStrategyOverlay.py module. Each message requires code that sends it and code that receives and handles it. It's not terribly complicated once you get the hang of it.

If you provide more specifics we might be able to help you better.

[davidlallen]

I highly recommend this guide to learn about oos.

[Capt_Blondbeard]

Ah-hem
**nonchalant whistling**

After reading the guide to multiplayer modding I now suspect that the problem is not related to the sorenrand function, but rather to problems with global vs. local reference.

I am enclosing the code below. Actually just one function, but the others are similar, so if I can fix this one I should be able to fix the others.

PHP Code:

        #New Merchant function
        if (inputClass.getNotifyCode() == 11 and inputClass.getData1() == 301 and inputClass.getData2() == 301):
            self.iPushedButtonUnit = g_pSelectedUnit
            pUnit = g_pSelectedUnit
            iX = self.iPushedButtonUnit.getX()
            iY = self.iPushedButtonUnit.getY()
            pMerchantLocation = CyMap().plot(iX, iY)
            pUnitOwner = gc.getPlayer( pUnit.getOwner( ))
            iUnitOwnerID = pUnitOwner.getID() 
            pyUnitOwner = PyHelpers.PyPlayer(pUnit.getOwner()) 
            if pMerchantLocation.isCity():
                TradeCity = pMerchantLocation.getPlotCity ()
                pTradePlayer = gc.getPlayer(TradeCity.getOwner())
                if pTradePlayer.getID() != pUnitOwner.getID() :
                    jRnd = gc.getGame().getSorenRandNum(100, "Trade")
                    if jRnd <= 50 :
                        pUnitOwner.changeGold( 40 + gc.getGame().getSorenRandNum(40,"Trade Result"))
                        CyInterface().addImmediateMessage(CyTranslator().getText("TXT_MERCHANT_GOLD",()),'')
                    else :
                        gc.getTeam(pUnitOwner.getTeam()).changeResearchProgress (pUnitOwner.getCurrentResearch(),50 + gc.getGame().getSorenRandNum(20,"Research"),iUnitOwnerID)
                        CyInterface().addImmediateMessage(CyTranslator().getText("TXT_MERCHANT_RESEARCH",()),'')

                    g_pSelectedUnit.kill(false,-1)

        # End 


Sooo ... to me the only thing looking suspicious (to me) is the pUnit = g_pSelectedUnit Would this then be different for each player, depending on what unit they are selecting? But if this is the problem, I am not sure how to fix.

Thanks!

PS: I suspect my variable naming still needs some work... but in my defence, my work keeps getting in the way of me spending more time modding...

[davidlallen]

In general, if you are displaying information only and not changing any game state, you can do anything you want. But, if you are changing game state, you need to call modnetmessage as soon as possible, and do all your actual work in the function which receives the modnetmessage. Otherwise we can guarantee your code will cause oos.

[EmperorFool]

The problem is that your code runs only on the machine of the player that clicked the button (or whatever triggered your code). This means only one player calls getSorenRandNum(), and that causes the sequences to go out of sync.

As davidallen says, you need to detect your UI event and then send a mod net message so every players' machine runs the code. Everything starting with the line

jRnd = gc.getGame().getSorenRandNum(100, "Trade")

should become a "command" that you broadcast via mod net message. Each machine will receive it, roll the die, and possibly give money to the player that owns the unit. Or you can move the net message higher up to include everything game-related (checking the unit and city and trade routes, etc). That's generally the better route. The UI should trigger the mod net message, and the handler for the message should contain all of the game logic.

In your code, BTW, both the use of Soren rand num and changeGold() cause OOS.

[Capt_Blondbeard]

Ok, cool, I wanted to define a lot of that stuff as commands anyway, as I have some duplication in my mod. But how exactly does the mod net message work? Is there a mod somewhere that uses this command so that I could see how to use it?

[xienwolf]

Fall from Heaven and Fall Further use it a few times each. Mostly you just need to figure out a way to condense any action you want to perform into 5 integer values.

[davidlallen]

I learned about modnetmessage from the Gods Of Old mod which is part of the BTS release. The Great Priest of each religion has an action button to cause a meteor strike, or tsunami, or whatever, and these use modnetmessage. I never wrote a proper guide. I just looked through the "python action button" tutorial in the tutorial subforum, written by TC01, but this actually has exactly the same problem you started with -- it does not use modnetmessage, so it is not MP-safe.

My older mod Fury Road also uses modnetmessage (see my sig), but it is based on the way Gods Of Old does it. Dune Wars also follows the same model. So there are a few places to look.

[EmperorFool]

BUG uses it in its StrategyOverlay.py module (see my sig). Specifically, search for onModNetMessage to see how to receive the message and sendModNetMessage for sending. Also, the Python API shows the details:

CyMessageControl.sendModNetMessage(int iData1, int iData2, int iData3, int iData4, int iData5)

[Capt_Blondbeard]

OK, let me see if Ive got this straight (and I'm not a programmer, so I am probably not using the correct terminology)

If you start with the gods of old mod:

The "action trigger" if you will, is within the CyMainInterface file. It basically watches for the widget that indicates a button has been pushed, then defines the five integer values that will be passed via the net message to the actual command, which will be executed (hopefully) identically on all networked computers.

CyEventManager is where the "action" takes place - it watches for the action to be triggered, then uses the 5 integers to perform the action.

Is there any other file that comes into play?

I assume that the tricky part will be converting everything to integer values to pass to the command, the converting back into pointers for actions that require pointers rather than integers as arguments.

One more question - why 5 integer values? Couldn't you define NetMessage to accept more? (Not that I need it, just curious)

[davidlallen]

That is a good explanation. It is not hard to pass integers; pPlayer.getID() returns the int for a player which you convert back with with gc.getPlayer(), pUnit.getID() returns the int for a unit which you convert back with pPlayer.getUnit(), etc. I am sure that modNetMessage originally passed three ints, then the civ engine authors had one application which needed four, then they had one which needed five.

[xienwolf]

The actual communication between the computers happens in the EXE, or at least in one of the DLLs we don't have access to. As such, we cannot change this message from 5 integers to some other set of values. What they gave us is all we get.

[EmperorFool]

But you can use CvMessageData to define your own net messages containing other data types. Look at CvMessageData.cpp and you'll see a bunch of subclasses for passing messages. Each one takes a specific set of data, e.g. for founding a religion:

CvNetFoundReligion::CvNetFoundReligion(PlayerTypes ePlayer, ReligionTypes eReligion, ReligionTypes eSlotReligion) : CvMessageData(GAMEMESSAGE_FOUND_RELIGION),
	m_ePlayer(ePlayer),
	m_eReligion(eReligion),
	m_eSlotReligion(eSlotReligion)
{
}

which it stores as member variables in the object. It uses PutInBuffer() and SetFromBuffer() to send them over a stream, just like writing a saved game:

void CvNetFoundReligion::PutInBuffer(FDataStreamBase* pStream)
{
	pStream->Write(m_ePlayer);
	pStream->Write(m_eReligion);
	pStream->Write(m_eSlotReligion);
}

void CvNetFoundReligion::SetFromBuffer(FDataStreamBase* pStream)
{
	pStream->Read((int*)&m_ePlayer);
	pStream->Read((int*)&m_eReligion);
	pStream->Read((int*)&m_eSlotReligion);
}

Each has an Execute() function to perform the work on each machine after the values have been read back in from the network:

void CvNetFoundReligion::Execute()
{
	if (m_ePlayer != NO_PLAYER)
	{
		GET_PLAYER(m_ePlayer).foundReligion(m_eReligion, m_eSlotReligion, true);
	}
}

Finally, there's a Debug() function for debugging your messages:

void CvNetFoundReligion::Debug(char* szAddendum) 
{ 	
	sprintf(szAddendum, "Non-simultaneous found religion notification");	
}

All of the messages are sent via CvMessageControl functions

void CvMessageControl::sendFoundReligion(PlayerTypes ePlayer, ReligionTypes eReligion, ReligionTypes eSlotReligion)
{
	gDLL->sendMessageData(new CvNetFoundReligion(ePlayer, eReligion, eSlotReligion));
}

which are called by the game code:

case BUTTONPOPUP_FOUND_RELIGION:
    CvMessageControl::getInstance().sendFoundReligion(
            GC.getGameINLINE().getActivePlayer(), 
            (ReligionTypes)pPopupReturn->getButtonClicked(), 
            (ReligionTypes)info.getData1());

I'm going to attempt to fix Reminder Mod so reminders from non-host players will be saved in multiplayer games. I'll write up a tutorial on what it takes.

[EmperorFool]

Yay! It works and wasn't too hard to do. [Edit: Turns out passing a string is easy too since there are Read/WriteString() functions on the stream.]

Note you only need to do this if you need to pass arguments that you cannot squish into 5 integers (e.g. a string).