Setting up an alternate random number generator

NovHak

Chieftain
Joined
Feb 7, 2019
Messages
8
Location
France
Dear forum readers,

I would like to replace the standard random number generator of Civ IV with another one, more precisely with one that is not pseudo-random, hence seedless. Is that possible ? I already did that for Unity-based games, writing my own UnityEngine.Random class, but I could as well write a DLL with C-style exports for API functions.

It seems this game is heavily based on Python, which I possibly could try my hand at too. While my intention is to use Intel's SecureKey (RDRAND) as an RNG, I may be happy using Windows' RNGCryptoServiceProvider, which I think Python implements already.

Ideally, I would like the RNG replacement to be across the whole game (game setup (incl. map generation) and play time), but if that's not possible, map generation at the very least.

I noticed a "random.pyc" binary file in the game files, does it contain all of the game's random generation procedures ? If yes, where could I find info about the functions used and their profiles, and could I write a replacement file ?

Any help is welcome !
 
I think all randomly generated numbers in the game ultimately come from the CvRandom class in the DLL. With the right library I'm sure you could rely on a different RNG by overriding this class. The DLL is responsible for game mechanics where RNG has the most impact (including combat). But as far as I know the Python code also relies on the RNG implemented in the DLL, instead of any Python native RNG. So with the above change Python should be taken care of as well.

Why are you trying to do this though? True randomness isn't really needed outside of specific use cases such as cryptography. If anything, deterministic random number generation is more desirable in games because reuse of the seed ensure reproducibility.
 
Thanks for pointing me to CvRandom, which I just checked. However I don't know how I could replace it, since it's likely part of a game DLL containing other pieces of code for which no source is available. I doubt "Civ4\Beyond the Sword\CvGameCoreDLL\CvRandom.cpp" is compiled at runtime, but rather only given for reference to modders. Maybe it could be achieved through techniques like DLL injection, but I have to say I'm not familiar with this.

Now since you asked me, there are a few reasons to prefer relying on true RNGs.

1) Too often, deterministic RNGs are either badly designed, or badly implemented (which is precisely the case of CvRandom, more on that later)

2) Even when they're correctly designed and implemented, they still can be badly seeded (I bet CvRandom is badly seeded too, but I didn't see that yet)

3) Even when correctly designed, implemented and seeded, precisely because of their deterministic nature, they're strongly reducing the possibilities in complex operations such as world generation.

4) Our ancestors used dice. Why not go for something conceptually similar in the computing world, now that it's available ?

Deterministic RNGs have been a necessary evil in the early days of computing science since computers are highly deterministic, and this was the best achievable, being cheap, giving enough illusion of randomness, and far easier and more efficient than having the computer perform true dice rolls. True RNGs have been existing for some time but were inaccessible or too expensive for the common folk. This is not true any more, and almost every modern enough home computer has a built-in true RNG, since Intel and AMD have implemented the RDSEED opcode in their processor instruction sets. And I'm not even speaking of the OS-backed RNGCryptoServiceProvider class (to speak only of Windows), which makes use of external events, deemed random, such as time between keystrokes, but not only that.

I understand that seedable RNGs have the big advantage of reproducibility, which can be useful for testing, or in some situations in games where reproducibility is crucial. But even then, in most of the cases (with the exception of testing, I agree), it would be possible to store the outcome instead of storing the seed, at the expense of a little more coding, but with a big gain in randomness.

Now, to the very specific case of CvRandom, it's based on a linear congruential generator, which is fairly basic (I would have prefered the Mersenne Twister, or Xorshift), and moreover it's badly implemented, not performing discards to ensure a "fair dice roll", hence would fail a uniform distribution test. No, no, and no, RoundDown((RANDOM*6)/(MAX_RANDOM+1)+1) won't give you a fair dice roll, but that's far too frequent a mistake. And it seems mistakes in this domain are even more frequent in the gaming industry.

Last but not least, concerning the seed, it's too often set by default as the number of ticks since system boot, and sometimes even worse choices are made. The tick count since boot is not very good, rarely more than a small fraction of the possible values will be likely because :

1) The user will have his machine rebooted long before
2) It's very sensitive to the user's habits : "I'm booting my computer at 8 AM, and start playing at 6 PM". Agreed, it's unlikely that the user will be precise to the millisecond, but that still restricts him to a very small fraction of what should be possible.

I've been positively surprised to see some games explicitly seeding with the lower bits of the number of ticks since the Epoch (1st of january, 1970, 00:00:00 UTC), but that's fairly rare. However, even then, imho, a true RNG should be used for seeding.

Those were the reasons why I'm constantly advocating that random number generation gets properly used in games, and try to modify them myself whenever (reasonably) possible.
 
The compile guide: https://forums.civfanatics.com/threads/the-easiest-way-to-compile-a-new-dll.608137/

The guide to avoid network desyncs: https://forums.civfanatics.com/threads/fix-and-avoid-oos-issues-dll-and-python.639164/

The latter is actually really important for this thread. If a piece of code is executed in parallel on all computers and that piece of code requests a random number, the game will go out of sync unless all computers will get the very same random number. This means the task for CvRandom is to create the illusion of creating random numbers while at the same time ensure that all the numbers are completely predictable.

If you can make a better random number generator then please do so and please share it as we would all like good random numbers. However if you manage to break multiplayer in the process of getting random numbers, do not expect your random number generator to be popular. A lot of work has gone into various mods to ensure they work well in multiplayer.
 
Okay, so the code present into CvGameCoreDLL is the whole DLL code, that's interesting, thanks for the info.

Is there any chance that it would work with VS2017 Community instead of VS2010 Express ? Because I already have VS2017 installed and I got reports of people having problems with different VS editions installed on the same computer.

I'm well aware about multiplayer very often relying on seed synchronisation, and I noticed the serialisation part in the CvRandom class. However there are workarounds, such as serialising the random numbers history for the current turn instead of the seed, but I guess I won't be able to use the RDRAND/RDSEED RNG since it's likely that some setups won't have it implemented yet. On the other hand, RngCryptoServiceProvider should be available on all systems.

Tbh, I didn't intend to share my work with anyone in the first place, since it seems to me that most people just don't care about this RNG issue, but of course any potential interest changes everything, and I will share my work once I come up with something satisfying.
 
Is there any chance that it would work with VS2017 Community instead of VS2010 Express ?
In short: yes you can use 2017.

The compiler has to be the same as the one used for the exe, meaning 2003. Since this one has a really dated user interface, the setup is to use a newer version and then use a makefile to call the 2003 compiler. This means you can use any version of MSVS, notepad++ or whatever you like. The only thing, which matters is calling the makefile with the right arguments, something which is already set in the MSVS project files. You can compile from cmd.exe if you like though I wouldn't recommend it unless you aim for scripted nightly builds or something like that. Using MSVS 2017 is likely the best choice.

it seems to me that most people just don't care about this RNG issue
There are two kinds of modders: those who just want the gameplay to (mostly) work and those who cares about details like that. The forum talk is mostly dominated by the first group though I can assure you that the latter group exist as well. WTP certainly aims to be in the latter category.
 
In short: yes you can use 2017. [.../...]
Okay, thanks, that gives me some insight about what does what as well, since I'm not much used to how the compilation process works on Windows.

Silly question: Isnt one of the main reasons for seeded rnd that it makes multiplayer more smoothly, since they work on "clientside" instead of "serverside"?
Well, in both cases, the RNG would work clientside, then synchronise with other parties, according to the operations performed in turn by each client. At least I suppose that's how it works. I may have to check this in more detail for proper serialisation if I ever try to implement a seedless RNG, but if I remain deterministic I won't have to touch anything.
 
In short: yes you can use 2017....

...Using MSVS 2017 is likely the best choice..

Ive been using vs2010, but would REALLY like to use 2017. I tried once but it didnt work, though looking at my makefile, it does seem a bit... dated... ;)
What would I need to change besides the makefile?

My makefile:
PHP:
#### Civilization 4 SDK Makefile 1.0 ####
####  Copyright 2010 Danny Daemonic  ####
#########################################

#### Paths ####
TOOLKIT=C:\Dev\Microsoft Visual C++ Toolkit 2003
PSDK=C:\Dev\WindowsSDK
CIVINSTALL=D:\Games\Civilization IV Complete\Civ4\Beyond the Sword
GLOBALBOOST=$(CIVINSTALL)\CvGameCoreDLL\Boost-1.32.0
GLOBALPYTHON=$(CIVINSTALL)\CvGameCoreDLL\Python24
## Uncomment to have newly compiled dlls copied to your mod's Assets directory
YOURMOD=$(CIVINSTALL)\Mods\VIP

#### Tools ####
CC="$(TOOLKIT)\bin\cl.exe"
CPP="$(TOOLKIT)\bin\cl.exe"
LD="$(TOOLKIT)\bin\link.exe"
RC="$(PSDK)\bin\rc.exe"
## Uncomment to build dependencies using fastdep
FD="$(MAKEDIR)\bin\fastdep.exe"

#### BLACKLIST ####
## Uncomment to block CvTextScreen (accidentally included by Firaxis)
BLACKLIST=CvTextScreens

#### You shouldn't need to modify anything beyond this point ####
#################################################################

#### Target Files ####
Debug_BIN=Debug\CvGameCoreDLL.dll
Release_BIN=Release\CvGameCoreDLL.dll

!IF [IF NOT EXIST CvGameCoreDLL.rc EXIT 1] == 0
Debug_RESOURCE=Debug\CvGameCoreDLL.res
Release_RESOURCE=Release\CvGameCoreDLL.res
!ENDIF

Debug_STATICLIB=Debug\CvGameCoreDLL.lib
Release_STATICLIB=Release\CvGameCoreDLL.lib

Debug_LIBDEF=Debug\CvGameCoreDLL.def
Release_LIBDEF=Release\CvGameCoreDLL.def

Debug_PCH=Debug\CvGameCoreDLL.pch
Release_PCH=Release\CvGameCoreDLL.pch

Debug_PDB=Debug\CvGameCoreDLL.pdb
Release_PDB=Release\CvGameCoreDLL.pdb

Debug_OTHER=Debug\CvGameCoreDLL.exp Debug\CvGameCoreDLL.ilk
Release_OTHER=Release\CvGameCoreDLL.exp

#### CFLAGS ####
GLOBAL_CFLAGS=/GR /Gy /W3 /EHsc /Gd /Gm- /DWIN32 /D_WINDOWS /D_USRDLL /DCVGAMECOREDLL_EXPORTS /Yu"CvGameCoreDLL.h"
Debug_CFLAGS=/MD /Zi /Od /D_DEBUG /RTC1 /Fp"$(Debug_PCH)" $(GLOBAL_CFLAGS)
Release_CFLAGS=/MD /O2 /Oy /Oi /G7 /DNDEBUG /DFINAL_RELEASE /Fp"$(Release_PCH)" $(GLOBAL_CFLAGS)

#### LDFLAGS ####
GLOBAL_LDFLAGS=/DLL /NOLOGO /SUBSYSTEM:WINDOWS /LARGEADDRESSAWARE /TLBID:1
Debug_LDFLAGS=/INCREMENTAL /DEBUG /PDB:"$(Debug_PDB)" /IMPLIB:"$(Debug_STATICLIB)" $(GLOBAL_LDFLAGS)
Release_LDFLAGS=/INCREMENTAL:NO /OPT:REF /OPT:ICF /PDB:"$(Release_PDB)" $(GLOBAL_LDFLAGS)

#### INCLUDES ####
GLOBAL_INCS=/I"$(TOOLKIT)/include" /I"$(PSDK)/Include" /I"$(PSDK)/Include/mfc" /I"$(GLOBALBOOST)/include" /I"$(GLOBALPYTHON)/include"
PROJECT_INCS=/IBoost-1.32.0/include /IPython24/include
Debug_INCS=$(PROJECT_INCS) $(GLOBAL_INCS)
Release_INCS=$(PROJECT_INCS) $(GLOBAL_INCS)

#### LIBS ####
GLOBAL_LIBS=/LIBPATH:"$(TOOLKIT)/lib" /LIBPATH:"$(PSDK)/Lib"  /LIBPATH:"$(GLOBALBOOST)/libs" /LIBPATH:"$(GLOBALPYTHON)/libs" winmm.lib user32.lib
PROJECT_LIBS=/LIBPATH:Python24/libs /LIBPATH:boost-1.32.0/libs/ boost_python-vc71-mt-1_32.lib
Debug_LIBS=$(PROJECT_LIBS) $(GLOBAL_LIBS) msvcprt.lib
Release_LIBS=$(PROJECT_LIBS) $(GLOBAL_LIBS)

#### Objects ####
Debug_LINKOBJS=$(Debug_OBJS)
Release_LINKOBJS=$(Release_OBJS)

#### Auto SOURCES/OBJS ####
!IF [ECHO SOURCES= \> sources.mk] == 0 && \
    [FOR %i IN (*.cpp) DO @ECHO. "%i" \>> sources.mk] == 0 && \
    [ECHO.>> sources.mk] == 0 && \
    [ECHO Debug_OBJS= \>> sources.mk] == 0 && \
    [FOR /F "delims=." %i IN ('dir /b *.cpp') DO @ECHO. Debug\%i.obj \>> sources.mk] == 0 && \
    [ECHO.>> sources.mk] == 0 && \
    [ECHO Release_OBJS= \>> sources.mk] == 0 && \
    [FOR /F "delims=." %i IN ('dir /b *.cpp') DO @ECHO. Release\%i.obj \>> sources.mk] == 0 && \
    [ECHO.>> sources.mk] == 0
!INCLUDE sources.mk
!IF [DEL sources.mk]
!ENDIF
!ENDIF

#### Targets ####
#################

.PHONY: all clean Debug_clean Release_clean Debug Release

all: Debug Release

clean: Debug_clean Release_clean

Debug_clean:
    @FOR %i IN ($(Debug_BIN) $(Debug_STATICLIB) $(Debug_LIBDEF) \
        Debug\*.obj Debug\*.@ $(Debug_RESOURCE) \
        $(Debug_PCH) $(Debug_PDB) $(Debug_OTHER)) DO @IF EXIST "%i" DEL "%i"

Release_clean:
    @FOR %i IN ($(Release_BIN) $(Release_STATICLIB) $(Release_LIBDEF) \
        Release\*.obj Release\*.@ $(Release_RESOURCE) \
        $(Release_PCH) $(Release_PDB) $(Release_OTHER)) DO @IF EXIST "%i" DEL "%i"

Debug: Debug_DIR Debug_unfinished $(Debug_PCH) $(Debug_BIN)
!IFDEF YOURMOD
    -COPY "$(Debug_BIN)" "$(YOURMOD)\Assets\."
!ENDIF

Release: Release_DIR Release_unfinished $(Release_PCH) $(Release_BIN)
!IFDEF YOURMOD
    -COPY "$(Release_BIN)" "$(YOURMOD)\Assets\."
!ENDIF

Debug_DIR:
    -@IF NOT EXIST "Debug\." MKDIR "Debug"

Release_DIR:
    -@IF NOT EXIST "Release\." MKDIR "Release"

Debug_unfinished:
    @ECHO.>Debug\unfinished.@
    @FOR /F "delims=@" %i IN ('dir /b Debug\*.@') DO \
        @IF EXIST "Debug\%i" DEL "Debug\%i"
    @FOR /F %i IN ('dir /b Debug\*.@') DO \
        @IF EXIST "Debug\%i" DEL "Debug\%i"

Release_unfinished:
    @ECHO.>Release\unfinished.@
    @FOR /F "delims=@" %i IN ('dir /b Release\*.@') DO \
        @IF EXIST "Release\%i" DEL "Release\%i"
    @FOR /F %i IN ('dir /b Release\*.@') DO \
        @IF EXIST "Release\%i" DEL "Release\%i"

$(Debug_BIN): $(Debug_LINKOBJS) $(Debug_RESOURCE)
    $(LD) /out:$(Debug_BIN) $(Debug_LDFLAGS) $(Debug_LIBS) $(Debug_LINKOBJS) $(Debug_RESOURCE)

$(Release_BIN): $(Release_LINKOBJS) $(Release_RESOURCE)
    $(LD) /out:$(Release_BIN) $(Release_LDFLAGS) $(Release_LIBS) $(Release_LINKOBJS) $(Release_RESOURCE)

.cpp{Debug}.obj:
    @ECHO.>"$*.obj.@"
    $(CPP) /nologo $(Debug_CFLAGS) $(Debug_INCS) /Fo$*.obj /c $<
    @DEL "$*.obj.@"

.cpp{Release}.obj:
    @ECHO.>"$*.obj.@"
    $(CPP) /nologo $(Release_CFLAGS) $(Release_INCS) /Fo$*.obj /c $<
    @DEL "$*.obj.@"

$(Debug_PCH) Debug\_precompile.obj:
    @ECHO.>"$(Debug_PCH).@"
    @ECHO.>"Debug\_precompile.obj.@"
    $(CPP) /nologo $(Debug_CFLAGS) $(Debug_INCS) /YcCvGameCoreDLL.h /Fo"Debug\_precompile.obj" /c _precompile.cpp
    @DEL "$(Debug_PCH).@"
    @DEL "Debug\_precompile.obj.@"

$(Release_PCH) Release\_precompile.obj:
    @ECHO.>"$(Release_PCH).@"
    @ECHO.>"Release\_precompile.obj.@"
    $(CPP) /nologo $(Release_CFLAGS) $(Release_INCS) /YcCvGameCoreDLL.h /Fo"Release\_precompile.obj" /c _precompile.cpp
    @DEL "$(Release_PCH).@"
    @DEL "Release\_precompile.obj.@"

.rc{Debug}.res:
    @ECHO.>"$*.res.@"
    $(RC) /Fo$@ $(Debug_INCS) $<
    @DEL "$*.res.@"

.rc{Release}.res:
    @ECHO.>"$*.res.@"
    $(RC) /Fo$@ $(Release_INCS) $<
    @DEL "$*.res.@"

!IFDEF BLACKLIST

Debug\$(BLACKLIST).obj: $(BLACKLIST).cpp
    @ECHO.>"$*.obj.@"
    @ECHO.>"$*-dummy.cpp"
    $(CPP) /nologo $(Debug_CFLAGS) $(Debug_INCS) /Y- /Fo$@ /c "$*-dummy.cpp"
    @DEL "$*-dummy.cpp"
    @DEL "$*.obj.@"

Release\$(BLACKLIST).obj: $(BLACKLIST).cpp
    @ECHO.>"$*.obj.@"
    @ECHO.>"$*-dummy.cpp"
    $(CPP) /nologo $(Release_CFLAGS) $(Release_INCS) /Y- /Fo$@ /c "$*-dummy.cpp"
    @DEL "$*-dummy.cpp"
    @DEL "$*.obj.@"

!ENDIF

!IFDEF FD

!IF [IF NOT EXIST $(FD) EXIT 1] == 0
!IF [$(FD) --objectextension=pch -q -O Debug CvGameCoreDLL.cpp > depends] != 0 || \
    [$(FD) --objectextension=obj -q -O Debug $(SOURCES) >> depends] != 0 || \
    [$(FD) --objectextension=pch -q -O Release CvGameCoreDLL.cpp >> depends] != 0 || \
    [$(FD) --objectextension=obj -q -O Release $(SOURCES) >> depends] != 0
!MESSAGE Error running fastdep.
!ENDIF
!ELSE
!IF [ECHO "fastdep.exe" NOT FOUND! && \
     ECHO Please edit Makefile to reflect the correct path of fastdep. && \
     ECHO. ]
!ENDIF
!ENDIF

!ENDIF

!IF EXIST(depends)
!INCLUDE depends
!ENDIF
 
What would I need to change besides the makefile?
Just download the makefile in my signature. It contains the project files as well, complete with the updated calls to the makefile. Add existing source files and add the source files from your project and you are good to go.

I'm not much used to how the compilation process works on Windows.
Normally windows projects have the compilation set in the project files, which works fine if you use the newest compiler (compiler and IDE match version). To use another compiler than the default one in MSVS you have to use a makefile to control the compilation. It works ok, but setting up a makefile in windows is tricky (harder than GNU makefiles) and it's rarely used.
 
Tbh, I didn't intend to share my work with anyone in the first place, since it seems to me that most people just don't care about this RNG issue, but of course any potential interest changes everything, and I will share my work once I come up with something satisfying.
That's mostly because it isn't actually an issue. From an end user perspective how random numbers are generated really does not change anything about the gameplay experience at all. As long as they are random enough that you can't guess the next one reliably and the odds shown in game are roughly analogous to what actually comes out of the system it just does not matter. The RNG could literally just be a piece of code that takes the current second, divides by 60 to normalize it to a [0-1] range and nobody would notice the difference.

Not that what you are doing isn't cool and that I don't applaud it as a project for its own sake. Because I totally do. As a programmer my self I can certainly appreciate the desire to do something like this. And I would definitively be interested in seeing your results both at the end and as they progress.
 
Now that I think about it, there is the option of checking if the game is a network game. This means if you write some super random number generator, which then happens to cause network desyncs, then we can write code which will make network games use the vanilla code while single player will use your new code. It seems to me that it's a bit messy to not use the same code in both single player and multi player, but it is an option, which can be used if everything else fails.

The RNG could literally just be a piece of code that takes the current second, divides by 60 to normalize it to a [0-1] range and nobody would notice the difference.
That's pretty much what I did with WTP. There was a request for using a random loading screen background each time the loading screen shows up and I ended up using the time passed since the computer booted modulo the number of images. It feels completely random to the player despite not actually containing anything at random at all.

Not that what you are doing isn't cool and that I don't applaud it as a project for its own sake. Because I totally do. As a programmer my self I can certainly appreciate the desire to do something like this. And I would definitively be interested in seeing your results both at the end and as they progress.
I will say pretty much the same thing. If you manage to come up with something, which actually works then I would be interested. While I do not consider the random numbers to be an issue, any improvement would still be an improvement. Also apart from the gameplay I would be interested in just reading what you have done and why as the need for proper random numbers is something we need once in a while, not just for modding.
 
That's pretty much what I did with WTP. There was a request for using a random loading screen background each time the loading screen shows up and I ended up using the time passed since the computer booted modulo the number of images. It feels completely random to the player despite not actually containing anything at random at all.
Another thing you could have used is the current millisecond count. You can typically get those strait out of the box with most programming languages with what ever time class they use. And it looks random enough.

The reason why pseudorandom generators are used in games really isn't to ensure randomness but to ensure repeatable randomness so that you can have features like Civ's "new random seed on reload" which allows you to permit or block save scumming. You can't have that without a seeded RNG.

I will say pretty much the same thing. If you manage to come up with something, which actually works then I would be interested. While I do not consider the random numbers to be an issue, any improvement would still be an improvement. Also apart from the gameplay I would be interested in just reading what you have done and why as the need for proper random numbers is something we need once in a while, not just for modding.
Quite. If nothing else it makes for fascinating reading.
 
Just download the makefile in my signature. It contains the project files as well, complete with the updated calls to the makefile. Add existing source files and add the source files from your project and you are good to go.

I added the 4 files from the makefile-2.5+project+fastdep-0.17.rar and changed the makefile to the following:
PHP:
#### Paths ####
#
# Note: $(PROGRAMFILES) converts to "C:\Program Files", "C:\Program Files (x86)" or whatever fits your system.
# run "nmake.exe /P" in cmd.exe to see what it is on your system
TOOLKIT=C:\Dev\Microsoft Visual C++ Toolkit 2003
PSDK=C:\Dev\WindowsSDK
## Uncomment to have newly compiled dlls copied to your mod's Assets directory
YOURMOD=$(CIVINSTALL)\Mods\VIP

## Civ install path
## Path to the directory where boost and python is stored
## Overwritten by enviroment variable CIV4_LIB_INSTALL_PATH
CIV4_PATH=D:\Games\Civilization IV Complete\Civ4\Beyond the Sword\CvGameCoreDLL

However, when I open the Civ4DLL.sln it opens up empty, (no sourcefiles or anything) and if I open CvGameCoreDLL.sln it comes up with some errors when i compile:
Error U1050 Target "" not supported. Supported targets: Debug Release Assert Profile CvGameCoreDLL C:\Dev\CvGameCoreDLL\makefile 218
Error MSB3073 The command "nmake /NOLOGO Debug_clean" exited with code 2. CvGameCoreDLL C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\Microsoft.MakeFile.Targets 33

any clues?

I should prolly note that Im using Vs2017 15.9.2, but trying to compile on 2010 brings a somewhat similar error:
1>------ Rebuild All started: Project: CvGameCoreDLL, Configuration: Debug Win32 ------
1>Build started 09/02/2019 11.09.04.
1>makefile(218): fatal error U1050: Target "" not supported. Supported targets: Debug Release Assert Profile
1> Stop.
1>C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\Microsoft.MakeFile.Targets(33,5): error MSB3073: The command "nmake /NOLOGO Debug_clean" exited with code 2.
1>
1>Build FAILED.
1>
1>Time Elapsed 00:00:00.14
========== Rebuild All: 0 succeeded, 1 failed, 0 skipped ==========
 
Last edited:
Error U1050 Target "" not supported. Supported targets: Debug Release Assert Profile CvGameCoreDLL C:\Dev\CvGameCoreDLL\makefile 218
Error MSB3073 The command "nmake /NOLOGO Debug_clean" exited with code 2. CvGameCoreDLL C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\Microsoft.MakeFile.Targets 33
You aren't using the included project file. Your file contains
Code:
nmake /NOLOGO Debug_clean
It's supposed to look like this:
Code:
set TARGET=Debug
nmake clean  /NOLOGO
TARGET is an added requirement for the new version of the makefile, which is why you need a new version of the project file, one which actually adds information to TARGET.

Another thing you could have used is the current millisecond count.
I just wrote time but it is actually counted in milliseconds. The code in question is this:
PHP:
std::string CvMainMenuInfo::getLoading() const
{
   // here we need a random number to pick an image.
   // Instead of messing with random seed, just use the number of milliseconds since windows started
   int iRandom = GetTickCount();
   iRandom = iRandom % m_a_szLoading.size();
   return m_a_szLoading[iRandom];
}
It's very simple, yet it gets the job done just fine.
 
You aren't using the included project file. Your file contains
Code:
nmake /NOLOGO Debug_clean
It's supposed to look like this:
Code:
set TARGET=Debug
nmake clean  /NOLOGO
TARGET is an added requirement for the new version of the makefile, which is why you need a new version of the project file, one which actually adds information to TARGET.

If I open the project which came with makefile2.5 (Civ4DLL.vcxproj) or the solution (Civ4DLL.sln) they open up empty:

upload_2019-2-9_17-2-27.png



do I need to run something before?

update: strangely enough Im now able to compile from 2017 with the old 2010 makefile 1.0...

Is it posssible to update that projectfile to use the new makefiel2.5?
 

Attachments

  • upload_2019-2-9_17-2-17.png
    upload_2019-2-9_17-2-17.png
    19.6 KB · Views: 102
Last edited:
I just wrote time but it is actually counted in milliseconds. The code in question is this:
PHP:
std::string CvMainMenuInfo::getLoading() const
{
   // here we need a random number to pick an image.
   // Instead of messing with random seed, just use the number of milliseconds since windows started
   int iRandom = GetTickCount();
   iRandom = iRandom % m_a_szLoading.size();
   return m_a_szLoading[iRandom];
}
It's very simple, yet it gets the job done just fine.
I am too used to C# and there I would use DateTime.Now.Millisecond.
lol.

Either way it's all smoke and mirrors.
 
The RNG could literally just be a piece of code that takes the current second, divides by 60 to normalize it to a [0-1] range and nobody would notice the difference.
You would be surprised. And your precise example would show its limits very quickly during world generation, considering the process only takes a few seconds, as there would be big series of identical draws. Of course, splitting the second in a thousand parts instead of the minute in 60 would be better, and may even be enough if no more than one draw occurs before waiting for user input, but otherwise patterns may appear, because there could be the same delay between two draws. Imagine a game where user input is followed by two draws deciding on victory or defeat (near zero (<0.500s) meaning defeat, near one (>0.500s) meaning victory). Imagine that how the game is coded makes for around 600 ms between the two draws. Consequently, most of the time, it would give victory followed by defeat, or defeat followed by victory. Maybe most people would not care, but still there would be a clear pattern, and depending on the gameplay it could really be game breaking.

Now that I think about it, there is the option of checking if the game is a network game. This means if you write some super random number generator, which then happens to cause network desyncs, then we can write code which will make network games use the vanilla code while single player will use your new code. It seems to me that it's a bit messy to not use the same code in both single player and multi player, but it is an option, which can be used if everything else fails.
It would be sad in particular during map generation, where the game didn't yet really begin. However I don't know, maybe all clients generate the world separately, after having agreed on a seed ? That would pose a challenge to the use of a seedless RNG indeed, since the only workaround I see to this would be for all clients to perform all draws directly on the game host, and that would require some additional network coding.

That being said, I will probably come up with a deterministic RNG such as Xorshift as a start, and only after go undeterministic.

That's pretty much what I did with WTP. There was a request for using a random loading screen background each time the loading screen shows up and I ended up using the time passed since the computer booted modulo the number of images. It feels completely random to the player despite not actually containing anything at random at all.
For that very use, it's likely more than enough, especially considering that the time elapsed before the first loading screen, and between two loading screens, is dependent on user input. But here's the problem, some random generation techniques can apply efficiently to some use cases, but not in general. Hence, better have something that's strong in the general case.

Another thing you could have used is the current millisecond count. You can typically get those strait out of the box with most programming languages with what ever time class they use. And it looks random enough.
I've seen games seeding their RNG with the millisecond count, which basically means for the player they will encounter 1000 different game situations tops, in a situation where even with a deterministic RNG they should have around 2147483647 possibilities at the very least. I wonder what came up their mind, and why the heck they didn't stick to the default that's the tick count since boot...

The reason why pseudorandom generators are used in games really isn't to ensure randomness but to ensure repeatable randomness so that you can have features like Civ's "new random seed on reload" which allows you to permit or block save scumming. You can't have that without a seeded RNG.
One can't really prevent save scumming in some cases, and Civ in particular. Let's suppose you attack that archer with your tank and you lose, even when keeping the same seed you will reload, and change the order in which you move your units so that your archer vs tank battle will be evaluated on a different dice roll.

And btw, I never understood why save scumming has been opposed by some developers, it's a game after all, where's the evil if some people enjoy doing that (I personally don't but I can understand some people do) ? In the case of competitive e-sports things are different of course, but usually iron man mode is the rule there, so it doesn't apply.

I am too used to C#
Me too, because I've mostly been modding Unity games. Not that I would consider myself a C# expert though, actually I started because developers of a game didn't want to implement an easy feature, saying they didn't have the resources for that. I was certain it wasn't that difficult so I tried myself, and it wasn't indeed.

Btw, here's what I mostly use when modding the RNGs in Unity games. In our Civ IV case I would only have to use the integer generation part and the float generation part (the two "Range" functions). The float generation part is the most complicated because I wanted a maximum precision generator, while still being uniform. Most (if not all) floating point RNGs generating in, say, [0,1] will only generate a subset of all the possible values in the interval, many values being unreachable, such as Epsilon (the smallest nonzero value). My RNG keeps all values reachable and can indeed generate all possible floating point values in [0,1], or any given interval, with exceptions though, some viciously crafted intervals being likely to make it freeze, but these are highly unlikely to be used in practice. I may address this in the future however. Oh and please bear with me, there is no exception management at all, I only wanted to replace some game's RNGs and I was lazy about the rest :D

PHP:
using System.Runtime.InteropServices;
using UnityEngine;

namespace NovHak
{
    public sealed class Random
    {
        [DllImport("Rdrexp.dll")]
        private static extern uint rdrget32();

        public static int Next()
        {
            return (int)rdrget32();
        }

        public static void InitState(int seed)
        {
        }

        private static float GenPFRR(float min, float max)
        {
            float maxexp = 1.70141183E+38f;

            while (maxexp >= max)
                maxexp /= 2f;

            if (maxexp == 0f) return 0f;

            float cval;
            do
            {
                uint mantissa = rdrget32() & 0x7FFFFF;
                uint cvali = mantissa + (1u << 23);
                cval = cvali;
                cval /= (1 << 23);
                cval *= maxexp;
                float curexp = maxexp;

                uint coins = rdrget32();
                int rem = 32;
                while (((coins & 1u) == 0u) && (curexp > min))
                {
                    coins >>= 1;
                    rem--;
                    if (rem == 0) { coins = rdrget32(); rem = 32; }
                    cval /= 2f;
                    curexp /= 2f;
                }
            }
            while ((cval < min) || (cval >= max));
            return cval;
        }

        public static float Range(float min, float max)
        {
            bool negisbig = false;
            bool doublesign = false;

            if (min >= max) return min;
            if ((min < 0f) && (max > 0f)) doublesign = true;
            if (-min > max) negisbig = true;

            if (!doublesign && !negisbig) return GenPFRR(min, max);
            if (!doublesign && negisbig) return -GenPFRR(-max, -min);

            float maxpos = max;
            float sign, cval;
            if (negisbig) maxpos = -min;
            uint coins = rdrget32();
            int rem = 32;
            do
            {
                if (rem == 0) { coins = rdrget32(); rem = 32; }
                if ((coins & 1u) == 1u) sign = -1f; else sign = 1f;
                coins >>= 1;
                rem--;
                cval = sign * GenPFRR(0f, maxpos);
            }
            while ((cval <= min) || (cval >= max));

            return cval;
        }

        public static Vector3 insideUnitSphere
        {
            get
            {
                float norm, x, y, z;

                do
                {
                    x = Range(-1f, 1f);
                    y = Range(-1f, 1f);
                    z = Range(-1f, 1f);
                    norm = x * x + y * y + z * z;
                }
                while (norm > 1f);
                Vector3 vect;
                vect.x = x;
                vect.y = y;
                vect.z = z;
                return vect;
            }
        }

        public static Vector3 onUnitSphere
        {
            get
            {
                float norm, x, y, z;

                do
                {
                    x = Range(-1f, 1f);
                    y = Range(-1f, 1f);
                    z = Range(-1f, 1f);
                    norm = x * x + y * y + z * z;
                }
                while ((norm > 1f) || (norm == 0f));
                norm = (float)System.Math.Sqrt(norm);
                Vector3 vect;
                vect.x = x / norm;
                vect.y = y / norm;
                vect.z = z / norm;
                return vect;
            }
        }

        public static Quaternion rotation
        {
            get
            {
                float norm, w, x, y, z;

                do
                {
                    w = Range(-1f, 1f);
                    x = Range(-1f, 1f);
                    y = Range(-1f, 1f);
                    z = Range(-1f, 1f);
                    norm = w * w + x * x + y * y + z * z;
                }
                while ((norm > 1f) || (norm == 0f));
                norm = (float)System.Math.Sqrt(norm);
                Quaternion quat;
                quat.w = w / norm;
                quat.x = x / norm;
                quat.y = y / norm;
                quat.z = z / norm;
                return quat;
            }
        }

        public static int seed
        {
            get
            {
                return 0x7FFFFFFF;
            }

            set
            {
            }
        }

        public static Vector2 insideUnitCircle
        {
            get
            {
                float norm, x, y;
                do
                {
                    x = Range(-1f, 1f);
                    y = Range(-1f, 1f);
                    norm = x * x + y * y;
                }
                while (norm > 1f);
                Vector2 vect;
                vect.x = x;
                vect.y = y;
                return vect;
            }
        }

        public static float value
        {
            get
            {
                return GenPFRR(0f, 1f);
            }
        }

        public static int Range(int min, int max)
        {
            if (min >= max) return min;
            uint diff = (uint)(max - min) - 1u;
            if (diff == 0u) return min;
            uint mask = 1u;
            while ((diff & mask) != diff)
                mask = (mask << 1) | 1u;
            uint cval;
            do
                cval = rdrget32() & mask;
            while (cval > diff);
            return min + (int)cval;
        }
    }
}

And here's the platform-dependent DLL (Rdrexp.dll), that invokes the CPU-integrated, seedless RNG. Most CPUs have it nowadays, especially considering I'm only using RDRAND, which in fact is a pseudo-rng that's frequently reseeded with a true RNG. More recent CPUs even have RDSEED, that can be used instead as a true RNG, but my CPU isn't capable. As you can see the code is fairly simple, however, there are no capability checks or exception management, as I was running this on my computer only.

PHP:
#include <immintrin.h>

extern "C" __declspec(dllexport) unsigned int rdrget32() {
    unsigned int val = 0;

    while (!_rdrand32_step(&val));
    return val;
}
 
Last edited:
You would be surprised. And your precise example would show its limits very quickly during world generation, considering the process only takes a few seconds, as there would be big series of identical draws. Of course, splitting the second in a thousand parts instead of the minute in 60 would be better, and may even be enough if no more than one draw occurs before waiting for user input, but otherwise patterns may appear, because there could be the same delay between two draws. Imagine a game where user input is followed by two draws deciding on victory or defeat (near zero (<0.500s) meaning defeat, near one (>0.500s) meaning victory). Imagine that how the game is coded makes for around 600 ms between the two draws. Consequently, most of the time, it would give victory followed by defeat, or defeat followed by victory. Maybe most people would not care, but still there would be a clear pattern, and depending on the gameplay it could really be game breaking.
Whilst you are technically correct it's telling that you had to pick a simplistic game with a fixed time between turns, only a single type of draw and relatively equal probabilities for individual draw items. That sort of thing is great for showing your idea but it's also not really what most games are. Certainly not CIV.

In any particular game of Civilization you have combat rolls, event rolls and all sorts of other stuff gets rolled for each player or AI many times per turn. And with different stated odds for each roll at that. So even if the player literally keeps drawing the same number 5 times in a row odds are that the stated odds on something like a combat roll are going to do a lot to mask that.

Not to mention that having a fixed turn rate in milliseconds and to the millisecond like that is just completely impossible short of tying your game to your CPU ticks directly like old DOS games some times did. But that's just not done ever any more for among many other reasons because it would run differently on different hardware and die a slow and painful death on any multithreaded system. And with a decent outcome list (like say combat odds witch range from 1 to 100) even small differences are going to be very significant.

Add to that combat animations of non standard length or other similar effects and your game is basically randomizing things for you by making the time between each two rolls different.

I've seen games seeding their RNG with the millisecond count, which basically means for the player they will encounter 1000 different game situations tops, in a situation where even with a deterministic RNG they should have around 2147483647 possibilities at the very least. I wonder what came up their mind, and why the heck they didn't stick to the default that's the tick count since boot...
The actual seed does not matter that much as long as it's different between consecutive draws or as long as you can keep drawing from the same pool without resetting. Which ever is easier to code in your case. And again, you don't really need more than 1k possibilities when you have all sorts of different rolls being done to mask each other and when you have animations and presentations of different lengths to disrupt patterns.

One can't really prevent save scumming in some cases, and Civ in particular. Let's suppose you attack that archer with your tank and you lose, even when keeping the same seed you will reload, and change the order in which you move your units so that your archer vs tank battle will be evaluated on a different dice roll.
Indeed. But presumably the people who would enable that game option are also self disciplined enough not to deliberately attempt and circumvent what they deliberately turned on in the first place.

And btw, I never understood why save scumming has been opposed by some developers, it's a game after all, where's the evil if some people enjoy doing that (I personally don't but I can understand some people do) ? In the case of competitive e-sports things are different of course, but usually iron man mode is the rule there, so it doesn't apply.
Me neither. Outside of competitive multiplayer or some sort of self imposed challenge I see no reason a player should not be allowed to "cheat" if that makes the game more fun. I certainly do it at times. But some people are just really anal about this sort of thing. And we developers have to cater to everyone.

Me too, because I've mostly been modding Unity games. Not that I would consider myself a C# expert though, actually I started because developers of a game didn't want to implement an easy feature, saying they didn't have the resources for that. I was certain it wasn't that difficult so I tried myself, and it wasn't indeed.
You have to give us game developers some slack. There is more to making a game than just making the game. You also have to make it in time and on budget and without making the suits upstairs angry. Good enough is very often better than perfect in that regard.

Btw, here's what I mostly use when modding the RNGs in Unity games. In our Civ IV case I would only have to use the integer generation part and the float generation part (the two "Range" functions). The float generation part is the most complicated because I wanted a maximum precision generator, while still being uniform. Most (if not all) floating point RNGs generating in, say, [0,1] will only generate a subset of all the possible values in the interval, many values being unreachable, such as Epsilon (the smallest nonzero value). My RNG keeps all values reachable and can indeed generate all possible floating point values in [0,1], or any given interval, with exceptions though, some viciously crafted intervals being likely to make it freeze, but these are highly unlikely to be used in practice. I may address this in the future however. Oh and please bear with me, there is no exception management at all, I only wanted to replace some game's RNGs and I was lazy about the rest :D
I would be curious to hear just what sort of exception a random number generator can even generate. I mean, I figure you wouldn't just do something utterly stupid like go over float.max or drive your self into an infinite loop. And beyond that I really don't see what you could do to cause an error.

I'll have to give your code a good read in the morning.
 
It would be sad in particular during map generation, where the game didn't yet really begin. However I don't know, maybe all clients generate the world separately, after having agreed on a seed ? That would pose a challenge to the use of a seedless RNG indeed, since the only workaround I see to this would be for all clients to perform all draws directly on the game host, and that would require some additional network coding.
I think the host will generate a map, save it and then send the savegame to the other players. At least player 0 always starts way faster than the other player(s) even if all players are on the same computer (running multiple instances). The same thing happens when a player joins mid game. The host pause the game, saves, send the savegame and once the joining player has loaded the savegame, the game is unpaused. It's fairly simple and it gets the job done.

If you can make some sort of "map generation only" random function, which in turn creates more diverse maps, then I have to say that I'm very interested.

You shouldn't care for python for the time being. It's fairly trivial to set up the DLL to accept python to call C++ functions, or at least it is trivial once you have tried it a few times.

The reason why pseudorandom generators are used in games really isn't to ensure randomness but to ensure repeatable randomness so that you can have features like Civ's "new random seed on reload" which allows you to permit or block save scumming. You can't have that without a seeded RNG.
I don't really care about savegame scumming. In fact I turn it around and view the ability to keep the same random seed as a debug feature. This means if somebody encounters a bug, the same bug can be reproduced over and over, which is beneficial for both debugging and testing fixes. Yes there are bug reports with savegames saying "hit end turn and the game freezes/crashes".
 
Back
Top Bottom