Massive Humans vs Humans Game/Lets Player Tracker

Who do you think will win?

  • AstralPhaser

    Votes: 5 7.5%
  • Il Principe

    Votes: 2 3.0%
  • Koshling

    Votes: 14 20.9%
  • Hydromancerx

    Votes: 17 25.4%
  • Vokarya

    Votes: 4 6.0%
  • Acularius

    Votes: 3 4.5%
  • Thunderbrd

    Votes: 8 11.9%
  • ls612

    Votes: 5 7.5%
  • JosEPh_II

    Votes: 8 11.9%
  • Praetyre

    Votes: 1 1.5%
  • Epi3b1rD

    Votes: 0 0.0%

  • Total voters
    67
Just for verification purposes, this is the save that I mentioned above. See if Whisper can load her turn from it. I'm guessing she won't be able to, but it would be interesting if she could. It will also give you another file to poke at and compare to the others to see what is wrong.
 

Attachments

  • C2C_Humans_Vs_Humans_BC-6740_to_Whisperrr.CivBeyondSwordSave
    2.3 MB · Views: 35
Just for verification purposes, this is the save that I mentioned above. See if Whisper can load her turn from it. I'm guessing she won't be able to, but it would be interesting if she could. It will also give you another file to poke at and compare to the others to see what is wrong.
It does, at least, seem to relate to the previous round's crash issue. What this means is I'm probably going to have to fundamentally fix some bad data state rather than smooth over past it.

Interestingly enough, I MAY just have to run a debug dll for his turn. There are some unique data state correction routines in the debugger dll that just might take care of the issue.
 
If this is a burden to T-Brd and the others helping him on the issues, I won't complain. But if it isn't, and/or by correcting these bugs the main mod gets improved too, I'd love to continue playing.

I'm not a part of the other game, and I believe the settings you guys chose on the US MP game are far from what I think is balanced, so I'm not so interested in that game if a spot becomes available. So I'd be out of a C2C MP game with you all.

And this game has been a great experience to me. I'll be sad to quit it.
 
I have no problem continuing to debug this. It may help with the main as well. It'll just take me a little bit here. Nowhere near as long as last time.
 
So T-Brd, I've been trying to understand that FFreeListTrashArray.h, and I've stumbled upon this section:

Code:
void FFreeListTrashArray<T>::init(int iNumSlots)
{
    int iCount;
    int iI;

    assert(iNumSlots >= 0);

    // make sure it's binary...
    if ((iNumSlots > 0) && ((iNumSlots - 1) & iNumSlots) != 0)
    {
        // find high bit
        iCount = 0;
        while (iNumSlots != 1)
        {
            iNumSlots >>= 1;
            iCount++;
        }
        iNumSlots = (1 << (iCount + 1));
    }

    assert(((iNumSlots - 1) & iNumSlots) == 0);

(...)

Is this what you meant by binary math?
 
So T-Brd, I've been trying to understand that FFreeListTrashArray.h, and I've stumbled upon this section:

Code:
void FFreeListTrashArray<T>::init(int iNumSlots)
{
    int iCount;
    int iI;

    assert(iNumSlots >= 0);

    // make sure it's binary...
    if ((iNumSlots > 0) && ((iNumSlots - 1) & iNumSlots) != 0)
    {
        // find high bit
        iCount = 0;
        while (iNumSlots != 1)
        {
            iNumSlots >>= 1;
            iCount++;
        }
        iNumSlots = (1 << (iCount + 1));
    }

    assert(((iNumSlots - 1) & iNumSlots) == 0);

(...)

Is this what you meant by binary math?
Yep. iNumSlots and all the operators being used to establish its initial value and manipulate it thereafter are extremely mysterious to me in how it all works.
 
All the 'make sure its binary' and related code is doing is making sure the num slots it winds up with is a power of 2 at least large enough to contain the originally requested size. Frankly its a rather obscure way of doing it!

Edit - actually its not only obscure, it's broken, since the result is not guaranteed to be a power of 2 (which I'm fairly sure the rest of the free list code relies on). If the value initially specified passes the first test it will be directly used, and some values other than powers of 2 (any pattern where the lowest set bit is separated by more than one place from the next lowest set bit I think) will pass.

Whether this matters depends on what value is being passed into that initialize function. Just running the code inside the 'if' unconditionally would probably have been more sensible. However, I'm fairly sure that once a numSlots is decided upon for a game it cannot subsequently be changed (because changing it would make all ids allocated in that pool invalid) without re-allocating all ids.
 
Last edited:
All the 'make sure its binary' and related code is doing is making sure the num slots it winds up with is a power of 2 at least large enough to contain the originally requested size. Frankly its a rather obscure way of doing it!

Edit - actually its not only obscure, it's broken, since the result is not guaranteed to be a power of 2 (which I'm fairly sure the rest of the free list code relies on). If the value initially specified passes the first test it will be directly used, and some values other than powers of 2 (any pattern where the lowest set bit is separated by more than one place from the next lowest set bit I think) will pass.

Whether this matters depends on what value is being passed into that initialize function. Just running the code inside the 'if' unconditionally would probably have been more sensible. However, I'm fairly sure that once a numSlots is decided upon for a game it cannot subsequently be changed (because changing it would make all ids allocated in that pool invalid) without re-allocating all ids.
All of this kinda vaguelly makes sense. That it's obscure AND broken is what makes the most sense of all. I'm not sure what slots we're allocating for exactly nor why anything at all has to be a power of 2 or what a set bit exactly is but it sounds like you have a much deeper grasp of the subject matter as a whole and what little real information I can glean from what you've said here seems to correlate to what I'm finding... that there is an extremely odd mathematical bug in an extremely mysterious mathematical system at the point where units are initialized and I've begun to wonder if it can be fixed at all.

Tracking this particular save, I've found that a unit, an animal barbarian, has been initialized. But if
template <class T>
T* FFreeListTrashArray<T>::getAt(int iID) const
is ever called for that unit, the response is always a NULL unit.

The reason for that is it doesn't pass this test:
Code:
    if ((iIndex <= m_iLastIndex) &&
       (m_pArray[iIndex].pData != NULL))
   {
       if (((iID & FLTA_ID_MASK) == 0) || (m_pArray[iIndex].pData->getID() == iID))
       {
           result = m_pArray[iIndex].pData;
       }
   }
iIndex is somewhere in the 4k range where m_iLastIndex is in the 600s. m_pArray[iIndex].pData is also NULL.

So something went horribly wrong during initialization and I suspect it has something to do with the last 'fix'. Perhaps what I did to get us past the last issue is causing horrible problems now. Not manifesting in the main mod or the other game because it was such a rare corruption bug in the first place and hardly ever needs to process through the 'fixed region'?

I don't know but I'm pulling my hair out again and running smack back into the fundamental problem that I could only understand 25% of what you just said and that means I'm understanding 5% of what this binary math system is trying to accomplish and why.
 
I'm sorry I couldn't reply earlier to this but I've been really busy.

This binary math is part of a subject I had in college not long ago. So I may be of help here indeed (I hope at least).


About that if statement, it's as Koshling said, but I can't see it being broken.

First it avoids negative numbers with iNumSlots > 0, which is good for its '&' bitwise operation later (2's complement notation for negative integers could potentially make the bitwise '&' operator run erroneously).

Then it guarantees (iNumSlots & (iNumSlots - 1)) is different from 0. This means only a number which isn't a power of 2 will trigger this 'if', because the only case where a [number & its predecessor] will yield 0, is when that number is a power of 2.

Eg.: The number 8

01000
00111 &
00000 = 0

Spoiler :
The & operator compares each number bit to bit (by columns), and yields 1 to each pair of bits if both are 1s, or yields 0 otherwise


Eg.: The number 14

01110
01101 &
01100 = 12

With some examples it becomes clear that only a power of 2 would yield 0 in this situation.

Then, when it enters the 'if', the only thing it does is finding the highest bit from the binary representation of iNumSlots and then it rounds up the iNumSlots to the next power of 2.

Spoiler :
The '>>' and '<<' operators shift the bits in a binary number, so if we do iNumSlots <<= 5, and iNumSlots was equal to 1, then it now is 100000 in binary, which is 32


Eg.: The number 14 again

If we had chosen 14 as iNumSlots then the if statement would be triggered and iNumSlots would become 16, the next closest power of 2


What I find most curious is this just after the if statement:
Code:
assert(((iNumSlots - 1) & iNumSlots) == 0);
assert((m_iNumSlots <= FLTA_MAX_BUCKETS) && "FFreeListTrashArray<T>::init() size too large");

uninit();

m_iNumSlots = iNumSlots;

The first assert is ok, it's exactly the one which guarantees iNumSlots is a power of 2, but the second one seems wrong, as it now compares another variable m_iNumSlots to the Max possible size (FLTA_MAX_BUCKETS) instead of iNumSlots, and then it makes m_iNumSlots get the value of iNumSlots. To me it's possible that here a number which is higher then FLTA_MAX_BUCKETS can enter as m_iNumSlots for the rest of the function.

I've been reading the rest of the file but now I need to sleep, so I hope this was helpful and I'm eager to continue discovering what is happening in these functions.
 
Ok, maybe this could help: What does the variable iNumSlots represent? What, exactly, are the slots we're taking a count of?

What really messed me up when I was looking into this was;
The '>>' and '<<' operators shift the bits in a binary number, so if we do iNumSlots <<= 5, and iNumSlots was equal to 1, then it now is 100000 in binary, which is 32
What on Earth could be possibly useful for this kind of function except to scramble numbers about? Why are we dodging the use of normal math here?
 
Last edited:
I believe the reason why binary math is preferable is its speed. At least that's what I've read about it, I don't know the proof. But it seems using decimal mathematical instructions is slower then binary operations.


About iNumSlots, it seems to be the initial value that an external function defines in its call. This initial value has to be a power of 2, so the first if statement checks that and change its value to a power of 2 if it isn't already. Its purpose seem to be the total slots we have to store data. Each node has a pointer to a generic T type of data, and other stuff.

As we have a growArray function, which is called inside the add function, I believe here we have a reason to why iNumSlots has to be a power of 2.
FLTA_MAX_BUCKETS is a power of 2;
FLTA_GROWTH_FACTOR is 2;
As each time we grow iNumSlots through growArray() we multiply its value by 2 (FLTA_GROWTH_FACTOR), we assure that FLTA_MAX_BUCKETS will be a viable value to be used, and not stepped over because of a growth progression which happens to let that happen.

I don't know who calls the functions in this file, so that's as far as I could understand about this iNumSlots.


The other uses of the '&' operator through the file that I've already seen just seem to be the use of a Mask to avoid indexes higher or equal to the maximum allowed by FLTA_MAX_BUCKETS
 
Last edited:
I believe the reason why binary math is preferable is its speed. At least that's what I've read about it, I don't know the proof. But it seems using decimal mathematical instructions is slower then binary operations.


About iNumSlots, it seems to be the initial value that an external function defines in its call. This initial value has to be a power of 2, so the first if statement checks that and change its value to a power of 2 if it isn't already. Its purpose seem to be the total slots we have to store data. Each node has a pointer to a generic T type of data, and other stuff.

As we have a growArray function, which is called inside the add function, I believe here we have a reason to why iNumSlots has to be a power of 2.
FLTA_MAX_BUCKETS is a power of 2;
FLTA_GROWTH_FACTOR is 2;
As each time we grow iNumSlots through growArray() we multiply its value by 2 (FLTA_GROWTH_FACTOR), we assure that FLTA_MAX_BUCKETS will be a viable value to be used, and not stepped over because of a growth progression which happens to let that happen.

I don't know who calls the functions in this file, so that's as far as I could understand about this iNumSlots.


The other uses of the '&' operator through the file that I've already seen just seem to be the use of a Mask to avoid indexes higher or equal to the maximum allowed by FLTA_MAX_BUCKETS
This is kinda making sense then. Ok, so then if you look at the add function, how must we be initializing a unit with an index higher than the max buckets and how can we avoid doing so?
 
The Max Buckets is 8192 (2^13)

Code:
#define FLTA_ID_SHIFT                (13)
#define FLTA_MAX_BUCKETS        (1 << FLTA_ID_SHIFT)

So I don't believe the problem is in iNumSlots being higher then the Max Buckets, but rather it only being higher then m_iLastIndex. The problem here is that I couldn't find a single line which changes the value of m_iLastIndex that isn't to FFreeList::INVALID_INDEX, with the exception of the add function which has m_iLastIndex--

I don't know the value of FFreeList::INVALID_INDEX, and I haven't touched anything besides FFreeListTrashArray.h
 
Oh sorry, the add function does have m_iLastIndex++ too. But even so, m_iLastIndex must have been initialized with another value somewhere else (not in this file), or in the add function it grows from -1 by the increment line until it reaches a proper value. The add function is still pretty mysterious to me
 
Last edited:
Thanks for the value of the invalid index T-Brd, with that I could follow the tracks in the add function and now I can see how it starts. With the initial value of -1, it suffers an increase to 0, and then Slot[0] is allocated and sent through the return value. This means that it's completely ok to m_iLastIndex to start in -1 (Invalid Index), and it is intended to, as when it's valued -1, Slot[0] is allocated and a pointer to its data is sent to the caller.

By running a few times it becomes clear that if we started with an empty list and have only used the add function, we will never trigger most of the conditional chains inside the function, as m_iFreeListCount is never* updated in the add function, only outside of the function, and it starts with value 0, so:
Code:
if (m_iFreeListCount > 0)
will never trigger.

Spoiler :
* It is actually updated inside of the conditional, but by starting with an empty list we are bound to begin not being able to enter the condition as m_iFreeListCount starts in 0, so without another function to change m_iFreeListCount, we would never be able to enter this conditional if we started with an empty list


And this seems to me that those other 2 variables of FFreeListTrashArrayNode (iLastUsed and iNextFreeIndex) serve the purpose of creating a secondary list, chained through the nodes of the primary list (that we have been analyzing so far), to keep track of the Freed Nodes that pop through the primary list by the remove functions

Now I'll try to understand how it uses this secondary list.
 
Top Bottom