Border and Area plot iterators

[whoward69]

So if I understand properly you are using coroutine to keep the function alive so that it's values do not disappear. Very interesting.

Yes. Co-routines are a way to do co-operative multitasking - see http://lua-users.org/wiki/CoroutinesTutorial

 

[General Tso]

whoward69 is there any problem with having a large number of your PlotAreaSpiralIterator functions open at the same time? I not certain if there is some type of limit due to the use of coroutines or perhaps something else that I'm not aware of.

I want to do the following: (I'm not using real code syntax to keep it simple. I also omitted all of the checks for proper plot type etc.)

for each player
   build unitList[player]

   coastUnitIterator[player] = PlotAreaSpiralIterator(blah, blah, blah)
   seaUnitIterator[player] = PlotAreaSpiralIterator(blah, blah, blah)
   nonCombatUnitIterator[player] = PlotAreaSpiralIterator(blah, blah, blah)
   combatUnitIterator[player] = PlotAreaSpiralIterator(blah, blah, blah)
end

while UnitsStillNeedPlaced
   for each player
      unit = GetNextUnitFromUnitList

      if unit is coastal unit then
         _, plot = coastUnitIterator[player]()
      elseif unit is sea then
         _, plot = seaUnitIterator[player]()
      elseif unit is noncombat then
          _, plot = nonCombatUnitIterator[player]()
      elseif
          _, plot = combatUnitIterator[player]()
      end

      PlaceUnit(unit, plot)
   end
end

So basically the worst case scenario (with a standard dll) would be 22 * 4 or 88 functions open at the same time. Would this cause any problems? I already started on the code then it occurred to me that there might be a problem with coroutines and threads and those type of things that might only show up on certain computers. I can't test something like that on my own computer so I thought I'd better ask you about it.

 

[DonQuiche]

There is no reason for that. Coroutines are just functions whose execution state is stored when you yield, and restored when you resume.

Especially, they do not involve system threads in any way: while the Lua manual describes them as a way to achieve cooperative concurrency, it is technically true and it is a good analogy for people familiar with the implementation of system threads by the OS at the CPU level, yet it is a very bad way to introduce them for the typical Lua developer and a quite uncommon usage. Coroutines are actually just a way to suspend a function call at predefined points with the ability to resume later after those points. Most of the time they are used as a syntactic sugar to write a process to be completed through many successive calls (iterative flow) under the form of a single function call (continuous flow). That is, rather than splitting your process into many functions and/or being forced to have state objects in the parameters, you may just write it naturally as a function to be ran once, making it more understandable.

The only limit with coroutines is that, for some reason, I was unable to resume one from some UI events. I guess it is a bug in the civ5 lua machine, whether introduced by Firaxis or present in the standard 5.1 VM.

 

[General Tso]

Thank you. That answers my question.

My code is executed within an event (Events.SequenceGameInitComplete) but everything is completed before exiting that event so hopefully there will be no problems.

 

[General Tso]

It looks like I might be having the same problem. It works for the most part but eventually instead of returning a plot or nil it returns a string that says "cannot resume dead coroutine". I have to do some more testing. If it's just returning that string instead of nil when the end of the list is reached I can make it work. If it's stopping before it's reaching the end of the list then I have a problem. Since it has never returned nil I'm hoping it's the former.

 

[General Tso]

After doing some testing I found that the iterator appears to work fine except that it returns the string "cannot resume dead coroutine" instead of nil when the end of the list is reached. Is this normal behavior?

Here is a simplified segment of the code that I'm using:

unitIterator = PlotAreaSpiralIterator(startPlot, 0, maxDistance, SECTOR_NORTH, DIRECTION_CLOCKWISE, DIRECTION_OUTWARDS);

function GetValidPlot()
   for plot in unitIterator do
      if type(plot) == "string" then
         break;
      end
			
      if IsValidLandPlot(player, plot, placementType) then
         return plot
      end
   end

   return nil;
end

I had to add the type(plot) == "string" to make it work. If I understand things correctly the iterator normally returns nil when it's reaches the end which causes the for loop to end. For some reason when I use it like this it returns the string. Am I doing something wrong?

 

[whoward69]

Is there any problem with having a large number of your PlotAreaSpiralIterator functions open at the same time?

Wot DonQuiche said


The only limitation I've found is that because co-routines, exceptions and pcall() all use the setjmp/longjmp methods you cannot interleave them - see the discussion here http://forums.civfanatics.com/showthread.php?t=485607 - and "cannot resume dead coroutine" is what you will (typically) see if you do. So if you're seeing that I'm guessing something in the Firaxis code is breaking the stack state.

My own code keeps the iterators "open" over many turns and I've not run into issues, but then I'm not calling any Lua methods which would generate events from inside them (eg InitUnit, MoveUnit, etc). Try wrapping the code within the loop with a pcall() to see if something inside the loop is mis-behaving.

 

[DonQuiche]

Maybe the problem is much simpler.

The variable "unitIterator" is initialized when the file is read, probably at the game's initialization. Then someone calls GetValidPlot and exhausts the iterator, leaving him dead. Finally someone tries to invoke it again and encounters an error.

The workaround would be to initialize unitIterator anytime the GetValidPlot function is invoked.

 

[General Tso]

I'm sorry, I over simplified things when I gave my example - in my code the iterator is created within a function. The problem is when the iterator is exhausted it normally returns nil which ends the for loop. In this situation it returns a string that consists of "cannot resume dead coroutine". I'm checking the result each time the iterator is called. It's plot, plot, plot, ... and ends with a string. I can make it work by adding the extra check for a string then breaking the loop but it bothers me that I don't understand what's happening. Plus there might be an some kind of problem that I'm not aware of.

 

[whoward69]

Are you sure you're not getting
plot, plot, plot, ... plot, nil, "cannot resume dead coroutine"

Other than that, you'll need to post the full code

 

[General Tso]

My mod is attached. All of the code discussed here is located in Lua\GTAS_PlaceNonCityUnits.lua. I included the entire mod in case you want to try printing some results. The PlaceNonCityUnits function creates a unitData table then cycles through it placing the units one at a time for each player using PlacePlayerUnit. PlacePlayerUnit calls GetValidPlot to get the next plot. The calls to the iterators are located in GetValidPlot. The print statements in that code show plot, plot, plot, ...., plot, string.

If you decide to try running it:
You will need to give at least one player enough units so that the limit of plot iterator is reached before all of the units are placed. Since the current maximum radius is set to 4 giving a player 70 or so warriors should guarantee that the limit is reached for any map. You can add the warriors 10 at a time after clicking on a player's edit button by selecting warrior and increasing the count to 10. You can access the mod by enabling it and then opening the Advanced Setup screen.

Thank you for offering to take a look.

 

[whoward69]

Wot DonQuiche said!

Basically you are using the iterators after they have returned nil

The iterators are setup ONCE - in BuildPlayerData() via PlaceNonCityUnits() from OnSequenceGameInitComplete()

You have a loop within a loop that places one unit for each player in turn, until there are no more units to place.

Let's assume THREE of those units for Player 0 are ships, but that there is only ONE valid sea tile for Player 0

PlacePlayerUnit(playerData) will be called three times, once for each ship, which in turn will call GetValidPlot(UNIT_SEA, playerData) for each ship

GetValidPlot() will therefore execute the following three times

  for plot in playerData.seaUnitIterator do
    if IsValidSeaPlot(plot, placementType, playerData) then
      return plot
    end
  end

The first ship unit will loop until it finds the first (and only) valid sea plot
The second ship unit will loop (from the next plot) until it exhausts the iterator and then return nil
The third ship unit will then attempt to loop, but at this point the iterator is dead and you will get the error message

Basically you have no check for dropping off the end of the for loop and then flagging all future calls as having to return nil (except for the error checking inside the loop which is fixing the symptom not the problem)

The quickest fix for your code is to nil out the iterator when it is exhausted

if playerData.seaUnitIterator then
  for plot in playerData.seaUnitIterator do
    if IsValidSeaPlot(plot, placementType, playerData) then
      return plot
    end
  end
  
  -- Iterator is exhausted, so don't use it again
  playerData.seaUnitIterator = nil
end

You will need to do this for each of the iterators used

 

[General Tso]

OK now I get it.

Thank you for your patience whoward69 and DonQuiche. Sometimes it requires the equivalent of getting whopped upside the head by a 2x4 but sooner or later I catch on.

 

[whoward69]

Can you try something for me please

Use the attached GTAS_PlotIterators.lua and GTAS_PlaceNonCityUnits.lua code and see if the iterators still throw the error message(s)

 

[General Tso]

I'll be glad to try them. But I don't see any attached files.

 

[whoward69]

I'll be glad to try them. But I don't see any attached files.

Sassafrassarassum Forum Software! removed the .lua files - now zipped

 

[General Tso]

Doggone it. Still no luck.

Edit: Never mind. I see that you attached them above. I'll try them now.

 

[General Tso]

Those files work fine! No errors and all of the print statements say "table". Thanks.

If I may ask. What did you change?

 

[whoward69]

In "your" file, I just removed the checks you added, in "my" file the iterators now return "plot, plot, ... plot, nil, nil, nil, ..." as opposed to "plot, plot, ... plot, nil, error"

So basically I made the iterators more robust (seach for "success" in that file to see the changes)

 

[General Tso]

I was looking at that earlier and was somewhat confused - probably because I'm not totally familiar with coroutines. I saw this part:

    local success, pAreaPlot = coroutine.resume(next)
    return success and pAreaPlot or nil

And I figured that it was part of the fix. But I don't understand how coroutine.yield(pEdgePlot) etc. returns "success, pAreaPlot". If it's going to take a long explination - don't worry about. The important part is that it works.