CivEffects

[Nightinggale]

I commited a major update to the tech advisor. The tech screen can now be displayed without python errors. However it's not drawn as intended. More precisely requirements aren't drawn yet, nor are the arrows. While I know what to do about those issues, there is a bigger issue, which is not all tech boxes are drawn and right now I have no idea why. Maybe the problem solves itself when all cases of canDoCivic() and similar are updated.

I added a check to ensure only techs are drawn. Without that check, perks are drawn too, but the XML (X,Y) values made them appear on top of the techs (which spoils the view for both types). This mean perks are drawn as intended and that the code can handle both perks and techs in the same tree just fine.

Back when I added PlotGroups, I copied some python exposure code from BTS without fully knowing why vanilla has it and then I forgot about it. Now I ran into big problems and after investigating the problem (as well as some ), I figured out not only how to make a proper python exposure, but also how and why it works. This mean I now know how to add both C++ classes and enums to python, complete with functions and everything, which sort of make python way more powerful (though still slower than pure C++). I think I better start writing some modder's programing guide or something for the next time we need this.

I added a python class for InfoArray. This mean the code written for C++ can be exposed to python using only a single line of code. That should make things easier inside the DLL. I wonder if I should do the same with JIT arrays, though the need is not as great as with InfoArrays.

 

[Nightinggale]

I have come to the conclusion, that error checking in python is too much of a hassle. Instead I will add asserts when reading XML. If there is an assert and somebody decides to ignore it, then the tech tree might fail completely due to lack of checks. However it should work as long as there aren't any asserts at startup.

I'm about to get a love-hate relationship with CivEffects. I think it's awesome, but it sure is a lot of hard work to finish and to be honest the GUI stuff isn't that interesting for me. Also the whole tech tree python file needs cleanup. However for now I'm happy with just getting it to work again.

 

[Nightinggale]

I have somewhat given up on def update() in the tech tree for now. Sometimes the index is iData1 and sometimes it's iData2 and it appears to be quite random. Also with no comments and checks like if iData2 == self.AMENDMENT_BUTTON + 2:, it will take quite a while to figure out what it is really supposed to do

At some point this part of the code will need a major cleanup. We should at least use some sort of enum to tell what the difference between self.AMENDMENT_BUTTON, self.AMENDMENT_BUTTON + 1 and self.AMENDMENT_BUTTON + 2 is supposed to be.

For now we will just have to live with the fact that help might miss text or be completely blank. It would be better if I spend my time fixing more serious stuff, like the python error, which happens when you click on a tech.

I have good news as well. Using my new gained knowledge of python classes, I have added the function parseHelp to CivEffects. This mean if we have a pointer to either a TechInfo or PerkInfo, we can just use parseHelp on it and the DLL will figure out which type it is and call the correct function to generate the text. Even better, if getting a ResearchCivEffects from the DLL can result in 3 types in the future, just adding parseHelp to the new one in the DLL and the python code will use it without being modified. We should expand on this concept and make python code, which is less likely to need updating when the DLL updates.

 

[Nightinggale]

Time to show some progress. I updated the pedia files to display CivEffects. There is not supposed to be a visual difference, but I changed the function calls to match the fact that each type has a class of its own. This turned out to be surprisingly easy, but I guess that is because I already studied how to handle the python interface as well as a well designed "CivEffect engine".

Meanwhile Kailric has started to fill in the help text for CivEffects. This text is used by both help popup and the pedia pages. This mean the pedia pages are now rapidly being filled with XML data. At long last is the CivEffects are starting to work in the GUI and produce something visually.

 

[Nightinggale]

After a lot of hard work, I finally managed to get the tech tree to look ok. That is, it looks like before I started working on it

I suspect that I might have made a tiny mistake though. Right now you have to add all arrows manually in XML. That made me think: would it be better if all arrows are added by default and then the list in XML tells which arrows, which should NOT be drawn? That way we can just add the requirements in XML and then generate the list of arrows on load and store the result in a bool array. Then the DontDrawArrow can be read and clear the array data accordingly. Finally that boolArray can be used to generate an InfoArray. The python code I have written now just relies on getting an InfoArray and replacing it with a different one is a one line change.

What do you say? Should we write the arrows, which should be drawn or the arrows, which should not be drawn?

 

[Trade Winds]

For an XML modder as me, it would be better automatic arrows. You just declare <required tech> and <required tech and> and then you get the whole thing drawn.

What I ignore is if this automatic drawing may lead to bigger limitations for modders' freedom.

 

[Nightinggale]

For an XML modder as me, it would be better automatic arrows. You just declare <required tech> and <required tech and> and then you get the whole thing drawn.

What I ignore is if this automatic drawing may lead to bigger limitations for modders' freedom.

The problem is that I changed requirements to be lists. As a result, there could be 5 arrows to each tech and it would look really confusing. I proposed just drawing everything, but that was not accepted and then I came up with the idea to be able to control which arrows to draw.

Since we will draw some, but not all, the real question is: should the modder select which to draw or which of the requirements, which should not be drawn?

 

[Lib.Spi't]

could it be tied to something like a globaldefine option, that sets the bool default to be on or off.

then the modder can choose whether they want to change the 'ons' or the 'offs' of arrow choice.

Or would this in some way massively increase performance issues?

So what I mean is:

Global define xml tag.

<bTreeArrowsOn>

Tree xml tag

<bTreeArrowOn>

If <bTreeArrowOn> is not present in the entry then it assumes the <bTreeArrowsOn> setting as default (So all on all off switch)

Then you include <bTreeArrowOn> in the tree/tech xml file in each entry that you want to go against the default.

So you have two choices, whether you want arrows to be all on or all off, then you can decide specifically which tech arrows to have.

This way if a modder wants a 'sparse' tree they don't have to go through turning 90% off, they can always choose the shorter side to work from, all on or all off.

 

[Nightinggale]

This way if a modder wants a 'sparse' tree they don't have to go through turning 90% off, they can always choose the shorter side to work from, all on or all off.

That's a good point. We will go with that approach.

 

[Nightinggale]

I think I finished the tech tree screen, at least for now.

I ended up adding two lists for arrows: RequiredCivEffectArrows and RequiredCivEffectArrowsRemove.

Arrows are then assigned using 3 steps during (actually right after) XML reading.

1. If RequiredCivEffectArrows is empty, fill it with all requirements.
2. remove any arrows mentioned in RequiredCivEffectArrowsRemove
3. remove all civeffects mentioned in RequiredCivEffectArrows from requirements

Last step is to generate a list for "also requires".

The result is two lists, one to add and one to remove arrows. Modders can then use the one, which makes the most sense for each CivEffect.

The only case where I can think of a scenario where it makes sense to use both is if there are say 4 requirements and the modder don't want any arrows. That can be done by assigning one civeffect to both lists.

I added code to list "also requires" techs in the GUI. If the DLL is a debug build, then after the requirements it will print "Arrows:" and then list the CivEffects with arrows. I actually planned for this to appear in assert builds as well (that's what XML modders should be using), but the DLL code, which allows python to detect that is in the trade screen branch. I will correct this after they are merged instead of duplicating code. Getting a list of arrows is surprisingly useful when setting up XML. It could be tricky to track some of them when there are too many and then the question is what to disable

I'm pleased with the result. The screen appears to be working well and I added support for network games in the DLL interaction (network support is coming, one feature at a time). This made updating a bit more tricky when changing what is being researched, but adding dirty bits allows the DLL to force a redraw. This turned out to be too slow and I split the screen into new research and full redraw. The new research appears to be instant, which makes the screen really responsive. It freezes for half a second or something if you gain a tech while looking at the screen (not 100% sure how that can happen, but the screen supports this in case it happens at some point).

The screen still look horrible at certain resolutions and the arrows appear to have some offset issue. That is just like it was before I started and I will give higher priority to getting CivEffects working and making a release than fixing this. After all the graphical glitches haven't stopped releases so far and it's fully playable... except for resolutions smaller than 1024x768. 1280x720 actually prints the progress bar outside of the screen. The same goes for exit. It's a bigger issue than it sounds like because that's the default resolution in window mode.

 

[Nightinggale]

I started adding the code for gifts. I have been holding it off until now because... well it didn't seem critical compared to not being able to pick what to research and such. Now that I'm nearly done, I start to look at this part of the code. The code matured somewhat since this part was written and I have decided to recode it, at least some parts of it.

I just (re)added the ability to gain free units and I decided to make some changes. Both first to discover and free for all stores free units in InfoArrays. In fact they are identical because that allows them to share activation code.
XML:

<FreeUnitClassesFirst>
	<FreeUnitClass>
		<UnitClass>UNITCLASS_WAR_GALLEY</UnitClass>
		<iNumFree>1</iNumFree>
	</FreeUnitClass>
</FreeUnitClassesFirst>

Both can have any number of FreeUnitClass and each will add iNumFree of the unit class in question. That should be fairly strait forward.

When adding, units are placed using this system:

1. valid owned city
2. first city, closest valid plot
3. starting plot, closest valid plot

This will presumably place most units in the first city. However ships will look for a city with access to the ocean and then they will just be near you if none is found. I don't think the starting plot will ever be used, but since there is a theoretical chance it could happen and the code will crash if it lacks a plot to work from, I added it to avoid crashes.

Next up is conversion from one unit class to another. I want to make that into an InfoArray as well. It will look better and it will allow adding more than one conversion rule to a CivEfffect. It might be useful at some point and I have put so much work into CivEffects that I refuse to slack off now and finish it with less flexible code than the rest.

The rest of the gift tags appears to be a whole lot easier and are unlikely to require recoding as they are all ints, like gaining gold or immigrants.


There is one thing I wonder about though. What should we do once CivEffects is done? Should we go for it and add more classes right away? The plan is Eras and Traits. Anything else, which might make sense to add? I haven't thought much about it because it was so far into the future, but the future appears to be now, possibly this week.

 

[Kailric]

There is one thing I wonder about though. What should we do once CivEffects is done? Should we go for it and add more classes right away? The plan is Eras and Traits. Anything else, which might make sense to add? I haven't thought much about it because it was so far into the future, but the future appears to be now, possibly this week.

What are all our options here? I would like to get the current stuff to work with 2071, or what would be the best solution?

 

[Nightinggale]

What are all our options here? I would like to get the current stuff to work with 2071, or what would be the best solution?

There are two options:

1: add Eras and Traits to CivEffects. It will be much faster because the engine is there, but since nothing has ever been added to the engine before, I can't say how much faster.

2: not add anything else to CivEffects. Traits and Eras will continue to function as they did in the last release. That is, unless I broke something. I'm not 100% sure everything still works.

Somehow I wonder if option 2 is the best one, but it mean that at some point we will likely get a new trait XML file. That may or may not be a problem. The same goes for eras, though adding Eras as CivEffects should be fairly easy from an XML point of view as all what is needed is to copy paste the empty taggroups into the existing XML file. It has to be done for each era, but still, it can be done. I made a script to add missing tags and it should be able to do it automatically.

 

[Nightinggale]

After finishing the tech screen, it suddenly occurred to me that the auto hiding of arrows could be improved a lot by a simple check. Now all arrows are hidden unless it goes from left to right. Also I removed some arrows drawn due to perk locations (the perks are hidden themselves) and now the autogenerated display of arrows look pretty decent. Not perfect, but now it only needs minor adjustments in XML.

Another issue is that tracking the long arrows could be a bit tricky. I added to debug DLLs, that if you hold down control while hovering the mouse over the arrow, a help popup will display the names of both ends of the arrow. Checking the control condition is a bit tricky though and as it turns out, the key needs to be held down before entering the arrow for this to work. I also added some help code to the tech box itself, which is turned on and off by control, but for some reason it will not listen to the keyboard. However since the arrow part works, it can display the info we want to display.

 

[Nightinggale]

I tried to make an XML friendly approach to discovering the spice route by entering a plot for it. I added EventCivEffects, which is a CivEffect, which will not show up in pedia. Instead it is something, which can be triggered by random events and whatever else needs to trigger a CivEffect.

I did this because currently the only way to store if a trade screen is accessible is by enabling it with a CivEffect. This mean I added a new XML file, currently with just a single item and the only effect of this item is to enable the spice route.

EuropeInfo is extended by keeping an EventCivEffect. Any player moving on to an access plot will automatically gain the civeffect in question.

This mean XML allows discovering multiple routes simply by moving on to the plots. However it's not quite working as intended

I will now redesign it a bit and try to reuse more of Kailrics code. However that makes the code less efficient and slower. Since the code is triggered whenever a unit is placed on a new plot, the checks better be really fast. Sadly I only see one solution, which I'm not completely unhappy about.

Proposal:
Hardcoding EuropeInfo in the DLL, just like yields. Either it gains an "EuropeGroup" like the yield groups and/or we hardcode the first and last Europe with events on plot entering and if they are all placed in one block. This approach will allow a quick loop and in most cases can be done using only simple get functions from CvPlot.

Writing this just gave me a new idea. Add a player bool array to each plot. In this array, cache if an event will trigger if the player enters the plot. This way the calculation for if something takes place is done on game start/load and when a player gains a new trade screen. In other words the checks at runtime when a unit move is as simple as checking if the plot has been discovered. This will likely be even faster than a hardcoded loop and will not require hardcoding. Good thing I started writing, which forced me to think about the problem in a new way

Another problem:
Currently the player has arrays to tell if the player has a civeffect or not. It works quite simple in the way that != 0 mean the player has the CivEffect. During doTurn, if the score is positive, it is reduced by 1. This allows for timed CivEffects as assigning an EventCivEffect to 4 will provide the effect for 4 turns. Assigning -1 will avoid the doTurn code and it is stuck with this value.

The problem is that permanently enabling a CivEffect is done by assigning the score -1. It's not really intuitive and since a bunch of other places requires +1 to enable, I predict a bunch of bugs due to this. Do you think it would be better to set 1 to permanently enable and -X to work for X turns? That is, to make an event last 4 turns, it should be assigned -4.

 

[Lib.Spi't]

I think -1 is better for permanent enable, as -1 is often used as 'unlimited'

 

[Kailric]

I think -1 is better for permanent enable, as -1 is often used as 'unlimited'

I kinda agree with this, if you want to set a turn limit you set a value, if not set it to -1 for infinite. Either way could introduce a bug because if a modder forgets or doesn't know to set this to a negative number then he will have issues.

 

[Nightinggale]

if a modder forgets or doesn't know ... then he will have issues.

All modders have issues Why else would we make such a masterpiece and then give it away for free?

Maybe we should make some naming convention to reduce the risk of a mistake. Something like iTurnDuration or iTurnDurationWithInfinitive. Perhaps the last one is good because we can use that one for -1 = permanent and positive for duration and make that a general rule. We should write such naming conventions somewhere and we should also look at that "somewhere" when assigning new names.

I'm slowly, yet steadily changing types in arguments to enum types as this will avoid a number of bugs and potiential modder mistakes. However that will not work in this case.

 

[Lib.Spi't]

All modders have issues Why else would we make such a masterpiece and then give it away for free?

Copyright Infringement...

I don't think it needs the 'withinfinite' part, most of the civ tags have the minus 1 as a preset anyway, so unless we go back and ID all of the tags 'with infinites' that extra bit is just going to inject more confusion I think..

 

[Nightinggale]

I finished adding XML for controlling trade screen discovery by entering an access plot. I didn't add any cache, but the functionality is there and the looping only takes place for units moving on plots with access to at least one trade screen. It might not be as bad as I first feared. it depends on how often the AI decides to move around in access plots.

That crossed out some more in the todo list. Now everything left on the list are actually new features, not something, which is needed to reproduce a game similar to the other branches (with the exception of pope not starting with techs). This mean the time has come to playtest to see if it works. It might take a little while before it ends up as a test build as I will play myself first and fix the easy to encounter asserts (there are 3 known ones at game startup/load).

Also help text for CivEffects is incomplete (a number of tags is ignored in GUI) and the tech tree requirements are still wrong. None of that prevents hunting for DLL bugs though.