CivEffects

[Nightinggale]

This should be reserved for a update by itselft probably?

You are right. This grow to become way bigger than what we should include in CivEffects. It would be better to simply keep the existing system and then deal with a new tech tree features later, possibly after next release.

 

[Trade Winds]

Don't add randomness, add reality.

We have a game where a lot of goods are to be discovered around your cities.

We could create a tech system similar to Kail's Inventor mod where techs need resources to be discovered.

But I propose a step forward. You are only able to discover techs related to the resources you have had contact with.

Instead of researching for "iron", one of your people must have stumbled over a bonus yield of a curious shiny rock and therefore discover that only bonus yield. And after having cut some for your monks, they end up able to discover "iron"

Or you can trade curious stones instead, and then let your people investigate. Curious goods may be fake and may only cost you money for nothing.

So your tree can't progress to masonry if you don't have stone and you must develop your civ on wood structures.

And if you don´t have metal, you may research better and better wooden weapons and leather armours, until you run across metal.

I remember here in the forums that the Chinese built mammoth ships with sails made of bamboo? don't remember well.

Why must cloth sails make better ships than bamboo ones? That happened on real life, didn't it?

 

[Nightinggale]

Don't add randomness, add reality.

It's not complete randomness. Right now you can set it to require either A or B. The idea regarding randomness is that it can be set that in some games it requires A and in some games it requires B, or in some games it requires either A or B. It's not like it will ever end up requiring something, which doesn't make sense.

Also techs, which might randomly not be researchable wouldn't be critical ones. They could do something like +10% production of some yield or something. We could have one tech for each yield and only make some available to some players. Other players would then have access to other such techs and this would make spies, diplomacy and conquest more important.

Instead of researching for "iron", one of your people must have stumbled over a bonus yield of a curious shiny rock and therefore discover that only bonus yield. And after having cut some for your monks, they end up able to discover "iron"

Interesting idea. Something like caching the number of each bonus in the plots you own and then set in techInfo that the tech requires a certain one to be researchable

Seems like a lot of work though and a lot of bookkeeping. Even worse, adding the bookkeeping can be for nothing if this feature is not actually used. Also if you need an iron bonus to get the tech, which reveals the iron bonus on the map, getting hold of a plot with an iron bonus would be game of luck. I don't think that will be fun.

ISo your tree can't progress to masonry if you don't have stone and you must develop your civ on wood structures.

And if you don´t have metal, you may research better and better wooden weapons and leather armours, until you run across metal.

I like this idea. The techtree should be a diverse spreading one with lots of optional techs. The player can then skip techs, where the bonuses makes little sense in the current situation or as you say, the game blocks because the player doesn't own land, which allows them.

At the same time the game shouldn't lock players from developing unless they get certain techs. The list of Or techs could be useful here. Something like requiring 2 out of 5 techs meaning you can move on in the tech tree regardless of which branch you are interested in. This requires some thinking and a lot of work on the tech tree itself, but could result in a way more interesting one.

II remember here in the forums that the Chinese built mammoth ships with sails made of bamboo? don't remember well.

They based the design on bamboo. Bamboo is a long straw with internal walls, making a lot of champers, which are water proof. The wooden ships were built with water tight walls, which gave them 7 compartments and it would still be seaworthy with two of them flooded. Titanic had a similar system with 12 or something compartments. It could survive 3 flooded ones, but ended up with a crack, which punctured the front 4. It's not a perfect system, but way better than the single compartment concept, which all European ships used until the mid 19th century.

The compartment walls adds strength to the hull, allowing it to be bigger and is added to most if not all modern ships. A modern term for them is bulkheads.

I think the Chinese used bamboo on the sails. The sail was made of canvas, but it had poles attached to control the movement. European ships could sail with full sails or half sails (the triangle sails only full). The Chinese bamboo design could change sail area in like 7 steps, allowing it more control to get he right size for the wind, wanted speed and load on the masts.

All this is interesting, but not directly linked to M:C or the tech tree

 

[Trade Winds]

All this is interesting, but not directly linked to M:C or the tech tree

It is. You can research up to grab hold of all seas without ever discovering "Weaving"

 

[Lib.Spi't]

All this is interesting, but not directly linked to M:C or the tech tree

Until the chinese mega junks show up on the shores of europe and destroy all the petty medieval kingdoms!

 

[Nightinggale]

After having been idle for almost a week for various reasons (including a dead CPU fan ) I have finally started having some progress again. Apart from missing GUI, the only major issue left is to get perks working again. To do this, I have started cleaning up the header and are moving the functions/variables into the subclasses. Most noteworthy is techs as they have to be taken out of perks in order to leave room to get perks working in a nearly identical way, but not completely identical.

Techs now look like this in the header:

bool isNoGoodyTech()                      const {return m_bisNoneTradeable;}
bool isNoneTradeable()                    const {return m_bisNoneTradeable;}

TechTypes getRequiredTech()               const {return m_eRequiredTech;}
TechTypes getRequiredTech2()              const {return m_eRequiredTech2;}
TechTypes getRequiredTechOr()             const {return m_eRequiredTechOr;}
UnitClassTypes getRequiredUnitClassType() const {return m_eRequiredUnitClassType;}
const InfoArray* getRequiredYields()      const {return &m_ia_RequiredYields;}

TechCategoryTypes getTechCategory()       const {return m_eTechCategory;}

I decided on the approach of writing the simple return statement in the header rather than having simple access functions. The reason is that I find it more readable to be able to tell that it is simply a read only access function to a variable. Also by doing this, the variable is read inline, which produce less code (smaller DLL file size) and removes a function call (faster execution at runtime).

The tradeoff is that inlined functions can't be accessed by the exe or python as there will not be a compiled function to point to. Also the inlined functions should preferably not have logics as that could have the opposite effect on performance. It's actually surprisingly hard to figure out when to inline, but simply accessing a variable benefits from it.

Changing required yields from an int array into an InfoArray actually produced an interesting result. It requires less code, which is also easier to code. Also the result uses less memory and will execute faster. On top of that it has type checking (it returns TechTypes instead of int) and this change actually uncovered two bugs, which I fixed as well. Even though the bugs was created by introducing CivEffects, the result looks good. Reading the XML file, it will also ensure that people write yields (will not longer accept UNIT_PEASANT or similar). It's like there are plenty of good things about it and no tradeoffs at all.

 

[Nightinggale]

I decided on a file layout design change. I added a new subdir called Info and the plan is to place all info files in there. However currently only CivEffect is moved to prevent possible conflict when merging. Having a subdir of it's own opens up for adding plenty of more files.

Also I split CivEffects into multiple files. Now CivicInfo.h and TechInfo.h are two different files (well there is one for each type). This allows including a header only in the files where it is actually used and if we modify say civic, then files using only tech will no longer be recompiled. In time we should split out often modified infos, such as units, buildings and so on.

This results in more cpp files, but recompiling the same code in 3 files instead of one is faster on multicore CPUs since they can be compiled in parallel. That too is a decent argument for splitting files.

 

[orlanth]

Awesome, those sound like cool changes.

With DLL and XML schemas having already changed so much, its clear all the XML generator scripts I'd worked on for enabling easy generation of modmods of M:C will have to be totally rewritten from scratch to have any chance of being compatible. Is there any goal or estimate of when XML schema will reach a final version, so that work can then start to focus on improving the AI and completing some playable modmods? It will be very worth it in the long run to have an awesome DLL code base that all mods can use, but real work on any planned XML modmods can't really be started until XML schema and game rules are in a relatively final state.

There's been a lot of discussion lately about the real need to improve the vanilla AI, which is very true and could improve gameplay a lot for all mods. So many additional features and new mechanics have been added, that the AI will also need to be trained to use these effectively in addition to fixing its vanilla problems, which can only happen once game rules and mechanics are in place and effort can focus on training the AI to be able to deal optimally with the set of rules/features that are in place.

There are still a lot of very interesting and good new feature ideas that have been proposed (improvements of the domestic and tradescreen economy and pricing system, big improvements to the combat system, civilian and military promotions system, and AI traders, etc).

These do all sound very cool and will likely be great ideas to add, but at some point we should try to declare a goal date for "feature lock", where we can finally say that enough additional rules / mechanics have been added, the XML schema can be finalized to ensure compatibility, and work can then focus on improving the AI to plan effectively and be a good/challenging opponent given the existing features, debugging/optimization of the existing code, and beginning work on interesting mods using the existing code base. With all the cool features in the current version plus the plans mentioned above, having a good stable code base like this would be more than enough for a variety of high quality modmods.

Anyway I'm not wanting to block development of new interesting features that would be cool to implement, but if we don't have some goal for a final codebase we would easily succumb to the all too common inevitable phenomenon of Feature Creep meaning that no playable modmods can develop until its 2021 and no one is playing Colonization anymore .

Also, if we do have a goal for a final common codebase, it could still be possible to make a branch for some additional mod-specific DLL feature that you later decide you really want to add specifically for a certain submod. So it's not to say it would become impossible to add another feature or mechanism for a mod if you really want, just that we could at least have a goal for reaching a final common code base, which would have a great core set of features and high quality challenging AI, and could then allow XML modmods like World History, 2071 etc to start moving toward a real stable / playable modmod. Anyway, what do you guys think?

 

[Lib.Spi't]

I think.... Wrong thread

 

[Nightinggale]

Is there any goal or estimate of when XML schema will reach a final version, so that work can then start to focus on improving the AI and completing some playable modmods?

I don't have a time set for this, but my plan is to finish CivEffects and then aim for a release without adding anything else.

There's been a lot of discussion lately about the real need to improve the vanilla AI, which is very true and could improve gameplay a lot for all mods.

I know and I would like to look into this at some point. However it will be after next release. Also AI updates should be DLL only, meaning submods should not have to be updated in order to benefit from improvements.

These do all sound very cool and will likely be great ideas to add, but at some point we should try to declare a goal date for "feature lock", where we can finally say that enough additional rules / mechanics have been added, the XML schema can be finalized to ensure compatibility, and work can then focus on improving the AI to plan effectively and be a good/challenging opponent given the existing features, debugging/optimization of the existing code, and beginning work on interesting mods using the existing code base. With all the cool features in the current version plus the plans mentioned above, having a good stable code base like this would be more than enough for a variety of high quality modmods.

I proposed a feature lock multiple times, but then somebody else came up with a new idea and just had to code it right away, breaking the feature lock.

Anyway I'm not wanting to block development of new interesting features that would be cool to implement, but if we don't have some goal for a final codebase we would easily succumb to the all too common inevitable phenomenon of Feature Creep meaning that no playable modmods can develop until its 2021 and no one is playing Colonization anymore .

We really need to release soon. If somebody starts adding features once CivEffects is done, then I will remove the git permissions for that person
I would prefer it to be now, but with the major redesign of the XML files, it makes no sense to release now and make submods use the old layout, which can then be scrapped with the next release.

Also I learned from CivEffects that I will not start such a massive redesign again. Once this is done, the XML layout is fixed and changes will be optional additions, which will not break mods even if they aren't updated. An example would be unlocking route building in certain terrain. If not used in XML, then it would stick to default behavior, which is always allowed (like it is right now).

Also, if we do have a goal for a final common codebase, it could still be possible to make a branch for some additional mod-specific DLL feature that you later decide you really want to add specifically for a certain submod. So it's not to say it would become impossible to add another feature or mechanism for a mod if you really want, just that we could at least have a goal for reaching a final common code base, which would have a great core set of features and high quality challenging AI, and could then allow XML modmods like World History, 2071 etc to start moving toward a real stable / playable modmod. Anyway, what do you guys think?

I think branching is bad because then we would end up with multiple codebases to maintain, which is precisely what we want to avoid. Fixing a problem in one branch and then moving that fix to two other branches is time consuming and this time is taken away from developing new features.

If we run into an issue where modA needs one code and modB needs another and those two can't exist in the same game, then we can set the makefile to define different flags at compile time. Alternatively we just define the flag in the yield headers (they are mod specific already, though long term plans aim to remove them). Either way will allow the code can use #ifdef and code, which only works on a single mod would be included for all, but ignored by the compiler when not compiling for that mod.

Currently the concept is to allow everything in all mods, but use XML to tune/enable/disable each feature to match the needs for each mod. We have multiple options to ensure that we share the DLL code, but still fulfill the needs for each mod.

If you want to see an example of a compile time setup, then look at RaRE. I added a switch to set city catchment radius between 1 and 2. I wanted it to be an ingame switch, but it ended up being a compile time switch due to performance reasons.

 

[Nightinggale]

I added a file with the sole purpose of displaying help popup text for CivEffects. There is a function for each CivEffect class and they call each other according to the hierarchy. (like Tech->Gift->Base). This mean the functions should only display the data for the class in question and adding something to the base class is automatically added to all. I ran a short test as it appears to work. However it doesn't display much as all the text needs to be copy pasted from the old civichelp function. I would prefer this to be done in the same order as the variables are in the header or it will be a pain to figure out if the list is complete.

I ran into a problem with adding buttons for this. The arguments were kind of unreadable. For instance what is the effect of setting iData to 2
To get around this, I added the file CvDLLStructDataIO.cpp. This file is a frontend to the regular structs and it can set and read data with human readable function names.

All CivEffect help texts can now be opened with

void CvDLLInterfaceIFaceBase::popupAddCivEffectHelpButton( CvPopup* pPopup, CvWString szText, const CvCivEffectInfo* pCivEffectInfo, bool bShowYields, bool bSkipName);

Reading the data later can be done with getCivic(), getTech() isShowYields() and so on. All easy to read and how it is actually stored in the struct doesn't matter outside CvDLLStructDataIO, which is a good way to prevent read and write from going out of sync. On top of that, there is some error checking. getTech() returns TechTypes, which causes the compiler to fail if it is used as CivicTypes. Changing to use getCivic() instead fixes the compile error, but then it will assert because popupAddCivEffectHelpButton() had a tech as argument, not a civic and the get functions as a positive list of types they can work on. This mean there are less ways to screw up without the game/compiler informing us of issues.

I haven't come up with an equally clean implementation for python, but I will deal with that later.

EDIT:
I ran into a problem with enum types. We have 3 enum types, which shows tech help. Those are:

• WIDGET_INVENTORS_HOUSE
• WIDGET_PEDIA_JUMP_TO_TECH
• WIDGET_PEDIA_JUMP_TO_TECHNOLOGY

Jump to tech opens the tech page in pedia. The inventor's house opens the choose tech popup window.

Jump to technology behaves as inventor's house when left clicking and jump to tech when right clicking.

Personally I find this inconsistent, confusing and it results in messy code. I think it would be best to simply remove technology.

The idea of right clicking to get the pedia is interesting though. We could get a consistent behaviour if we add that to as many widgets as possible. In fact doing that would allow us to remove WIDGET_PEDIA_JUMP_TO_TECH because right clicking WIDGET_TECH_HELP would do the same.

However all this appears to grow and I think it would be a bad idea to mix it in with CivEffects. I better just update the code to handle help popup of all of them and then we can consider pedia access later (like after next release).

 

[Kailric]

The tradeoff is that inlined functions can't be accessed by the exe or python as there will not be a compiled function to point to.

This want interfere with the current python setup of the Tech tree will it? Like it currently uses gc.getCivicInfo(iListCivic).getRequiredInventionOr(), so python needs to be able to read these.


About the WIDGET_'s, yeah that is code I wrote years ago when I was just learning this stuff so who knows what I was thinking

@orlanth

Well said, tomorrow evening I will have finished my most time consuming classes at school and then will start back on finishing what I was working on. Once we get our current changes complete and pushed the release we should work up a set of Goals we absolutely want to reach then make a plan to reach those goals.

 

[Nightinggale]

This want interfere with the current python setup of the Tech tree will it? Like it currently uses gc.getCivicInfo(iListCivic).getRequiredInventionOr(), so python needs to be able to read these.

I'm thinking something like making CyTechInfo, which contains a pointer to CvTechInfo. The Cy version will not inline the functions and since it is part of the DLL, it can access inlined functions. I made CyPlotGroup, meaning it can be done. However I have to check the log to remember how I did it.

About the WIDGET_'s, yeah that is code I wrote years ago when I was just learning this stuff so who knows what I was thinking

The button works if I write it like this
Something like that without considering performance or risk of bugs. I guess that's a good thing because if you started out wanting the standard we want now, I think you wouldn't dare to have touched anything back then.

 

[Nightinggale]

I have done some cleanup of techs and perks and now they appear to be as intended. This mean I think the game is playable if we fix the GUI

There are still a few issues left, like first to discover will not get anything yet and traderoutes can't be unlocked by discovering an access plot. Naturally this should be fixed, but I prefer to get the game up and running normally without any big graphical issues before fixing details.

My next big step is to add python access. I looked up what I did for plotgroups (git is great for that) and I will now try to copy paste that into doing the same for TechInfo. That will be a huge milestone because it will be our first full scale non-vanilla class with python access.

I have been thinking about required techs. I want to make requiredTechs and RequiredTechsOr. Both should be a list of techs. On XML load, the length of the first is stored as requiredNumTechs and then the two lists are merged into one. The check for canResearchTech is then counting how many of the requirements the player has and compare it to requiredNumTechs.

We talked about other fancy features of tech research and requirements and it ended up being an advanced feature. I want to keep it this simple, but at the same time do it this way because then the XML layout is set. The DLL can then add research bonus and stuff like that later if we want that without having to update every single tech in XML. In other words doing this is about doing the minimum required amount of work to turn the XML layout into something, which we will not change later. Maybe the first version will not update the DLL as such and just read up to 2 techs from requiredTechs and max one from RequiredTechsOr, simply to make the implementation as quick as possible.

 

[Lib.Spi't]

Sounds Interesting.

 

[Nightinggale]

Fullerene tried to understand CivEffects and how they could be used for WHM and luckily didn't fully get it. Luckily because that resulted in questions, which made me question the tech/perk design.

RP Research Point
FP Founding Father Point

Currently CivEffects have Techs, which are researched with RP and Perks, which are essentially the very same thing, but they are researched with FP. The question is what about a tech, which requires both? We can't have that right now, but what if we want that?

Fullerene said something about producing RP with experts in their fields. I changed that into FP. The concept would then be something like expert farmers producing a farming FP each turn when working as farmer profession. Improved plowing tech would then require 40 RP and 10 farming FP.

Something I have been thinking about even before this is to make a CvPlayer function to add FP. It will add the points to the player and the team. The team can then use it to "buy" founding fathers while the player can use it to research perks. At the end of the turn, the player's FP storage is capped at what the current research cost. This mean you can't store 5000 of one type you haven't used in a while and then suddenly flash research lots of perks with those.

I like this concept of having perks and techs merged. The only real problem I see is that trade techs are perks and if all perks are changed into techs, then we will not be able to research one tech and one perk at the same time. Maybe the solution is to make two kinds of techs, those which cost RP and those, which doesn't. You then pick one of each kind to research in parallel.

What do you guys say? Should we go for a merge of techs and perks? and how do we solve the problem with parallel research?

 

[Lib.Spi't]

Right now Perks and FFs get their points from more than one thing.

They get it from yields (Political(bells) Religious(crosses), etc.).

I think the Trade Perk also gets it from Gold(or sales), Trading Posts, plotgroup connections(maybe)

Things like that..(I think)

Exploration gets it from tile discovery.

Military from Conquest, etc.

So are all these things defined by xml or hardcoded dll or a mix of both at the moment?

It seems like we have 3 things that all use the same underlying code system.

FFs, Perks and Techs.

FFs and Perks are fed by FPs and Techs by RPs.

I think mixing FPs and RPs into Techs would not be that good.

I think FPs should come from your nations day to day 'Specialist/Specialised' activities.

Like Masters Working their Trade, Exploring the Land, making deals and connections with foreigners (like roads, trade posts, alliances, yield sales, etc.), stabbing things, making converts, etc.

RPs work as they are now with research shops and all the current systems that are unused.

Then instead of a Tech Cost X RP and Y FP, techs stay as they are but Perks are made by their attending FP input. Agricultural, War, Production/Industry, Economy, Religion, etc.

Then Perk Trees can be used to give special bonuses, or even used as shortcuts along the 'Normal' Tech Tree.

Then Perks and FFs could be mixed and 'bought' in the same way. (Maybe with perks being for everyone and FFs being like Wonders).

Then as the game progresses you decide what you want to 'save and spend' on, do you want to save up and grab that Agri FF that is key to your strategy before anyone else locks him out, or do you want to buy 2 or 3 perk innovations and get a 'shortcut' and be the first to develop cotton planting. (Add your own specifics).

That way tech stays 'normal' and ticks along at normal rate and in the normal ways.

However if you want to get ahead in something you have to tailor your nation and society to feed a particular FF stream, so that you get the National FFs(Perks) and Wonder FFs (FFs) before your rivals, and get a bump along the research tree too (In your chosen Societal Area)

 

[Kailric]

The way I designed Perks was to add a way to use FF points with out having to have actual Historical Founding Fathers listed. Vanilla Col has a Trade category but I didn't find many Trade related FF types in the dark age to medieval period so that is what inspired the Trade Perks. I'm sure there are many Trade related FFs for this period I just haven't done enough research into it.

At the moment I don't want to merge Techs and Perks as the system works fine as it is. I want to actually expand on Trade Perks and give players more choices as it is quite easy to make it through the list in the given time. Besides the system can easy be removed and made none existent in XML, if not at the moment it can be programmed for this.

We could also Expand on the whole Perk system such as having a Governmental Administration branch which could unlock new Civic types, or perhaps give Bonuses to certain Civics.

I think I already coded in the possibility to have FF points as a requirement to Techs. Just never used it in M:C.

But, the whole system of Techs, Perks, FF's could be looked at and perhaps redesigned. We still have things such as Wonders that can be added as well. So we could at some point look at those things and see what could be done to make it all better. But for now, what we have works functionally, and what we need is an improved and more historically accurate tech tree, perk tree, and FF's as far as Bonuses and their effects goes.

 

[Nightinggale]

I think I already coded in the possibility to have FF points as a requirement to Techs. Just never used it in M:C.

Since it was unused, it died in the transfer to CivEffects. I figured it was for trade techs only and it ended up in CvPerkInfo and not in CvTechInfo.

It looks like we should keep perks and techs separate. That's fine especially if we consider the options and agree on the solutions. This mean I will not make any changes related to perks and techs at the moment.

One thing, which we could add later is perk requirements for techs and vice versa. Units, which produce FF points in certain professions is also an interesting idea, which we can look into later.

Still we should consider what to do with FF points regarding player vs team as the current setup is less than ideal. Maybe something like all point production goes to the player producing the points. In the add point function, any points not needed by the current research will be forwarded to the team. This mean say we produce 3 military points. The current research requires 0 military points (it's a trade perk), then the team gains those 3 points. If the research required 20, then the player would keep those 3 points and the team will gain nothing. If the research require 20 military and trade, then the 3 can extend the player owned points to 23, then reduce it to 20 and forward 3 to the team.

I think that would be the best system if we are to set up something, which preserves the current FF system and make a player based perk system, which also use FF points.

 

[Lib.Spi't]

That sounds like a reasonable solution.

One thing I can't remember, when you choose an FF do the points get consumed, like a purchase?

Or do they remain?

I am trying to visualise it and I can't.. (I need to reinstall Col. and M:C on my new hard drive.. so many tasks!)

The reason I ask is I am sort of thinking about this merging of FP and RP points into tech costs, and to my mind it seems to be backwards.. let me explain.

So you want to research swords and it costs 20 (military) FP, After that you want to research shields and it costs 30 FP.

This means (in essence) that the Next Military FF will cost 50 points more than normal (because you have to make the 50 research FP before your FP points start flowing into FF Pool) So in a society obsessed with fighting, that is constantly thinking about how to fight more and in better ways, this society is less likely to produce a Military Great Person/Legend...

I know it would probably even itself out after time, as you would probably be doing more FF things like fighting and getting Master Weaponsmiths, but does it not seem odd that it would be like that?