The Expression System

A

AIAndy

Deity
Joined
Jun 8, 2011
Messages
3,428
Introduction
This will be a kind of specification/explanation as there is no implementation yet.

The idea is to move more modding ability to a generic XML syntax that currently requires very specific code support (e.g. by specialized tags). For instance now units can require one of a list of specific resources but if you want to require all of the list you need a new tag. And that functionality is specific to units only and for buildings it might look different again.

Definitions
Since this is an abstraction approach, first some definitions (some new, some old):
Game objects are the major objects that are instantiated in the Civ4 game world and have a representation in memory for each instance.
That are:
  • Game - only one is present, it represents everything that is global
  • Team - a group of players that are permanently allied and share tech and some wonder effects
  • Player - in a normal single player Civ4 game without set up teams there is no real difference between player and team
  • City
  • Unit
  • Plot
Game objects have properties (from the generic property system) and attributes (an abstract interface to the properties that are not modelled by the generic property system). There are relations between game objects (near, same plot, trade, associated).
Game objects can have game object modifiers (GOMs, that includes buildings, promotions, features, terrains, resources, traits, ...). Game objects either have a specific GOM or not (so you can query a GAMEOBJECT_CITY if it has the GOM_BUILDING BUILDING_PALACE or not).

Expressions
Now I want to add boolean expressions (conditions) and integer expressions that are evaluated in the context of a game object and return a boolean/integer. They could be used in the XML where now there are either fixed booleans or fixed integers. The limitations are that they can only be used where the results are not used for long term caching, especially cached accumulated values. In general there also needs to be a code change at each position it is used that replaces the integer or boolean with an expression variable.
As an example: Currently you can have a property source that produces 5 crime per turn. Using an integer expression you could have it produce (unhappiness + population) / 2 crime per turn.

Values
Boolean Expressions
  • Global Define Value
  • Option Value
  • Boolean Attribute
  • Has GOM
Integer Expressions
  • Global Define Value
  • Integer Attribute
  • Property
  • Constant (integer literal)

Operators
Boolean Expressions
  • Boolean Expression AND Boolean Expression
  • Boolean Expression OR Boolean Expression
  • NOT Boolean Expression
  • Boolean Expression EQUAL Boolean Expression
  • IF Boolean Expression THEN Boolean Expression ELSE Boolean Expression
  • Integer Expression EQUAL Integer Expression, similar for other comparisons of ints
Integer Expressions
  • Integer Expression PLUS Integer Expression
  • Integer Expression MINUS Integer Expression
  • Integer Expression * Integer Expression
  • Integer Expression / Integer Expression
  • Integer Expression MODULO Integer Expression
  • ABS Integer Expression
  • IF Boolean Expression THEN Integer Expression ELSE Integer Expression
  • Probably more

Integrators/Aggregators
This evaluates expressions on related game objects and then aggregates the result.
Boolean Expressions
  • IntegrateAND Boolean Expression - True if the expression evaluates true on all related objects
  • IntegrateOR Boolean Expression - True if any are true
Integer Expressions
  • IntegrateSUM Integer Expression - Sum of the integer expression evaluated on the related objects
  • IntegrateMEAN Integer Expression - Mean of the expression evaluated on the related objects
  • IntegrateNUM Boolean Expression - Number of related objects for which the boolean expression evaluates to true

The XML syntax for all of that is not yet decided.
 
AIAndy said:
Introduction
This will be a kind of specification/explanation as there is no implementation yet.

The idea is to move more modding ability to a generic XML syntax that currently requires very specific code support (e.g. by specialized tags). For instance now units can require one of a list of specific resources but if you want to require all of the list you need a new tag. And that functionality is specific to units only and for buildings it might look different again.

Definitions
Since this is an abstraction approach, first some definitions (some new, some old):
Game objects are the major objects that are instantiated in the Civ4 game world and have a representation in memory for each instance.
That are:
  • Game - only one is present, it represents everything that is global
  • Team - a group of players that are permanently allied and share tech and some wonder effects
  • Player - in a normal single player Civ4 game without set up teams there is no real difference between player and team
  • City
  • Unit
  • Plot
Game objects have properties (from the generic property system) and attributes (an abstract interface to the properties that are not modelled by the generic property system). There are relations between game objects (near, same plot, trade, associated).
Game objects can have game object modifiers (GOMs, that includes buildings, promotions, features, terrains, resources, traits, ...). Game objects either have a specific GOM or not (so you can query a GAMEOBJECT_CITY if it has the GOM_BUILDING BUILDING_PALACE or not).

Expressions
Now I want to add boolean expressions (conditions) and integer expressions that are evaluated in the context of a game object and return a boolean/integer. They could be used in the XML where now there are either fixed booleans or fixed integers. The limitations are that they can only be used where the results are not used for long term caching, especially cached accumulated values. In general there also needs to be a code change at each position it is used that replaces the integer or boolean with an expression variable.
As an example: Currently you can have a property source that produces 5 crime per turn. Using an integer expression you could have it produce (unhappiness + population) / 2 crime per turn.

Values
Boolean Expressions
  • Global Define Value
  • Option Value
  • Boolean Attribute
  • Has GOM
Integer Expressions
  • Global Define Value
  • Integer Attribute
  • Property
  • Constant (integer literal)

Operators
Boolean Expressions
  • Boolean Expression AND Boolean Expression
  • Boolean Expression OR Boolean Expression
  • NOT Boolean Expression
  • Boolean Expression EQUAL Boolean Expression
  • IF Boolean Expression THEN Boolean Expression ELSE Boolean Expression
  • Integer Expression EQUAL Integer Expression, similar for other comparisons of ints
Integer Expressions
  • Integer Expression PLUS Integer Expression
  • Integer Expression MINUS Integer Expression
  • Integer Expression * Integer Expression
  • Integer Expression / Integer Expression
  • Integer Expression MODULO Integer Expression
  • ABS Integer Expression
  • IF Boolean Expression THEN Integer Expression ELSE Integer Expression
  • Probably more

Integrators/Aggregators
This evaluates expressions on related game objects and then aggregates the result.
Boolean Expressions
  • IntegrateAND Boolean Expression - True if the expression evaluates true on all related objects
  • IntegrateOR Boolean Expression - True if any are true
Integer Expressions
  • IntegrateSUM Integer Expression - Sum of the integer expression evaluated on the related objects
  • IntegrateMEAN Integer Expression - Mean of the expression evaluated on the related objects
  • IntegrateNUM Boolean Expression - Number of related objects for which the boolean expression evaluates to true

The XML syntax for all of that is not yet decided.
Click to expand...

Minor ommissions:

boolean literal

integer option (BTW - is this BUG option or game option or player option - might need any/all?)

More significantly I think you should allow for arbitrary named integrators/aggregators, where the name is not signifiant tio the XML syntax, but is bound at runtime to a well-known function (i.e. - it maps to a registered function in the DLL for the nam in question). That allows us to add more of them without any chnage to the XML schema, or the XML parsing, or the evaluation scaffolding. For exmaple, maybe something needs a geometric-mean - having the late-bound names would allow us to just write such a function in the DLL and publish the nam it registers against, and immediately be able to use it without any other code changes.
 
Koshling said:
Minor ommissions:

boolean literal

integer option (BTW - is this BUG option or game option or player option - might need any/all?)

More significantly I think you should allow for arbitrary named integrators/aggregators, where the name is not signifiant tio the XML syntax, but is bound at runtime to a well-known function (i.e. - it maps to a registered function in the DLL for the nam in question). That allows us to add more of them without any chnage to the XML schema, or the XML parsing, or the evaluation scaffolding. For exmaple, maybe something needs a geometric-mean - having the late-bound names would allow us to just write such a function in the DLL and publish the nam it registers against, and immediately be able to use it without any other code changes.
Click to expand...
I have not decided on the actual XML syntax yet, but yes, I agree that the aggregator function will likely be referenced in a parameter tag.

And in case of the options, best is probably one parameter for option type and then depending on the option type reference it by string or other id.
 
OK, two quick questions.

1. Could you have properties expressed as a function involving other properties? Your example has Crime expressed as a function of Population and Unhappiness, but could it be expressed as a function of properties?

2. Could you use this to allow limited Property spread between Multiple Maps? I don't know too much about the actual code underlying Multiple Maps, but I think it would be nice to allow certain properties/commerces to be expressable on the Earth map based on things going on in the Galactic Map. Would this system be useful for such a setup, or would that just be part of making Multiple Maps compatable with the Property System?

As always, great job with this, I hope we see it soon.
 
ls612 said:
OK, two quick questions.

1. Could you have properties expressed as a function involving other properties? Your example has Crime expressed as a function of Population and Unhappiness, but could it be expressed as a function of properties?
Click to expand...
Yes, see Integer Expressions under Values.

2. Could you use this to allow limited Property spread between Multiple Maps? I don't know too much about the actual code underlying Multiple Maps, but I think it would be nice to allow certain properties/commerces to be expressable on the Earth map based on things going on in the Galactic Map. Would this system be useful for such a setup, or would that just be part of making Multiple Maps compatable with the Property System?
Click to expand...
Multi Maps will probably see the introduction of GAMEOBJECT_MAP and map properties (not only plot or game properties).
And given proper relations you could use this to express some values on earth depending on things on other maps or use propagators to spread some properties beyond maps.
 
I have pushed the first version of the boolean expressions to the SVN now.
For starters it can be used to provide an Active tag to property manipulators.

Like this:
Code: 
<Active>
  <And>
    <Has>
      <GOMType>GOM_FEATURE</GOMType>
      <ID>FEATURE_FOREST</ID>
    </Has>
    <Or>
      <Has>
        <GOMType>GOM_ROUTE</GOMType>
        <ID>ROUTE_ROAD</ID>
      </Has>
      <Has>
        <GOMType>GOM_ROUTE</GOMType>
        <ID>ROUTE_PAVED</ID>
      </Has>
    </Or>
  </And>
</Active>
That one would mean the property manipulator is active on plots with a road or paved road running through a forest.
 
The Has tag allows to query a lot of different GOMs (game object modifiers).
That are:
GOM_BUILDING,
GOM_PROMOTION,
GOM_TRAIT,
GOM_FEATURE,
GOM_OPTION,
GOM_TERRAIN,
GOM_GAMESPEED,
GOM_ROUTE,
GOM_BONUS,
GOM_UNITTYPE,
GOM_TECH

With GOM_OPTION you can make a certain source or propagator only active when a specific game option is chosen.
The result of most of those depends on which game object it is evaluated. A GOM_FEATURE evaluated on a city will return true if there are any plots with that feature belonging to the city. Evaluated on a plot it will only return true if that specific plot has the feature.
 
Nice! I'll have to check out what I can do with this when I get back. Great job as always!:goodjob:

Edit: Actually, I have one quick question. Does this only apply to Properties atm, or can it work with other things as well?
 
ls612 said:
Nice! I'll have to check out what I can do with this when I get back. Great job as always!:goodjob:

Edit: Actually, I have one quick question. Does this only apply to Properties atm, or can it work with other things as well?
Click to expand...
For starters I have only added it to the new Active tag for the property manipulators but its usage is not meant to be restricted to there and the code is generic enough to be added easily to other places.
So if you have specific bools that you would like to have expressions for then tell me and I will check if it is possible for that bool and add it.
 
@AIAndy - observation/reminder - while the usage remains confined to property code there is no AI impact. However, if/when you extend this to other areas don't forget you'll need to update various AI evaluation routines to take account of the dynamic logic. For example:
  • Resource evaluation takes account of what builduings it enables that were not previously enabled (so it has to check that the new resource enables something that was not enabled without the resource). Similarly enabled units
  • Building evaluation takes account of what units or other buildings this building enables
  • Building evaluation takes account of the value of any new resources that building provides (which can recurse)
  • Tech evaluation has to evaluate buildings and units it enables (and resources it reveals or enables). If the requiremnts on the (putatively enabled) entity include other factors than the tech, which are capable of including dynamic logic, then this will be impacted
I think the main manifestation is logic in the above routines that evaluates AND and OR requirements (where they existed in the static schema). Hopefully they can be somewhat universally replaced by some general isEnabledBy() code that encapsulates both the old and the new (though since the old is very class specific that might be tricky - almost like we need to write a runtime compiler that compiles the old static schema stuff ALL into the new dynamic mold, so that the underlying engine just sees the uniform dynamic logic??)
 
I think one of the easiest ways to implement that is to allow the evaluation of an expression given a list of overrides to game object states.
The overrides take precedence over really querying the game object. So if there is an override of "Game Object Player X has GOM_TECH TECH_PHYSICS true" then an evaluation of that as part of an expression will return true even if player X currently does not have TECH_PHYSICS.
Then you can query once without the override and once with and you know if TECH_PHYSICS will enable this.
 
AIAndy said:
I think one of the easiest ways to implement that is to allow the evaluation of an expression given a list of overrides to game object states.
The overrides take precedence over really querying the game object. So if there is an override of "Game Object Player X has GOM_TECH TECH_PHYSICS true" then an evaluation of that as part of an expression will return true even if player X currently does not have TECH_PHYSICS.
Then you can query once without the override and once with and you know if TECH_PHYSICS will enable this.
Click to expand...

Seems plausible. Eventually however, I think the best thing would be if all entities had a simple isEnabled() method (that takes the list of overrides you mention), and internally (not in the XML schema) ALL enablement criteria are XML-parse-time compiled into a single dynamic logic statement. That way the runtime all uses a single method for all entity types which will shrink the complexity of the evaluation code consirably. The work here is obviously the (bespoke per entity type based on their XML schema) parse-time compilation of the dynamic logic statement.
 
@AIAndy: OK, here are the boolean UnitInfo tags that I could see some use for expressions in.

  • bOnly Defensive
  • bSpy
  • bFirstStrikeImmune
  • bNoDefensiveBonus
  • bIgnoreBuildingDefense
  • bCanMoveImpassable
  • bHiddenNationality

And in PromotionInfos;

  • bBlitz
  • bAlwaysHeal
  • bEnemyRoute
  • bLeader

Could you please expose these tags to the Boolean Expression System? Thanks in advance.:goodjob:
 
ls612 said:
@AIAndy: OK, here are the boolean UnitInfo tags that I could see some use for expressions in.

  • bOnly Defensive
  • bSpy
  • bFirstStrikeImmune
  • bNoDefensiveBonus
  • bIgnoreBuildingDefense
  • bCanMoveImpassable
  • bHiddenNationality

And in PromotionInfos;

  • bBlitz
  • bAlwaysHeal
  • bEnemyRoute
  • bLeader

Could you please expose these tags to the Boolean Expression System? Thanks in advance.:goodjob:
Click to expand...
Some of those will be difficult because of different reasons but especially lots of calls from places that do not really have the necessary object context or use caching.
Others should work fine.
What type of expression were you considering for those bools?
 
AIAndy said:
Some of those will be difficult because of different reasons but especially lots of calls from places that do not really have the necessary object context or use caching.
Others should work fine.
What type of expression were you considering for those bools?
Click to expand...

Alright, here goes.

1. bOnlyDefensive; I would like to make Pacifism make many special/limited/unique units have bOnlyDefensive.

2. bSpy; How did that get on the list:confused:

3. bFirstStrikeImmune; To be put on some units when you have Unmanned Warfare civic active.

4. bNoDefensiveBonus; Not sure on that one, but it may be nice in the future.

5. bIgnoreBuildingDefense; Basically certain techs would enable more and more air units to get this tag, as bombs become more and more precise and powerful.

6. bCanMoveImpassable; I would like for units with a combination of promotions (not sure which ones specifically yet) to be able to move impassable terrains in Multi-Maps. More to come on this one.

7. bHiddenNationality; For Banditry, I could make some melee units have hidden nationality while Banditry is active, thus making it more useful in the midgame.

8. bBlitz; This one is quite interesting, I would like for tanks to have a free promotion that allows them to have Blitz, but only on certain terrains/routes, that way we could make Armored Warfare more realistic. (think North Africa in WWII)

9. bAlwaysHeal; Techs would give this to certain units, as nanomedicine in the TH era becomes more advanced.

10. bEnemyRoute; Err, now that I read through this again I notice that there is no GOM_RELIGION. Is that intentional?

11. bLeader; I am still hashing this one out, but I want it to tie in somehow with SO's new Advanced Traits (I assume he is still working on those)

Which ones are hard to do and which ones are OK?
 
ls612 said:
Alright, here goes.

1. bOnlyDefensive; I would like to make Pacifism make many special/limited/unique units have bOnlyDefensive.
Click to expand...
The AI assumes that does not change and it will not understand by default what changes this.

2. bSpy; How did that get on the list:confused:

3. bFirstStrikeImmune; To be put on some units when you have Unmanned Warfare civic active.
Click to expand...
That is one of those that seemed to be doable without too much effort on a fast check. The problem are always direct queries to the unit type info which often do not have a usable context but in this case there are few.

4. bNoDefensiveBonus; Not sure on that one, but it may be nice in the future.

5. bIgnoreBuildingDefense; Basically certain techs would enable more and more air units to get this tag, as bombs become more and more precise and powerful.
Click to expand...
Probably doable as player context is enough to evaluate that.

6. bCanMoveImpassable; I would like for units with a combination of promotions (not sure which ones specifically yet) to be able to move impassable terrains in Multi-Maps. More to come on this one.

7. bHiddenNationality; For Banditry, I could make some melee units have hidden nationality while Banditry is active, thus making it more useful in the midgame.
Click to expand...
Interesting, and probably doable from the mechanic point. But the AI does not know how to deal with Hidden Nationality units except for pirate ships I think.

8. bBlitz; This one is quite interesting, I would like for tanks to have a free promotion that allows them to have Blitz, but only on certain terrains/routes, that way we could make Armored Warfare more realistic. (think North Africa in WWII)

9. bAlwaysHeal; Techs would give this to certain units, as nanomedicine in the TH era becomes more advanced.
Click to expand...
Promotions in general have the problem that their effect is cached on the unit object. So it needs some extra code changing the caching. Also, AI would not know how to deal with Blitz that is only available under certain conditions I think.

10. bEnemyRoute; Err, now that I read through this again I notice that there is no GOM_RELIGION. Is that intentional?
Click to expand...
No, just forgotten like GOM_CORPORATION. I will add both.

11. bLeader; I am still hashing this one out, but I want it to tie in somehow with SO's new Advanced Traits (I assume he is still working on those)
Click to expand...
Isn't that the tag that marks the warlord promotions?
 
To do the simple fix to the basic forestry mod I need to be able to say that the build/improvement can't be done on forest or jungle. I have noticed that the AI replaces forest with tree nursery. Can this be done with this expression system? Otherwise I am going to have to put in a cannotBuldImprovement callback.
 
You must log in or register to reply here.

Similar threads

Ferocca
"not a member" problem
Replies
2
Views
176
Ferocca
Ferocca
C
MaxPerPlayer not working right?
Replies
0
Views
809
ComradeNoot
C
M
CIV4 Handicap Info: iBarbarian Bonus; iAIBarbarian Bonus, no effect? Kmod?
Replies
1
Views
311
MassRiflemen
M
Tigranes
Let's celebrate 10 year anniversary of DoC!
Replies
7
Views
457
AOS9001
AOS9001
T
Interesting bug allows super sized cities
Replies
3
Views
468
Blake00
Blake00
Top Bottom