Thoughts about the Domestic Market

[triplebassist]

There are a few things that have been floating in my head about how the Domestic Market works that I'm not quite satisfied with. I'm putting this here because I'd like to see if other people have the same issues, and to brainstorm ways to make changes that make the system work better--I'm far better at reading code than writing it.

Before going into my issues in-depth, I'm going to explain how the Domestic Market works since it's not especially intuitive for someone who hasn't looked at the code or spent a lot of time with the mod. Each type of Colonist, determined by unit type, not unit profession (so a Free Colonist Farmer is equivalent to a Free Colonist Statesman) demands a certain quantity of goods. In the case of the Free Colonist, this is one-fifth of one unit of Leather, Salt, Beer, Wool Cloth, and Coffee, per turn. The Town Market increases this demand value by 50%, and the regional market doubles it.

Units in the game can be divided into the following classes:

Unfree: Petty Criminal, African Slave, Native Slave

Low: Free Colonist, Indentured Servant, Converted Native, Freed Slave, Hardy Pioneer, Evangelist/Jesuit, All Veteran Land Units, All Expert Professions except Expert Rancher

Spoiler Land Units :

Veteran Town Guard, Veteran Soldier, Veteran Cavalryman, Veteran Dragoon, Seasoned Scout

Spoiler Expert Professions :

Expert Miner, Expert Prospector, Expert Stonecutter, Expert Tobacco Planter, Expert Hemp Planter, Expert Cotton Planter, Expert Indigo Planter, Expert Sugar Planter, Expert Coca Collector, Expert Cocoa Planter, Expert Coffee Planter, Expert Red Pepper Planter, Expert Farmer, Expert Grape Picker, Expert Hunter, Expert Seal Hunter, Expert Trapper, Expert Fisherman, Expert Pearl Hunter, Expert Native Trader

Middle: Expert Rancher, All Master Professions

Spoiler Master Professions :

Master Blacksmith, Master Gunsmith, Master Fur Trader, Master Tanner, Master Carpenter, Master Furniture Maker, Master Brewer, Master Distiller, Master Vintager, Master Weaver, Master Dyer, Master Drier, Master Spice Trader, Master Tobbaconist, Master Roper, Master Sailmaker, Master Train Oil Cooker, Master Butcher,

Upper: Firebrand Preacher/Devout Priest, Elder Statesman, Bishop, Governor, Political Refugee, Famous Nobleman, Renowned Medic

These classes define the demand a unit generates, as follows:

Unfree pops generate no demand.

Lower class pops generate a demand of one-fifth of one unit of Leather, Salt, Beer, Wool Cloth, and Coffee, per turn, with some exceptions.

Spoiler Exceptions :

Indentured Servants only generate one-tenth of one unit of demand per turn.
Veteran Land Units generate only three-twentieths of one unit of demand per turn.
Veteran Cavalry demand Rum instead of Beer.
Veteran Dragoons and Veteran Soldiers demand Cloth and Tobacco instead of Coffee and Wool Cloth.
Converted Natives and Freed Slaves demand one-tenth of one unit of Coca Leaves per turn in addition to all other lower-class demand.

Middle-class pops generate a demand of one-tenth of one unit of Luxury Goods, Cigars, Cocoa, Spices, Furniture, and Whale Oil, per turn, and a demand of one-fifth of one unit of Coffee, Cloth, Coats, Salt, and Rum, per turn.

Upper-Class pops generate a demand of one-tenth of one unit of Coca Leaves, three-tenths of one unit of Luxury Goods, and one-fifth of one unit of Cocoa, Coffee, Cigars, Coloured Cloth, Premium Coats, Salt, Spices, Wine, Whale Oil, and Furniture, per turn.

Question: Why is lower-class demand so different?

Three of the five goods demanded by lower-class pops are not demanded by either middle- or upper-class pops, and the other two are the only goods with consistent demand across the social classes. There appears to be an "upgrade chain" of sorts between Wool Cloth, Cloth, and Coloured Cloth, and another between Beer, Rum, and Wine, but I'm not sure those make a lot of sense. The lack of demand for Leather among middle- and upper-class pops is also very confusing to me.

Possible Solution: Change demand among middle- and upper-class pops to be demand on top of the demands of the lower-class, rather than to demand completely different goods. (This is something I could easily do myself; it's just XML work)

Question: Why are Experts considered lower-class?

The thing I find most strange about the current system is that there is no difference between a Free Colonist and an Expert when considering demand, even though the expert is twice as productive and should, therefore, be earning a higher income. There is also no Domestic Market benefit to Learning-by-Doing for an Expert Profession, even as there could be huge changes for a Statesman or a Preacher getting promoted.

Possible Solution: Create a new class between existing lower- and middle-class for the Expert Professions, including the Veteran Land Units. Once again, this is an XML task that I could do myself.

Question: Does the demand have to be statically defined?

This question is why I haven't just done the XML work I want to see. I think there's room for something much more interesting and dynamic than the Domestic Market that we have now, but I don't know how to go about it. I would love to see a system where the relative scarcity of goods in a city's surroundings affects its residents' demand for goods and the prices they are willing to pay for them. Coats are useless in the jungle, but they are necessary at higher elevations and latitudes. Coffee may be common to those surrounded by bushes, but to people far away from ideal growing conditions, it may be a luxury. I would love to see a system where demand and price are influenced by the terrain surrounding a city, as well as the amount of a good that the city produces.

The problem for me is that I have no idea how to go about implementing such a system, let alone balancing it. It would obviously require a lot of work to program, and I'm not sure how to do it. I assume the modifiers would primarily be demand modifiers, and that there might need to be a simulated amount of money that each citizen can spend on goods per turn, but I'm not sure where to go other than that.

 

[ConjurerDragon]

...
Question: Why is lower-class demand so different?

Three of the five goods demanded by lower-class pops are not demanded by either middle- or upper-class pops, and the other two are the only goods with consistent demand across the social classes. There appears to be an "upgrade chain" of sorts between Wool Cloth, Cloth, and Coloured Cloth, and another between Beer, Rum, and Wine, but I'm not sure those make a lot of sense. The lack of demand for Leather among middle- and upper-class pops is also very confusing to me.

Leather is a very robust material for clothing. Craftsmen working at a forge would wear it for protection from the sparks, Miners would historically wear leather for some protection.All other sorts of craftsmen (e.g. butchers) would wear it where the wear and tear from their craft would quickly destroy or spoil normal clothes
https://lederschuerzeshop.de/

The "aristocrats" or leading colonists would have no need for that. They would dress in the finest their colony has available - because they can afford it and want to show it and their clothing has not to endure physical hardships like those of the working masses.

Beer has historically been a very basic nourishment. Not an alcoholic beverage, but (with very low-alcohol beer variants) even used as beer soup for breakfast in vast parts of the european continent because it was affordable for everyone. Wine on the other hand would be something that serves the rich as a memory of their homelands - the wines of Bordeuax, of the Rhine Valley etc.
And the rich in the colonies did emulate the aristocracy of their homelands for a long time before the revolution.

Possible Solution: Change demand among middle- and upper-class pops to be demand on top of the demands of the lower-class, rather than to demand completely different goods. (This is something I could easily do myself; it's just XML work)

I object. The demands of the "classes" are quite different. It makes sense that everyone drinks something - but it does not make sense that the "rich" drink wine on top of rum and beer just because they are the top class and have to use up everything that the lower classes use. On the contrary the nobility of the colonies would shy away from using the stuff that the poor working masses would have to use to ply their trade.

Question: Why are Experts considered lower-class?

The thing I find most strange about the current system is that there is no difference between a Free Colonist and an Expert when considering demand, even though the expert is twice as productive and should, therefore, be earning a higher income. There is also no Domestic Market benefit to Learning-by-Doing for an Expert Profession, even as there could be huge changes for a Statesman or a Preacher getting promoted.

Possible Solution: Create a new class between existing lower- and middle-class for the Expert Professions, including the Veteran Land Units. Once again, this is an XML task that I could do myself.

The existing system is simple and easy with 3 levels of domestic demand and having all population be part of one level. Adding a fourth or more and splitting up the population to more artificial division lines to have different demand - to me - adds nothing worthwhile. Statesmen and Preachers are at the level of university learning, while the others are a college or normal school level of learning. So it make sense that from a consumption level a Free Colonist is on the same level as the Experts.

Question: Does the demand have to be statically defined?

This question is why I haven't just done the XML work I want to see. I think there's room for something much more interesting and dynamic than the Domestic Market that we have now, but I don't know how to go about it. I would love to see a system where the relative scarcity of goods in a city's surroundings affects its residents' demand for goods and the prices they are willing to pay for them. Coats are useless in the jungle, but they are necessary at higher elevations and latitudes. Coffee may be common to those surrounded by bushes, but to people far away from ideal growing conditions, it may be a luxury. I would love to see a system where demand and price are influenced by the terrain surrounding a city, as well as the amount of a good that the city produces.

The problem for me is that I have no idea how to go about implementing such a system, let alone balancing it. It would obviously require a lot of work to program, and I'm not sure how to do it. I assume the modifiers would primarily be demand modifiers, and that there might need to be a simulated amount of money that each citizen can spend on goods per turn, but I'm not sure where to go other than that.

Sure - like Frederick of Prussia who found out that it was to the advantage of the population and the state if his soldiers spend their money at home instead of abroad while at war. Soldiers during war could double their pay/demand if that would be possible.

However I am not quite sure myself how the system transfers the demands of units on the map (e.g. scouts, soldiers) to the city display?

 

[raystuttgart]

Question: Does the demand have to be statically defined?

The answer is extremely simple and you have partially given it yourself.
(In terms of "static" just checking UnitTypes in Cities.)

• A dynamic system would have been much more effort to implement.
• A dynamic system would have been less performant (because there would be much more checks necessary).
  
• A dynamic system would have been more difficult to understand by players.
• A dynamic system is more difficult to balance if something in the mod would change (e.g. Yields).
  
• ...

The "static system" simply worked good enough for us.

Otherwise:

In general, I am relatively happy with the current system and there was some thought put into it.
It was however not a simple Class-System. (We tried to have the demands a bit more realistic to history.)

But go ahead and make a suggestion for a "Demand Distribution".
You can simply implement it in XML and post a few Screenshots later.

Please however use this version as a base, since it is the most current one.
If team and community like it, we can then integrate it easily.

 

[raystuttgart]

However I am not quite sure myself how the system transfers the demands of units on the map (e.g. scouts, soldiers) to the city display?

Wow, that is pretty much a no-go for me.

• It would ruin performance (because we would loop through all Units on the Map and in Cities)
  
• We would end up with beaming goods (or how should it work otherwise?)
  
• It is almost impossible to visualize for domestic demand and would just confuse people
  
• We are planning to implement an Up-Keep System (for Military Units on the Map) anyways

 

[triplebassist]

The answer is extremely simple and you have partially given yourself.
(In terms of "static" just checking UnitTypes.)

• A dynamic system would have been much more effort to implement.
• A dynamic system would have been less performant (because there would be much more checks necessary).
  
• A dynamic system would have been more difficult to understand by players.
• A dynamic system is more difficult to balance if something in the mod would change (e.g. Yields).
  
• ...

The "static system" simply worked good enough for us.

If I didn't make it clear, I am very aware of the issues with that dynamic system, especially on the side of writing the code. I think it could be decently done using XML, but it would be a shoestring that I'm not sure is worth the effort, and certainly not something I want to do without having an idea of what would work.

Otherwise:

In general, I am relatively happy with the current system and there was some thought put into it.
It was however not a simple Class-System. (We tried to have the demands a bit more realistic to history.)

The class system was a simplification for the sake of explanation, but I did notice reading the XML that there is a very clear 3 step demand progression. Every Expert but the rancher has the same <iYieldDemand> values, etc. I can understand from a balance perspective why you wouldn't want there to be a ton of variation between Fishermen and Farmers, though.

But go ahead and make a suggestion for a "Demand Distribution".
You can simply implement it in XML and post a few Screenshots later.

Please however use this version as a base, since it is the most current one.
If team and community like it, we can then integrate it easily.

I'll try my hand at in here in the next couple of days!

Leather is a very robust material for clothing. Craftsmen working at a forge would wear it for protection from the sparks, Miners would historically wear leather for some protection.All other sorts of craftsmen (e.g. butchers) would wear it where the wear and tear from their craft would quickly destroy or spoil normal clothes
https://lederschuerzeshop.de/

The "aristocrats" or leading colonists would have no need for that. They would dress in the finest their colony has available - because they can afford it and want to show it and their clothing has not to endure physical hardships like those of the working masses.

While I agree that, say, Elder Statesmen are unlikely to use significant amounts of leather, I want to point out that Butchers, Carpenters, Blacksmiths, and other laborers in the city do not currently demand leather. That, I think, is a problem and rather absurd.

Beer has historically been a very basic nourishment. Not an alcoholic beverage, but (with very low-alcohol beer variants) even used as beer soup for breakfast in vast parts of the european continent because it was affordable for everyone. Wine on the other hand would be something that serves the rich as a memory of their homelands - the wines of Bordeuax, of the Rhine Valley etc.
And the rich in the colonies did emulate the aristocracy of their homelands for a long time before the revolution.

What I think is weird is that there is almost an "upgrade chain" for preferred alcoholic beverage. I'd prefer a system where beer is mostly consumed by the "lower-class" citizens, but where carpenters demand a small amount as well. Something not entirely different would occur with the other drinks.

I object. The demands of the "classes" are quite different. It makes sense that everyone drinks something - but it does not make sense that the "rich" drink wine on top of rum and beer just because they are the top class and have to use up everything that the lower classes use. On the contrary the nobility of the colonies would shy away from using the stuff that the poor working masses would have to use to ply their trade.

The existing system is simple and easy with 3 levels of domestic demand and having all population be part of one level. Adding a fourth or more and splitting up the population to more artificial division lines to have different demand - to me - adds nothing worthwhile. Statesmen and Preachers are at the level of university learning, while the others are a college or normal school level of learning. So it make sense that from a consumption level a Free Colonist is on the same level as the Experts.

The consumption patterns of Statesmen and Preachers should be very different from that of a Free Colonist. I agree. What I do not agree with, however, is that they have no demand for Cloth, since Coloured Cloth is available. Surely they would demand Coloured Cloth for some things and Cloth for others. Or certain "middle-class" professions like Fur Traders would consume some level of wine. Less, perhaps, than their richer counterparts, but surely not *none*. I also think that having a Free Colonist have the same demand as an Expert Farmer misses the fact that the Expert Farmer would have a higher income, all else equal, because he is clearly better at his job. This is currently not represented at all. There are also certainly arbitrary things about the current system. Explain the Veteran Land Units without falling back on them!

 

[triplebassist]

Wow, that is pretty much a no-go for me.

• It would ruin performance (because we would loop through all Units on the Map and in Cities)
  
• We would end up with beaming goods (or how should it work otherwise?)
  
• It is almost impossible to visualize for domestic demand and would just confuse people
  
• We are planning to implement an Up-Keep System anyways

I don't think he realized that you can have a Veteran Land Unit work a "normal" profession. They're not necessarily "on the map", as it were

 

[raystuttgart]

They're not necessarily "on the map", as it were.

That is true.

But "Military Experts" still might have a little different "rough" taste thus different Domestic Demand.
If I remember the balancing correctly, I think it was not modelled that badly currently.

But as I said:
Go ahead and make a suggestion - it is easier to discuss details then.
(Directly in XML and posting ScreenShots - and if you like an Overview Table.)

Just a few comments from my side though:

Please do not overexaggerate total demand.
Domestic Market already is a Cash Cow - it should not become overpowered.

Also please do not make it too complicated to understand or use in game play.
Players should still have a general feeling and be able to keep overview.

We should not end up in endless personal taste discussions either.
Sometimes compromises or simplifications are needed.

 

[raystuttgart]

There is also no Domestic Market benefit to Learning-by-Doing ...

There is by the way.
It is however an indirect effect chain.

Fulfilling Domestic Demand --> Increasing Happiness --> Faster Learning By Doing

There are way more of these indirect effect chains than players realize.

Some others of these indirected chains I had planned for in my game design have simply not been implemented yet.
(e.g. I was originally planning to implement "Religions" or "Civics" that would add further ones.)

 

[raystuttgart]

I want to point out that Butchers, Carpenters, Blacksmiths, and other laborers in the city do not currently demand leather. That, I think, is a problem and rather absurd.

I agree generally.
Well yes, they are Expert Units (technically) - but they do resemble at least a little bit their intended Profession.

Professions like Butchers, Carpenters and Blacksmiths need Leather for their daily business.
And what has always been shown in reality: The daily Profession shapes the Person.

Matching Domestic Demand of Expert Units to their intended Profession seems perfectly valid for me.
So if you want to add leather for their domestic demand, that is absolutely fine to me.

 

[triplebassist]

I assume that you were aware of the Domestic Market changes in the new version, but I'm going to put them here just so they're documented before the changes I make. Once again, I am using the class division as a matter of convenience, rather than any code labeling units as such.

ALL Colonists labeled in my initial post as lower- or middle-class, except the Expert Land Units, now demand one-tenth of one unit of Beer, Rum, and Wine*. Upper-class units still demand no Beer or Rum but do demand Wine.

*Note that Master Brewers, Distillers, and Vintigers demand 20, rather than 10, of the beverage they produce.

Now for my changes:

Expert Ranchers are given demand in line with the other "raw material" specialists

Master Blacksmiths, Master Carpenters, Master Butchers, Master Tanners, Master Rope Makers, Master Sail Makers, Master Gunsmiths, and Master Train Oil Cookers now demand half as much Leather as a raw-material producing specialist, up from zero.

"Lower-class" units now demand only Beer and Rum. "Upper-class" units now demand some amount of Rum.

I did not adjust anything relating to the Expert Land Units, nor did I adjust the increased alcohol demand from Brewers, Distillers, and Vintigers.

I did not make any changes to the demands of Missionaries. (Incidentally, why does the Failed Missionary demand Wine instead of Beer?)

That's all done in the code, but I'm getting Python exceptions when I try to load up the mod to make sure it's all working right I'm fairly certain I didn't screw anything up, but code is always so picky.

EDIT: Of course I messed something up. I missed a closing brace on one of the blocks of code

 

[raystuttgart]

... I'm getting Python exceptions when I try to load up the mod to make sure it's all working right
I'm fairly certain I didn't screw anything up, but code is always so picky.

Code? Your changes should be XML only.

The Python exceptions you might get considering the MapScript FaireWeather are nothing to worry about.
That would not be your mistake.

They simply appear if you load the mod through starting Vanilla first.
(It is related to Player preferences from playing WTP stored in the Player Profile about MapScripts and Settings Vanilla does not have.)

However:
If you get Error messages about XML loading, that is a different story.
But it would tell you exactly the line where it finds an error.

 

[raystuttgart]

Incidentally, why does the Failed Missionary demand Wine instead of Beer?

Well, because it was just copied and pasted from another Unit.
I did not care that much about such tiny details, when merging the feature from Ramstormp.

 

[triplebassist]

Turns out I'd just forgotten to put the new DLL in the assets folder

It did give me a chance to clean what was admittedly bad XML though

 

[triplebassist]

Well, because it was just copied and pasted from another Unit.
I did not care that much about such tiny details, when merging the feature from Ramstormp.

I was thinking it could be something like that. I may as well change it, since I'm doing things in that file anyway

 

[raystuttgart]

By the way:
Your changes considering Alcohol and Leather are looking ok to me.
Please do not focus on Alcohol and Leather only however.

 

[triplebassist]

By the way: Your changes considering Alcohol and Leather are looking ok to me.
Please do not focus on Alcohol and Leather only however.

I am glad to hear it. I began with these changes because I thought they would be the most likely to be accepted and would not require me to worry too much about making the Domestic Market too powerful.

I think that tonight I will consider how to best distinguish the Expert Professions from the Free Colonist. My initial thought is to give them a small amount of demand for Coats and Cloth (weight 10 for each), which should be enough to increase the value those Colonists provide without causing too much imbalance to the Domestic Market.

I have other ideas for further down the line, but I think they would severely affect the balance of the Domestic Market in a way that I am hesitant to do (I am thinking here about the Goods that have no Domestic Market demand, like Tools)

EDIT: Coats and Cloth demand implemented

 

[Nenos]

Ya youre right with tools and I also would say wood for cooking and heating but it would break the current game mechanik. We have to add a tool to avoid the consumention of trading to the locals. And I think we have to incrise to amoung of wood which we receive from wood.
By the way good work until now.

 

[raystuttgart]

I am thinking here about the Goods that have no Domestic Market demand, like Tools

Please do NOT implement any Domestic Demand considering strategic goods like e.g. Tools, Weapons, Horses, Wood, Stone ...

There are many good reasons (e.g. AI) why we have purposely not done that.
It would also be extremely annoying to not being able to finish Buildings because Domestic Market has e.g. consumed some of the Tools needed to do that.

 

[triplebassist]

Please do NOT implement any Domestic Demand considering strategic goods like e.g. Tools, Weapons, Horses, Wood, Stone ...

There are many good reasons (e.g. AI) why we have purposely not done that.
It would also be extremely annoying to not being able to finish Buildings because Domestic Market has e.g. consumed some of the Tools needed to do that.

Understood

 

[raystuttgart]

Understood

Sorry, I did not mean to sound rude.
I just wanted to warn you about negative impacts and side effects.

It can however easily damage game play and AI if such impacts and side effects are not considered.
(I fully undestand though that modding beginners do not yet know about such things.)

Summary:

The absolutely only Yields the Domestic Market should consume are final Cash Yields. (Coca Leaves, Rum, Beer, Wine, Leather, Cloth, Luxury Goods ...)
(The Yields that can not be used for anything else than selling.)

• Food should not be consumed by Domestic Market (because it is needed for Growth or equipping Settlers)
  
• Raw Yields (Tobacco, Sugar, Cotton, ...) needed to produce final Cash Yields should not be consumed by Domestic Market
• Yields needed to equip Professions (Guns, Horses, Blades, Trade Goods ...) should not be consumed by Domestic Market
• Yields needed to construct Buildings and Ships (Lumber, Stone, Tools, Sailcloth, Ropes, ...) should not be consumed by Domestic Market
• Immaterial Yields (e.g. Liberty Bells, Crosses, Health, Happiness, ...) should of course not be touched either.

Only the Yields we already have configured for consumption in Domestic Demand can be used safely.
Everything else will most likely mess up another game system, mess up AI or might even cause real bugs.