[3.6.2] Some buildings does not show up in civilopedia

[skodkim]

Hi

Tested with CIN/CAT/CEP 3.6.2 and no other mods.

I've found that some buildings does not show up in civilopedia. I've narrowed it down the introduction of a number of Building that can be purchased with faith using CEP.

I've also narrowed the culprit down to the following in CEC__End.sql:

Spoiler :

UPDATE Buildings SET UnlockedByBelief = 1, FaithCost = 1 * Cost
WHERE (FaithCost > 0 AND Cost > 0)
OR BuildingClass IN (
	'BUILDINGCLASS_LIBRARY'			,
	'BUILDINGCLASS_WORKSHOP'		,
	'BUILDINGCLASS_STABLE'			,
	'BUILDINGCLASS_WATERMILL'		,
	'BUILDINGCLASS_WINDMILL'		,
	'BUILDINGCLASS_FACTORY'			,
	'BUILDINGCLASS_CARAVANSARY'		,
	'BUILDINGCLASS_MARKET'			,
	'BUILDINGCLASS_BANK'			,
	'BUILDINGCLASS_STOCK_EXCHANGE'	,
	'BUILDINGCLASS_LIGHTHOUSE'		,
	'BUILDINGCLASS_HARBOR'			,
	'BUILDINGCLASS_SEAPORT'			
);

Furthermore you can't right-click on them on the tech-tree etc. to get building information.

Looking at the normal bnw XML files it looks as though UnlockedByBelief should be "true" instead. I tried this and it kind of worked. The buildings are now shown but it doesn't say that they cant be purchased with faith any more. The variable UnlockedByBelief is used elsewhere in the code so that may the problem.

I haven't tested (using the "fix") whether it's just a display thing to that the buildings can atually be purchased with faith.

\Skodkim

 

[ExpiredReign]

UnlockedByBelief is a boolean so setting it to "1" or "true" is exactly the same thing.
If you examine any of the booleans used in the XML files with a program like SQLiteSpy they will display as "1" even when the code sets it as "true".

So the error most likely is UnlockedByBelief should NOT be "true" or "1" as this is the main difference between CEG and vanilla when it comes to those buildings.
I'm not sure why those buildings should be UnlockedByBelief anyway. They are routine buildings that any civ needs to build, regardless of any beliefs.

Thank you Skodkim.

 

[stackpointer]

Those are the buildings purchasable by faith for the new reformation beliefs (gold and production buildings).

I just got an idea for a reformation policy that allows you to buy buildings based on era (ancient, industrial, information) type as opposed to function (science, gold, production).

 

[ExpiredReign]

I don't believe the UnlockedByBelief is set in the vanilla for these buildings though which means it is not necessary for those beliefs.

Jesuit Education is the vanilla belief that gives buildings by faith and it appears to not need that tag. So our extra beliefs should be able work without too.


@stackpointer
I am intrigued. Tell us more.

 

[skodkim]

Sorry, but I don't really understand what the solution is here then - if any.

Do you experience missing civilopedia entries too?

\skodkim

 

[EricB]

There's a different table in the XML that defines if a building is purchasable with faith. I don't have the code in front of me or I'd give the name of it, but ExpiredReign is right. If you want to make a building purchasable with faith, then what you need to do is edit that table.

Add rows to that table specifying which belief makes which building purchasable with faith. Not very hard to do. Just a lot of copying and pasting.


The problem is that it's done with BuildingClass.

For example, if you make the BuildingClassMarket available to be purchased with Faith by a Belief called BeliefThrift, then not just the Market is purchasable with Faith. Also a Bazaar (a unique type of Market) is purchasable with Faith. There's no restrictions to which civilization can purchase it either, so civilizations other than Arabia can buy a Bazaar with Faith.

This is the problem that Jesuit Education has in the vanilla game where it makes BuildingClassUniversity purchasable with faith. The Siamese Wat is a type of University, so that happens. By expanding on this concept of making buildings purchasable with faith, Thal has expanded this bug too. We really shouldn't make buildings purchasable with faith until that Jesuit Education bug is resolved.

 

[stackpointer]

I don't believe the UnlockedByBelief is set in the vanilla for these buildings though which means it is not necessary for those beliefs.

UnlockedByBelief is required by vanilla BNW for the science buildings for the Jesuit Education reformation belief.

This is the problem that Jesuit Education has in the vanilla game where it makes BuildingClassUniversity purchasable with faith. The Siamese Wat is a type of University, so that happens. By expanding on this concept of making buildings purchasable with faith, Thal has expanded this bug too. We really shouldn't make buildings purchasable with faith until that Jesuit Education bug is resolved.

This might get addressed in the fall patch, which will hopefully be released within the month.

 

[ExpiredReign]

Sorry, but I don't really understand what the solution is here then - if any.

Do you experience missing civilopedia entries too?

\skodkim

Yes. Exactly as you described.

I can't see how the change you made can "fix" this problem as it is not actually doing anything.
I have not yet made any edits to fix this problem but I am currently running the beta fall patch and am focused on other issues at the moment. Sorry. Try changing them all to false or 0.

EDIT:
@skodkim
While I stand by the statement that booleans should work the same way with true==1 and false==0, on further examination it appears the SQL in the game database does in fact act strangely with regard these things.
I have found numerous booleans that are displayed as either "1 or 0" or "true and false", so your 'fix' may in fact be just that, a fix. Try changing some of them and see if only those buildings show up. If it is the case, then try more.


@stackpointer
Of course you are correct. My mind is elsewhere.

 

[skodkim]

EDIT:
@skodkim
While I stand by the statement that booleans should work the same way with true==1 and false==0, on further examination it appears the SQL in the game database does in fact act strangely with regard these things.
I have found numerous booleans that are displayed as either "1 or 0" or "true and false", so your 'fix' may in fact be just that, a fix. Try changing some of them and see if only those buildings show up. If it is the case, then try more.

Sonehow it does have an effect as shown in the attached screen shots. One of them is with standard CEP files and one with edited Cities\CEC__End.sql:

Spoiler :

UPDATE Buildings SET UnlockedByBelief = true, FaithCost = 1 * Cost
WHERE (FaithCost > 0 AND Cost > 0)
OR BuildingClass IN (
	'BUILDINGCLASS_LIBRARY'			,
	'BUILDINGCLASS_WORKSHOP'		,
	'BUILDINGCLASS_STABLE'			,
	'BUILDINGCLASS_WATERMILL'		,
	'BUILDINGCLASS_WINDMILL'		,
	'BUILDINGCLASS_FACTORY'			,
	'BUILDINGCLASS_CARAVANSARY'		,
	'BUILDINGCLASS_MARKET'			,
	'BUILDINGCLASS_BANK'			,
	'BUILDINGCLASS_STOCK_EXCHANGE'	,
	'BUILDINGCLASS_LIGHTHOUSE'		,
	'BUILDINGCLASS_HARBOR'			,
	'BUILDINGCLASS_SEAPORT'			
);

Theres a lot of references to UnlockedByBelief whecking if its 1, so it may be a bigger problem than just the civilopedia display.

Concerning the Civilopedia display, theres CAT has checks on FaithCost in the buildings section of \Interface\CivilopediaScreen.lua. My lua experience is however very limited.


EDIT: using this "fix" lua.log says that CEC_END fails to load.

\Skodkim

\Skodkim

 

[ExpiredReign]

This is very puzzling.

The question posed in the code is simply "Do I need a belief to be able to build this?" The answer is just an "affirmative or negative" expressed either with words or numerals. As far as the code is concerned, inside the database, there is no difference. Yet, as your edits show the result is somehow related to which way you answer that question.
This shouldn't be ?

However, as my Lua experience is also limited I can't be sure how a Lua check of a boolean such as:

if UnlockedByBelief < 1 then

would work if the value is set as "true" and not "1".

[Note: there is no check like this made in the mod.]

I'll have to ask higher authorities. Thalassicus, help!

EDIT:
I see you checked while I was typing, good.
Now all you have to do is examine where the file fails.
Go back over any edits you made and reverse them and see if the file loads.
Unfortunately the whole file falls over if there is just a small error.
If you have made minimal edits, like just to fix this fault, then change them back and see.
If we get back to the file loading but the pedia errors persist we will have to look elsewhere to fix this.

Thanks for your patience in this.

 

[stackpointer]

EDIT: using this "fix" lua.log says that CEC_END fails to load.

\Skodkim

This "fix" works because the file fails to load in the first place and the buildings aren't assigned the boolean value and faith values. So Expired you don't have anything to worry about on that front.

 

[ExpiredReign]

I can't do anything with the fix at the moment.

I will be able to get to look at the problem a little later tomorrow (my time).

Your help with this is gratefully appreciated in this matter.

Thanks.

 

[skodkim]

Now all you have to do is examine where the file fails.
Go back over any edits you made and reverse them and see if the file loads.

That's quite easy as All I've done is to change the line
UPDATE Buildings SET UnlockedByBelief = 1, FaithCost = 1 * Cost

to

UPDATE Buildings SET UnlockedByBelief = true, FaithCost = 1 * Cost

I've also tried:

UPDATE Buildings SET UnlockedByBelief = 'true', FaithCost = 1 * Cost

but the result is the same - the file still fails to load.

Yes - the good question is whether we want to include this at all. Personally I think the idea that certain buildings are purchasable with faith is good as we need something to spend faith on (espacially mid to late game), but if it's crippled by bugs anyway it may just be better to remove it alltogether.

Another good question is of course why it prevents buildings from showing up in the Civilopedia - looking forward to a fix / more input on that one!

\Skodkim

 

[stackpointer]

I am 73.5% sure that this is a problem with CEP's CivilopediaScreen.lua and not with the reformation belief itself. As you may have noticed, the Civilopedia is pretty buggy right now as Thal adds features into it.

I think the offending code is below and there might be some errors with some logic checks:

Spoiler :

CivilopediaCategory[CategoryBuildings].PopulateList = function()
	-- add the instances of the building entries
	
	sortedList[CategoryBuildings] = {};
	
	function AddArticle(categoryID, entryID, building)
		local article = {};
		local name = Locale.ConvertTextKey( building.Description )
		article.entryName = name;
		article.entryID = building.ID;
		article.entryCategory = CategoryBuildings;
		article.tooltipTextureOffset, article.tooltipTexture = IconLookup( building.PortraitIndex, buttonSize, building.IconAtlas );				
		if not article.tooltipTextureOffset then
			article.tooltipTexture = defaultErrorTextureSheet;
			article.tooltipTextureOffset = nullOffset;
		end				
		
		sortedList[CategoryBuildings][categoryID][entryID] = article;
		
		-- index by various keys
		searchableList[Locale.ToLower(name)] = article;
		searchableTextKeyList[building.Description] = article;
		categorizedList[(CategoryBuildings * absurdlyLargeNumTopicsInCategory) + building.ID] = article;
	end
	
	local sectionID = 0;
	
	if(Game == nil or not Game.IsOption(GameOptionTypes.GAMEOPTION_NO_RELIGION)) then
		--Add Faith Buildings first
		sortedList[CategoryBuildings][sectionID] = {};
		local tableid = 1;
		--Cep
		local query = [[
			SELECT Buildings.ID, Buildings.Description, Buildings.PortraitIndex, Buildings.IconAtlas
			FROM Buildings INNER JOIN BuildingClasses ON Buildings.BuildingClass = BuildingClasses.Type
			WHERE ( Buildings.IsVisible = 1
				AND Buildings.ShowInPedia = 1
				AND Buildings.FaithCost > 0
				AND Buildings.Cost = -1
				AND BuildingClasses.MaxGlobalInstances <> 1
				AND BuildingClasses.MaxPlayerInstances <> 1
				AND BuildingClasses.MaxTeamInstances <> 1
			);]]
		for building in DB.Query(query) do
			AddArticle(0, tableid, building);
			tableid = tableid + 1;
		end
		sectionID = sectionID + 1;
	end
	
	local sql = [[
		SELECT Buildings.ID, Buildings.Description, Buildings.PortraitIndex, Buildings.IconAtlas 
		FROM Buildings 
		INNER JOIN BuildingClasses ON Buildings.BuildingClass = BuildingClasses.Type 
		INNER JOIN Technologies ON Buildings.PrereqTech = Technologies.Type 
		--Cep
		WHERE ( Buildings.IsVisible = 1
			AND Buildings.ShowInPedia = 1
			AND Buildings.FaithCost = 0
			AND BuildingClasses.MaxGlobalInstances <> 1
			AND BuildingClasses.MaxPlayerInstances <> 1
			AND BuildingClasses.MaxTeamInstances <> 1
			AND Technologies.Era = ?
		);]]
	
	local BuildingsByEra = DB.CreateQuery(sql);
	
	
	-- for each era
	--Cep
	for era in GameInfo.Eras() do
	
		local eraID = era.ID + sectionID;
	
		sortedList[CategoryBuildings][eraID] = {};
		local tableid = 1;
		
		for building in BuildingsByEra(era.Type) do
			AddArticle(eraID, tableid, building);
			tableid = tableid + 1;
		end
	
		-- put in all of the buildings that do not have tech requirements in the Ancient Era for lack of a better place
		--Cep
		if(eraID == 1) then
			local sql = [[
				SELECT Buildings.ID, Buildings.Description, Buildings.PortraitIndex, Buildings.IconAtlas 
				FROM Buildings 
				INNER JOIN BuildingClasses ON Buildings.BuildingClass = BuildingClasses.Type 
				--Cep
				WHERE ( Buildings.IsVisible = 1
					AND Buildings.ShowInPedia = 1
					AND Buildings.PrereqTech IS NULL
					AND Buildings.FaithCost = 0
					AND BuildingClasses.MaxGlobalInstances <> 1
					AND BuildingClasses.MaxPlayerInstances <> 1
					AND BuildingClasses.MaxTeamInstances <> 1
				);]]
		
			for building in DB.Query(sql) do
				AddArticle(eraID, tableid, building);
				tableid = tableid + 1;
			end
		end
		
		--Cep
	end
		
	for k,v in pairs(sortedList[CategoryBuildings]) do
		table.sort(v, Alphabetically);
	end
end

I will take a closer look tomorrow.

 

[skodkim]

I am 73.5% sure that this is a problem with CEP's CivilopediaScreen.lua and not with the reformation belief itself. As you may have noticed, the Civilopedia is pretty buggy right now as Thal adds features into it.

I think the offending code is below and there might be some errors with some logic checks:

Spoiler :

CivilopediaCategory[CategoryUnits].PopulateList = function()
	-- add the instances of the unit entries
	sortedList[CategoryUnits] = {};

	function AddArticle(categoryID, entryID, unit)	
		local article = {};
		local name = Locale.ConvertTextKey( unit.Description )
		article.entryName = name;
		article.entryID = unit.ID;
		article.entryCategory = CategoryUnits;
		article.tooltipTextureOffset, article.tooltipTexture = IconLookup( unit.PortraitIndex, buttonSize, unit.IconAtlas );				
		if not article.tooltipTextureOffset then
			article.tooltipTexture = defaultErrorTextureSheet;
			article.tooltipTextureOffset = nullOffset;
		end				
		
		sortedList[CategoryUnits][categoryID][entryID] = article;
		
		-- index by various keys
		searchableList[Locale.ToLower(name)] = article;
		searchableTextKeyList[unit.Description] = article;
		categorizedList[(CategoryUnits * absurdlyLargeNumTopicsInCategory) + unit.ID] = article;
	end
	
	local sectionID = 0;
	
	-- Add units which cost faith to a "Faith" category first.
	if(Game == nil or not Game.IsOption(GameOptionTypes.GAMEOPTION_NO_RELIGION)) then
		local faithUnits = {};
		sortedList[CategoryUnits][0] = {};
		local tableid = 1;
		--Cep
		for unit in DB.Query("SELECT Units.ID, Units.Description, Units.PortraitIndex, Units.IconAtlas From Units where FaithCost > 0 and Cost = -1 and not RequiresFaithPurchaseEnabled and ShowInPedia == 1") do 
			AddArticle(0, tableid, unit);
			tableid = tableid + 1;
		end
		sectionID = sectionID + 1;
	end
	
	-- for each era
	local sql = [[
		SELECT Units.ID, Units.Description, Units.PortraitIndex, Units.IconAtlas 
		FROM Units INNER JOIN Technologies on PreReqTech = Technologies.Type 
		--Cep
		WHERE (Units.FaithCost = 0 or RequiresFaithPurchaseEnabled) and Units.ShowInPedia = 1 and Technologies.Era = ?;]];--Cep
						
	local UnitsByEra = DB.CreateQuery(sql);
	
	--Cep
	for era in GameInfo.Eras() do
	
		local eraID = era.ID + sectionID;
	
		sortedList[CategoryUnits][eraID] = {};
		local tableid = 1;
	
		for unit in UnitsByEra(era.Type) do
			AddArticle(eraID, tableid, unit);
			tableid = tableid + 1;
		end
			
		-- put in all of the units that do not have tech requirements in the Ancient Era for lack of a better place
		--Cep
		if(eraID == 1) then
			--Cep
			for unit in DB.Query("SELECT Units.ID, Units.Description, Units.PortraitIndex, Units.IconAtlas From Units where PreReqTech is NULL and Special is NULL and (Units.FaithCost = 0 or RequiresFaithPurchaseEnabled) and Units.ShowInPedia = 1") do
				AddArticle(eraID, tableid, unit);
				tableid = tableid + 1;
			end
		end	
		
		--Cep
	end
	
	-- sort this list alphabetically by localized name
	for k,v in pairs(sortedList[CategoryUnits]) do
		table.sort(v, Alphabetically);
	end	
end

I will take a closer look tomorrow.

You're looking in the units section of the code (not that I know lua).

\Skodkim

 

[stackpointer]

You are correct, I copied the wrong section of code (they are pretty similar). The right one should be up now. You can see that the buildings are added in sections and something may have been overlooked.

 

[skodkim]

You are correct, I copied the wrong section of code (they are pretty similar). The right one should be up now. You can see that the buildings are added in sections and something may have been overlooked.

I believe you're right - had been looking at exactly that section, but having hardly any lua skills I wasn't really able to do anything about it.

\Skodkim

 

[stackpointer]

Replace line 770 in CAT\Interface\CivilopediaScreen.lua with the following:

AND (Buildings.FaithCost = 0 or Buildings.Cost > 0)

This should fix it. Probably.

 

[skodkim]

Fantastic. Just troed it and it seems to work like a charm

\Skodkim

 

[ExpiredReign]

I am 73.5% sure that this is a problem

I think you may have to re-evaluate that estimate.

This is great, I can go to bed thinking about a problem and wake the next day and it is fixed. Woohoo.
If only every situation worked like that.

Nice work you two.