1. We have added a Gift Upgrades feature that allows you to gift an account upgrade to another member, just in time for the holiday season. You can see the gift option when going to the Account Upgrades screen, or on any user profile screen.
    Dismiss Notice

The Medieval Economy

Discussion in 'Civ4Col - Medieval: Conquests' started by Kailric, Sep 15, 2013.

  1. Lib.Spi't

    Lib.Spi't Overlord of the Wasteland

    Joined:
    Feb 12, 2009
    Messages:
    3,707
    Location:
    UK
    I think for AI for electricity (glad you are talking about this :D) I don't think it would be too hard to give the ai some basic guidance\plans\rules as it is a fairly unique thing and also fairly simple, it just needs to 'know' about it to 'see' it and then it is just X - Y X production and Y consumption and to always aim to keep it positive, then teach it some buildings to build (or fuel yields to get) if it isn't positive, something like that..

    If you can program all the mechanics in the first place, then adding a few rules for the ai should be a doddle :) then just refine the rules if people see the ai 'missing a tirck'
     
  2. Kailric

    Kailric Jack of All Trades

    Joined:
    Mar 25, 2008
    Messages:
    3,095
    Location:
    Marooned, Y'isrumgone
    I agree, in my experience in working with the AI you can generally make the AI at least seem like they know what they are doing while you test things out on the human side. I made the AI start using the Peddler profession and land trade routes pretty easily, more so than I thought. Also, Barbarians and Foreign Cities will even send out their own Traders if they see a player city that is Importing a good it has plenty of.

    To add again to the uses of plot groups, for the most part I wanted it implemented in order to add options for diplomacy. For one example, right now you can negotiate Trade Relations and start gaining Trade Points each turn but we can make it so that there must be a trade route connected (plot group) to at least one foreign city, then each foreign city connected after that will gain a bonus to Trade Points earned, so that some Trade Relations will be more beneficial than others.

    I also want to separate "Trade Relations" with "Open Borders" in that first you would open Trade Relations with a Civ, thus allowing Peddler and non military units to cross borders then as relations improved you can add Open Borders so that all units can move about freely. This would make a bit more sense as no historical Civ of that time would, for the most part, meet a strange new Civ and immediately start letting a mass of military units march through its territory.

    I have to agree with Ray and Nightinggale in that we may be getting ahead of ourselves on some of these ideas with bonus resources. Col has such a unique hands on approach to its production and trade system that its not so easy to add elements from Civ4 so we should take baby steps and add first the things that make the most sense, as Nightinggale has mentioned above.

    I defiantly want to see the core mod though enhanced so that it can support such ideas as Electricity resources and such so that way we can continue our march right on up into the modern age and beyond:scan:
     
  3. Lib.Spi't

    Lib.Spi't Overlord of the Wasteland

    Joined:
    Feb 12, 2009
    Messages:
    3,707
    Location:
    UK
  4. orlanth

    orlanth Storm God. Yarr!

    Joined:
    Nov 17, 2001
    Messages:
    1,776
    Yeah it's not worth adding too much complication to it; as NG says the resource plotgroup access could be useful mainly in situations where you'd want to check the number of resources or resource buildings connected.

    In github 2071 as you may have noticed there is a YIELD_ENERGY & I'd actually planned to treat that as a Yield as ray mentioned - my tentative idea has been to use a non-stored and non-transported virtual yield (like Crosses/Hammers etc) which gets consumed as a secondary input by several of the advanced manufacturing Professions. A given amount of Energy per turn would be produced for free by a Reactor building in each colony, so you can only handle a certain volume of intensive manufacturing before needing to upgrade your Reactor to a higher tier.

    In a modern/industrial age mod you could consider Electricity production by a Profession (ie converting Coal to Electricity) but then for balance reasons it becomes far too costly, difficult, and disproportionately important in relation to the other yields it produces (it should be a very tiny fraction of economic value added, inconsistent with being a production chain).

    I actually planned a nontransportable virtual yield and hadn't considered the possibility of having YieldGroupShared in plotgroups; that could be an interesting feature if its feasible though I didn't know if we want to go that far in using plotgroups. If these are all virtual yields I think they wouldn't be stored from turn to turn.

    In old 2071, it actually did locally consume nonstorable virtual yields like Propaganda (Crosses) and Liberty as inputs for researching certain techs in the Sociology category. This worked fine for humans though I'm not as sure about AI (it used the earliest version of Inventor modcomp where I think the AI got an exemption from using yields as a research input.)
     
  5. Nightinggale

    Nightinggale Deity

    Joined:
    Feb 2, 2009
    Messages:
    4,354
    I have been thinking about this a bit more and I have come to the decision to remove bonus resources from plotgroups. Because we might need them later I will do it by defines and leave most of the code for it intact while the compiler will ignore most of it. This way we can easily re-enable the code while we will not affect performance when it is disabled.

    If we ever enable it is another question, but I see no reason to delete the already committed code to handle this as we simply do not know if it will be needed.

    This shouldn't be too tricky to implement. Plotgroups spreads on all plots not owned by somebody you are at war with. They should be able to spread to cities not belonging to yourself.

    To be honest I always disliked the merge of those two. If I build cities across a continent, then I certainly will NOT open borders as the land behind the blockade is mine alone.

    I would say that first step would be what Japan did during it's years of self chosen isolation (to prevent European colonial powers). They gave a small island to the Netherlands, which then became a Dutch colony/trading port. They built a bridge from the mainland to the island and declared that nobody from the island were to cross the bridge. On the other side only a select few were allowed to cross in order to actually do the trading. This way they practiced a well functioning closed border trade for around 250 years.

    To implement this in the game I would say that we could make open border for trading units, but only in the area of influence of a single selected city. You can then trade with somebody without allowing them to explore your entire kingdom. It would be kind of like being allowed in a store, but not the backroom.
     
  6. Nightinggale

    Nightinggale Deity

    Joined:
    Feb 2, 2009
    Messages:
    4,354
    A little update.

    I used #ifdef USE_PLOTGROUP_RESOURCES to disable all bonus resource code. Next I added a python group for PlotGroups meaning data is now accessible from python. Surprisingly civ4 lacks a class for plotgroups and I had to figure out how the system works and make one from scratch.

    I added the number of plots in the plotgroup to the city screen. It is the first line in the export yield list. The location was chosen purely because it was the easiest location from a coding point of view. It isn't a permanent solution. Right now it shows 0, which mean the DLL isn't done yet. Hopefully some day in the near future it will show the correct number and print a list of connected cities.

    I wonder about one thing in the python interface. When you from python declares a class, it allocates the class in the DLL. However there is no code to free the class again. Either it happens automatically in the python library or we have a memory leak. If it is the latter, then we leak 4 bytes each time we use a class in python. Granted that isn't much, but I still find it odd if the vanilla is designed to leak memory :confused:
     
  7. Nightinggale

    Nightinggale Deity

    Joined:
    Feb 2, 2009
    Messages:
    4,354
    I finished Plotgroups
    :dance::dance::band::dance::dance:
    :cheers::cheers::cheers:

    Plotgroups are assigned to each city, spreads on roads and merges when two cities are connected. Owner of the plot doesn't matter as long as you aren't at war with the owner. In fact it is perfectly ok to connect to native cities and use those to reduce the number of road plots you have to build.
    I disabled rivers and terrain for spreading a plotgroup. The reason is that the goal went back to basics, which is to tell which cities are connected by road. If we can all of a sudden spread a plotgroup across water, then using plotgroups to find connected cities for transports becomes pointless. Also it isn't just a matter of enabling the code again as it is only half implemented and it managed to crash the game before I disabled the non-road spread.

    Saving and loading works. However from the looks of it, making it load old savegames is tricky. The problem isn't fixing the loading code, but the fact that it would leave the game without existing plotgroups to build on. This mean it would need new plotgroup build code, which is called after map, cities and units are loaded. I don't know a suitable location for code like that. Presumably this is inside the exe itself :(
    On the other hand I might debug loading to figure out how the game actually loads data. It could be useful to know order of classes and so on.

    EDIT: Did some debugging. It appears like the last read function called by the exe is CvPlayerAI::read(). Adding "if (m_eID == (MAX_PLAYERS - 1))" at the end of it should only trigger when the last player is read and as such we can call a function, which is only called when the entire savegame is loaded. The correct place to write this would be a few lines later inside the exe, but this hackish approach should work too.

    It ended up in a way where we can ask if two cities are connected, but we can't get a list of connected cities (this is how civ4 works). My plan for domestic sales relies heavily on such a list, which mean I have to design an efficient way of getting such a list. The main problem with a list like this is that it will have to be saved or the design will desync network games.

    I'm thinking as we might as well break savegames now, then we could clean up the load code and kill the unclean code for loading old savegames. This will make code reading and modding easier in the future. I'm considering other cleanups of the saving code as well and I might as well do it now as it will be a lot easier if I don't have to worry about old savegames.

    Testing:
    The code is not really tested yet. It would be nice if I'm not the only one to do that.

    @Trade Winds:
    This could actually be a really good task for you. People not knowing the code and possibly without the background to understand it is often used professionally to test the code in question. The reason is that development talk often "pollutes" the mind of the developers to test the usage they considered when coding. Fresh eyes without knowledge of the internals are free of this problem and tend to be better at testing the cases not considered when designing.

    Getting the code in question is a matter of reading the guide (WIP)
    https://github.com/Nightinggale/Medieval_Tech/wiki/GIT-start-guide
    https://github.com/Nightinggale/Medieval_Tech/wiki/GIT-usage (you only need the branch selection from this one)
     
  8. Nightinggale

    Nightinggale Deity

    Joined:
    Feb 2, 2009
    Messages:
    4,354
    I fixed the savegame loading and old savegames can be loaded once again.

    To do this I added a function to CvGameCoreUtils.cpp at the bottom:
    This could be useful later.

    I also committed all the changes to master branch and pushed it. After all now that the old savegames can be loaded and plotgroups appears to work, there is no reason to keep a separate branch for them. The whole idea with the branch was that it could break savegames and I didn't want to do what more than I have to in the master branch.

    Next step is to get the list of cities in a plotgroup and plotgroups ready to be exploited :woohoo:

    Also where did you go :confused:
    I feel like this thread became a developer's diary rather than actual communication between people :mischief:
     
  9. orlanth

    orlanth Storm God. Yarr!

    Joined:
    Nov 17, 2001
    Messages:
    1,776
    Well I'm still here, polluted mind and all! :devil: Wow congrats on finishing such a major feature. :bowdown::hatsoff:I will DL it for some testing.

    If your group spreads to anyone you aren't at war with, won't it include foreign cities? I guess thats ok actually as long as domestic markets can still be limited to your own cities (actually maybe cross-border trade could even be allowed under certain laissez-faire Civics.. I know I consume a lot of Canadian beer :p:beer:)

    Polluted minds, plots to be exploited, fresh eyes without knowledge of internals.. sounds vaguely like a Clark Ashton Smith story.:satan: Elder Gods modcomp ftw! :devil:
     
  10. Nightinggale

    Nightinggale Deity

    Joined:
    Feb 2, 2009
    Messages:
    4,354
    I knew of your polluted mind the minute I realized you added uranium to the colo engine and then enriched it :nuke::lol:

    You make a very good point here. Civ4 can make traderoutes to cities owned by other players using this feature. However I actually planned on just making a list of cities in the plotgroup owned by the plotgroup owner. Each plot can hold up to the same number of plotgroups as there are players, but a player can't have more than one plotgroup in a single plot.

    Rather than caching all cities in the plotgroup, I think I will stay with just owners plotgroups. This makes looping his own cities a lot easier and somewhat faster.

    Using this to trade between different civs. That is actually a good idea and finding a list of cities for the other player wouldn't be hard.

    Say we have player 1, who has a trade agreement with player 2.
    Player 1 wants to trade from plotgroup A and wants to search for cities owned by player 2 in that plotgroup. The calculation is done as follows:

    -Player 1 looks up plotgroup A and finds a plot in the plotgroup (any plot will do)
    -Fetch the plotgroup (B) from that plot where owner is player 2
    -If plotgroup B isn't null, then we know that plotgroups A and B covers precisely the same plots (except in case of war).

    We now have two lists of cities. A has player 1 cities while B has player 2 cities. If we want to know for certain that a player 2 city is in plotgroup A (accounts for war etc), then we can ask if the plot for that city is in plotgroup A.

    All these checks are virtually instant and we can do them freely without performance penalties. The question is if we can find something useful to use it for, but technically we can do with minor effort (given that the city list is implemented in plotgroups). I will not code anything to help for this usage unless we figure out a useful feature to use this for.

    Missions:
    Maybe missions can become more effective if they are in the same plotgroup as a church or whatever. Attracting converts would be easier if they have a road to travel on rather than if they have to walk though dense with wild animals, mouontains etc. In fact the simplest check here would be to see if a plotgroup exist for the mission owner in the mission plot. We will not even need additional "core code" to allow a check like this as all the needed data and access functions are already available.
     
  11. Trade Winds

    Trade Winds Warlord

    Joined:
    Nov 21, 2013
    Messages:
    235
    I am really gratefull about your work and I'm sure I am not alone.

    Don't feel like that. I myself may be off for a couple of days but I often keep an eye here.

    And yes, I am pleased to test the plotgroups, It will be fun and also I hope it could be of some help. I will test it over the weekend because I have more time to follow your guide to the github and test the plotgroups.:clap::D
     
  12. Lib.Spi't

    Lib.Spi't Overlord of the Wasteland

    Joined:
    Feb 12, 2009
    Messages:
    3,707
    Location:
    UK
    hehe Night, most of the time when you start talking my eyes glaze over and my brain trickles out of my ears :D so I have come to the point of 'skimming' most of what you say and waiting for you to finish and it gets released so I can play with it in a slightly less headache inducing system :D
     
  13. Kailric

    Kailric Jack of All Trades

    Joined:
    Mar 25, 2008
    Messages:
    3,095
    Location:
    Marooned, Y'isrumgone
    This is a great idea. It could also increase the effectiveness of Trading Posts in other cities as well as help Pilgrims along and increase their frequency.

    You are working so much faster than me, but great work so far! :goodjob:
     
  14. Nightinggale

    Nightinggale Deity

    Joined:
    Feb 2, 2009
    Messages:
    4,354
    I looked into caching the cities in a plotgroup. Fundamentally it is fairly simple. Sure it needs to be saved and stuff, but that's not the problem. The problem is keeping the list up to date. It turns out that there are a lot of operations, which might have to update the list and all of a sudden it looks like maintaining the cache can use up more resources than it saves. Also finding all the places where it has to update is tricky as they don't have to be the same places as plotgroup updates. One example would be building a city on a plot with a road. That plot is already in a plotgroup, which mean the plotgroup will not update. Another problem with a cache this complex to keep up to date is that even if it becomes bugfree (hard task), it might not stay that way for long as it is easy to break when modding something, which looked like unrelated code.

    I came up with a plan B. Inside CvPlayer::doTurn() I made a vector of a vector of city pointers. In other words it's a 2D array, which can resize dynamically at runtime to fit city and plotgroup count of a player. The first index is plotgroup ID while the second is the cities in the plotgroup in question.

    This will allow using the cache in doTurn(), which is precisely what the plotgroup shared domestic market needs without the expensive need to maintain the cache.

    I consider moving this temporal cache into CvPlayer itself. This way whatever is called by CvPlayer::doTurn() can use it (for example CvCity::doTurn()). I don't know what it should be used for right now, but adding it right now appears to be minor extra work. Memory usage is not really an issue either, specially not if doTurn() clears it when it is done. The danger is that somebody might call it outside doTurn(), which will cause a bug. Maybe an assert is enough to prevent this. After all that is the safety level of a bunch of other functions in regard to incorrect usage.

    This will not provide a cache outside doTurn(). However I imagine outside usage to be something like this:
    Say we need to know if a plot(with or without a city) is connected to a city with a church.
    The code will then loop all owner cities and use bool CvPlot::isConnectedTo( const CvCity* pCity). This will check if the two plots in question belongs to the same plotgroup and is a fairly fast calculation. It might even be able to break fast as the church is found quickly.

    The performance issue without a cache in doTurn is that for the market code, the number of times the cities are looped is 2*num of plotgroups and all cities are looped. This mean the total amount of cities fetched by the loop is 2*(num plotgroups)*(num cities). With 25 cities not connected (think native civ) it becomes 2*25*25=1250. With the cache it becomes 2*(num cities)=2*25=50. It's well worth the cache, which loops all cities once to be generated.

    What is that "but" doing there? Are you saying that you would expect anybody coding faster than you to do a worse job than you :confused:
     
  15. Trade Winds

    Trade Winds Warlord

    Joined:
    Nov 21, 2013
    Messages:
    235
    I have been testing the plotgroups. It works as Nightinggale showed: every plot with a road is added to the counter under the left bottom Import button. But I found two crashes very early in the game, one in turn 175 or so with 4 cities connected with roads adding about 12 to the plotgroup count; and another crash on turn 85 in a different game with only two cities and no roads. It has been so early in the game that I have been able to test it twice in little time. I didn’t even have time to build a marketplace and share resources or whatever. I don’t know if you want any other details about the testing.

    Something I imagine that could be done with this, when we have a list of foreign cities connected to your plotgroup, it is for example to learn a foreign profession present in the cities in the plotgroup in one of your connected school houses (for an international cost), or ask a yearly tithe on goods or population if the foreign attached civilization is a vassal o as a demand from war.

    But about checking road plots, I remember reading on civilization forums that somebody was moding the game in such a way that the computer checked every turn if some resources were present within the city radius. As new resources appeared with technology advances and new cities were created, all the plots had to be checked every turn, and they said that the game slowed a great deal. So checking road plots may be very slow when big plotgroups are present in latter stages of the game.:(
     
  16. Nightinggale

    Nightinggale Deity

    Joined:
    Feb 2, 2009
    Messages:
    4,354
    Can I get a savegame from just before such crashes? It would help quite a lot in tracking down what could cause such crashes. I have a faint idea of the concept of the crashes, but not where in the code they come from. Letting the game crash in the debugger usually tells precisely where it goes wrong.
     
  17. Kailric

    Kailric Jack of All Trades

    Joined:
    Mar 25, 2008
    Messages:
    3,095
    Location:
    Marooned, Y'isrumgone
    Heh, the "but" is about being late at night and I didn't catch that. You understand English better than me:)
     
  18. Nightinggale

    Nightinggale Deity

    Joined:
    Feb 2, 2009
    Messages:
    4,354
    I figured out how to cache the cities in a plotgroup. The answer turned out to be dead simple. The tricky part was to keep the cache up to date when conditions changed. The simple answer is: if it is too hard, then don't do it :lol:

    Instead I added a bool to tell if the cache is ok. I then narrowed down when the cache is outdated, which is when the player adds or remove a city and when merge or split plotgroups. These locations have no idea how the cache broke, just that it is likely broken. When the cache is accessed and it is out of date, it will be recalculated automatically by the cache access functions.

    I pushed this change to github where it is now no longer looping cities in the order the player gained them. Instead it loops the plotgroups and for each plotgroup it loops the cities. Next step is to use this change to share the domestic market. However first I will look at the crashes.
     
  19. Nightinggale

    Nightinggale Deity

    Joined:
    Feb 2, 2009
    Messages:
    4,354
    After a lot of debugging it turns out that the crashes are due to AI pillaging roads. This cause a crash because the data inside GC.getPlotGroupFinder() isn't allocated and data is read from a NULL pointer. Civ4 allocates it with this line
    Code:
    gDLL->initGlobals();	// some globals need to be allocated outside the dll
    M:C has it too, but initGlobals() is a function inside the exe and only the civ4 exe inits this data. There is no way to get the colo exe to do it. I tried to work around it and init it in the DLL despite the comment, but it turns out that the class FAStar is defined in an exe only header. All DLL header references to it just tells that there is a class called that, but what it is meaning we can't allocate it outside the exe.

    We are so screwed regarding getting vanilla civ4 plotgroup code to work in M:C [pissed]

    If I read the offending code correctly, what it does is that it takes plot A, which is the plot, which just lost the road. It then looks at all the plots around it and those inside the plotgroup will try to find a path to the head plot (first plot in the plotgroup), traveling only on plots allowed by trade network. If at least if at least one plot fails to connect, then it goes into "error mode" as it knows the plotgroup has been split into multiple plogroups. It is only the check, which is the problem, not the plotgroup splitting code.

    The only thing I can think of right now is to look into RaR and look at the train code. The train pathfinder finds a path between two plots using only railroad plots. A modified version of this pathfinding into using only plots with roads (all types except no road) will be able to provide the same answer as we need here. Using this approach will forever prevent us from spreading plotgroups across anything but roads. I considered expanding to rivers as well, but that appears to be out of reach due to exe differences. Plot type spreading (across water etc) is also extremely difficult.

    I have no idea when I will be able to solve this. I'm considering making a dirty hack in the meantime. Instead of checking, it can just assume failure and split the plotgroup, which then grows and reconnects if splitting was a mistake. This is slower than checking, but at least it will not crash. I have no idea how much slower it will be, but I have a bad feeling about it because Firaxis appears to have spent a great deal of time making a function to check instead of just recalculate. I don't think they would have done that if it wasn't a performance issue to begin with.


    Note (to self?) about performance in a homemade check:
    Loop all neighbor plots.
    Have a null pointer, which is set to point to the first plot in plotgroup found.
    Any other plots in plotgroup has to be able to find a path to this plot

    Presumably the paths using this approach will be really short and as such found with as little CPU time as possible. Head plot could be far away in a maze of branching roads
     
  20. Kailric

    Kailric Jack of All Trades

    Joined:
    Mar 25, 2008
    Messages:
    3,095
    Location:
    Marooned, Y'isrumgone
    If this crash bug is present here then it should have been present in my Forts mod and the Westward Ho mod correct? I don't recall this being brought up, but I do know my Forts mod wasn't tested to the fullest so something like this could have been over looked.

    Anyway, I am sure we will come up with some kind of solution.
     

Share This Page