VP Community Roadmap/Vision Brainstorming

What are the current options for modding?
- extra files for adding
- whole file overwrites for changing/deleting
- hooks

Diffpatching is what users manually do now if they want to port mods to the newer version. It looks like this:

Diff:
void CvTacticalAI::ExtractTargetsForZone(CvTacticalDominanceZone* pZone)
{
-    int iMaxRadius = GetTacticalAnalysisMap()->GetMaxZoneRadius();
+   int iMaxRadius = GetTacticalAnalysisMap()->GetMaxZoneRadius() + 1;

So a new VP release might not break old mods, since the mod itself is just a few lines.

It would help if the files were more modular. I would like to see more folders and smaller file sizes for organization and easier overriding.

The "system" would look for mods and apply the diffpatch. It would be easy to spot conflicts.
 
Regarding AI though machine learning, @Recursive would it be feasible to add an option (probably in "Interface" tab, disabled by default) to record a game state and action taken in the game state in json? It'd allow as to collect data on choices from high skilled players, so then it could be used for training AI models for many different tasks, like choosing technology/policy/building. Maybe even how to control units, although the current approach with simulations is very good.
 
So a new VP release might not break old mods, since the mod itself is just a few lines.

It would help if the files were more modular. I would like to see more folders and smaller file sizes for organization and easier overriding.

The "system" would look for mods and apply the diffpatch. It would be easy to spot conflicts.
VP was modular back then, but since not a lot of people playing around things to their liking, the mod ended up merged several separated stuff for better updating process and dependency. I don't see reverting it back is an option as of now.

People also like that VP already setup stuff for them all at once. No need to manually update stuff that are changed to the right folder, which were confusing as heck when the mod was separated into several files and folders.

One thing that could be improved though is to make a patch-only installer while also providing the usual VP installer.
So then each super major update, the mod provide an installer for installing/reinstalling the whole mod. But on each smaller updates with nothing major on it, there's a patch-only installer that only contains the DLL and some stuff that are changed, nothing else.
 
VP was modular back then, but since not a lot of people playing around things to their liking, the mod ended up merged several separated stuff for better updating process and dependency. I don't see reverting it back is an option as of now.

People also like that VP already setup stuff for them all at once. No need to manually update stuff that are changed to the right folder, which were confusing as heck when the mod was separated into several files and folders.

One thing that could be improved though is to make a patch-only installer while also providing the usual VP installer.
So then each super major update, the mod provide an installer for installing/reinstalling the whole mod. But on each smaller updates with nothing major on it, there's a patch-only installer that only contains the DLL and some stuff that are changed, nothing else.
Was it? I want to see this ancient manuscripts :) Is there anywhere this modular version?
 
Regarding AI though machine learning, @Recursive would it be feasible to add an option (probably in "Interface" tab, disabled by default) to record a game state and action taken in the game state in json? It'd allow as to collect data on choices from high skilled players, so then it could be used for training AI models for many different tasks, like choosing technology/policy/building. Maybe even how to control units, although the current approach with simulations is very good.
This should be feasible, yes.
 
It's should be Github issue, not a VP proposal, right?
Sorry, I meant it's feasible as a longterm goal. It can't be feasibly added right now. We'd need a revision of how logging works.
 
Was it? I want to see this ancient manuscripts :) Is there anywhere this modular version?
The old-old version when the mod was called CBP (Community Balance Patch)/CBO (Community Balance Overhaul). It was years ago.
 
Should we have more than one person responsible for making releases? I can offer to make them too, I know now how to do it. @Recursive What do you think?
 
Without judging any ideas, here is the summary of everything in the thread, plus my own brainstormed goals, and a few mentioned on the Discord server.

Next, I'm going to categorize the ideas.

Declared Goals
Code:
- Remove all unnecessary #ifdefs in the code (azum4roll)
- Rewrite some of the Lua files, both EUI and non-EUI (azum4roll)
- If desired, integrate the EUI utility functions into non-EUI so that non-EUI can also use them (azum4roll)
- Create a Community Patch Kit (Vaderkos)
- Assist with multiplayer stability (Vaderkos)
- RE: Modmods Repository - "we need some kind of organization (sub forums for civs, QOL, overhaul-level changes, etc.) and then also moderation to archive things that aren't being maintained" (hokath)
   - Suggestion that we just have the following subs: released mods, WIP mods, and archive/retired
   - From there have the colored labels on each to denote civs, QoL, game rules etc. Like how the Congress threads get color labels
   - Also if possible, a label that will tell the last compatible version of VP
- Try to do AI API with machine learning (CppMaster)
   - Way more feasible to train an AI for building, policy, tech selection than a complete AI

Potential Goals
Code:
- Create a GitHub Wiki or Mediawiki to replace the FANDOM wiki
- Use AI to generate higher quality artwork and leader dialogue
- Tidy up the modmods repository
- Change documentation (recent & total)
- Document Lua API
- Document database
- Make events system easier to understand
- Separate active and inactive developers on Discord
- Contract art assets / code
- Upgrade EUI to latest version (rename it if doing so)
- Make EUI entirely DLC
- Make VP multiplayer compatible
- Add SQL Triggers
- Better categorization of VP Congress proposals and/or a proposal limit
- Decide whether 43 Civ and non-EUI versions should continue to be developed
- Upgrade the tools used to compile the code (not sure if possible)
- Make the code less fragmented and repetitious
- Use commintlint for better changelog documentation
- Display civilization icons on the bottom of the screen rather than the top right corner (EUI)
- Separate logging between games
- Superuser AI that uses machine learning (will require a custom API)
- Rework the event system as a community
- Improved communication between modmodders and developers, including required communication for database and Lua changes
- Mark features as deprecated and then delete them in a future release
- RE: mod compatibility - diffpatch for Lua files
- Use the unciv repo as inspiration for code organization
- Multiple machine learning AI wars
- Make a VP Lite version
- Continue removing null checks for impossible scenarios (and similar garbage cleaning)
- Record a game state (Interface option) in JSON - would require significant changes to logging
- Make the mod more modular
- Make a patch-only installer for savegame compatible versions
- Have more than one person responsible for making releases - axatin has offered
- Make the Discord server more accessible

My Ideas (mostly; a few were lifted from previous discussions with others in the community)
Code:
- More support for new contributors to the mod
- Regularly run games for stats (expansion/formalization of L. Vern's work)
- Discussion on how to organize the subforum
- Diplomacy AI guides
- DoContactMinorCivs() rework
- DoContactMajorCivs() rework
- Deal AI Documentation / Rework
- Custom Leader Diplomacy
- Clear GitHub Issues Backlog
- CustomModOptions Sweep
- (1) Community Patch Sweep
- Attempt to combine 43 civ and standard versions into one
- Improve AI for
    - Tech selection
    - Policy selection
    - Belief selection
    - City production selection
    - Promotion selection
    - Victory pursuit selection
    - ...and document all of it
- General DLL cleanup, including:
   - Better handling of teams vs. players
   - Memory value consolidation
   - Traits cleanup
   - Lua methods cleanup
   - Memory objects (some aren't necessary; City-States and Barbarians don't need a DealAI or DiplomacyAI; major civs don't need a MinorCivAI)
   - Cache overhaul (CvTacticalAI, TradePathCache, DangerPlots) - separate UI from DLL core caches (might not be possible?)
   - Overall memory consumption investigation and reduction
   - Game State Hash (makes debugging multiplayer easier)
   - Database anti-NULL (cleans up the database on the DLL side so there aren't empty entries, which can cause crashes)
- "Restart game with new map" button for multiplayer (unsure if possible)
- Automatic modpack creation
- Automatic mod/DLL building
- Boldness Flavor Balancing (skewed high)
- Non-Diplomacy Flavor Balancing, and fix the system so it works well
- Fulfill the Permanent Pantheon proposal, which was passed prior to the creation of the VP Congress but never implemented
- Fulfill the Recon Rework proposal that was passed but never implemented
- Bug Report Button ingame
- New Policies re: GitHub (making bug reports, handling pull requests, handling issues etc.) after reviewing the existing situation
- Use Clang to build the DLL instead of Visual Studio
- Expand the Civilopedia
- Ingame documentation of added abilities
- Instrumentality Project
 
Last edited:
Categorized Ideas

Also added eighteen (NEW) ones that came to mind. The next step will be to judge all the ideas.

AI Improvements
Spoiler :
- Superuser AI that uses machine learning (will require a custom API)
- Try to do AI API with machine learning (CppMaster)
- Way more feasible to train an AI for building, policy, tech selection than a complete AI
- Multiple machine learning AI wars
- DoContactMinorCivs() rework
- DoContactMajorCivs() rework
- Deal AI Rework
- Custom Leader Diplomacy
- Improve AI for
- Tech selection
- Policy selection
- Belief selection
- City production selection
- Promotion selection
- Victory pursuit selection
- Boldness Flavor Balancing (skewed high)
- Non-Diplomacy Flavor Balancing, and fix the system so it works well
- Minor Casus Belli system (NEW)
- Train AI to build Citadels while at war (NEW)
- Train Shoshone AI to pick Ancient Ruins (NEW)
- Train Polynesia/Rome/Spain AI to use their units' secondary functions (NEW)

Compatibility with Modmods
Spoiler :
- Tidy up the modmods repository
- RE: Modmods Repository - "we need some kind of organization (sub forums for civs, QOL, overhaul-level changes, etc.) and then also moderation to archive things that aren't being maintained" (hokath)
- Suggestion that we just have the following subs: released mods, WIP mods, and archive/retired
- From there have the colored labels on each to denote civs, QoL, game rules etc. Like how the Congress threads get color labels
- Also if possible, a label that will tell the last compatible version of VP
- Add SQL Triggers
- Improved communication between modmodders and developers, including required communication for database and Lua changes
- Mark features as deprecated and then delete them in a future release
- RE: mod compatibility - diffpatch for Lua files
- Make the mod more modular
- Make a patch-only installer for savegame compatible versions


Discord Server
Spoiler :
- Separate active and inactive developers on Discord
- Make the Discord server more accessible
- Appoint moderators for the Discord server (NEW)
- Clean up project channels on the Discord server (NEW)
- More streaming on the Discord server (NEW)

Documentation
Spoiler :
- Create a GitHub Wiki or Mediawiki to replace the FANDOM wiki
- Change documentation (recent & total)
- Document Lua API
Add extra validity checks for Lua API, remove from general usage (NEW)
- Document database
- Make events system easier to understand
- Use commintlint for better changelog documentation
- More support for new contributors to the mod
- Diplomacy AI guides
- Deal AI Documentation
- Document improvements to other AI
- New Policies re: GitHub (making bug reports, handling pull requests, handling issues etc.) after reviewing the existing situation
- Expand the Civilopedia
- Ingame documentation of added abilities

Memory, Logging & Bugfixing
Spoiler :
- Separate logging between games
- Continue removing null checks for impossible scenarios (and similar garbage cleaning)
- Record a game state (Interface option) in JSON - would require significant changes to logging
- Regularly run games for stats (expansion/formalization of L. Vern's work)
- Clear GitHub Issues Backlog
- Memory value consolidation
- Memory objects (some aren't necessary; City-States and Barbarians don't need a DealAI or DiplomacyAI; major civs don't need a MinorCivAI)
- Cache overhaul (CvTacticalAI, TradePathCache, DangerPlots) - separate UI from DLL core caches (might not be possible?)
- Overall memory consumption investigation and reduction
- Database anti-NULL (cleans up the database on the DLL side so there aren't empty entries, which can cause crashes)
- Bug Report Button ingame
- Master table for AI difficulty bonuses (Stalker0) (NEW)
- Automated telemetry branch for AI training (NEW)

Multiplayer
Spoiler :
- Assist with multiplayer stability (Vaderkos)
- Make VP multiplayer compatible
- Automatic modpack creation
- Game State Hash (makes debugging multiplayer easier)
- "Restart game with new map" button for multiplayer (unsure if possible)

Organization & Consolidation
Spoiler :
- Remove all unnecessary #ifdefs in the code (azum4roll)
- Rewrite some of the Lua files, both EUI and non-EUI (azum4roll)
- If desired, integrate the EUI utility functions into non-EUI so that non-EUI can also use them (azum4roll)
- Create a Community Patch Kit (Vaderkos)
- Decide whether 43 Civ and non-EUI versions should continue to be developed
- Upgrade the tools used to compile the code (not sure if possible)
- Make the code less fragmented and repetitious
- Use the unciv repo as inspiration for code organization
- Have more than one person responsible for making releases - axatin has offered
- Discussion on how to organize the subforum
- CustomModOptions Sweep
- (1) Community Patch Sweep
- Attempt to combine 43 civ and standard versions into one
- General DLL cleanup, including:
- Better handling of teams vs. players
- Memory value consolidation
- Traits cleanup
- Lua methods cleanup
- Automatic mod/DLL building
- Use Clang to build the DLL instead of Visual Studio
- Instrumentality Project

VP Congress
Spoiler :
- Better categorization of VP Congress proposals and/or a proposal limit
- Fulfill the Permanent Pantheon proposal, which was passed prior to the creation of the VP Congress but never implemented
- Fulfill the Recon Rework proposal that was passed but never implemented
- Fulfill my unfulfilled proposals from Session #6 (NEW)
- Change CST to CDT because azum4roll got me with his pedanticness (NEW)
- Reconsider VP Congress thread labels
(NEW)

Other
Spoiler :
- Use AI to generate higher quality artwork and leader dialogue
- Contract art assets / code

- Upgrade EUI to latest version (rename it if doing so)
- Make EUI entirely DLC
- Display civilization icons on the bottom of the screen rather than the top right corner (EUI)
- Rework the event system as a community
- Decide on a standard process for stacking modifiers (NEW)
- Rebalance difficulty bonuses as a community (NEW)
- Make deals start at the beginning of the next turn instead of instantly (NEW)
- Make a VP Lite version
- Detach the gamecore and create a separate engine to make AI tests much faster since they have no graphics
(NEW)
- DLL integration for 4UC (NEW)
 
Last edited:
Judgement of Ideas, Part 1
To begin, I will veto ideas that I think are not improvements, and explain why. These will be crossed out in the Categorized Ideas post above.

A negative judgement of the idea doesn't reflect anything about its proposer - this is brainstorming, so coming up with as many ideas as possible is to be encouraged. There will always be some bad ones along with the good.


Vetoed Ideas
- Multiple machine learning AI wars
It was acknowledged as a crazy idea. As cool as it would be, training competing machine learning AIs would be an inefficient use of resources, and I don't see much practical value with AI wars. However, we could make variations of the same AI - i.e. have AI variant A control one player with strategy A, have AI variant B control another player with strategy B, and see who wins. That could be part of the machine learning process.

- RE: mod compatibility - diffpatch for Lua files
Problems with this idea:
  • How is it supposed to update all of the mod files on CivFanatics?
  • If it updates the files locally (whether on a user's PC or on GitHub or something), how will we identify which mods can be changed and which ones can't?
  • How will we identify when a mod is compatible and when it isn't?
  • If it can be done locally, won't some users end up with compatible versions and others won't? That could make debugging challenging.
  • Security concerns with a program that edits other mods (executable could be flagged by antivirus software etc.).
  • Feels like a more convoluted solution than just communicating when the mod has changed Lua and/or database files like axatin proposed.

- Mark features as deprecated and then delete them in a future release
- Make a VP Lite version
axatin effectively refuted these:
I‘m afraid that if we did it that way it would often be forgotten to remove the deprecated features and we‘d end up with a lot of non-functional columns in the database.

How would modders be notified? If they don‘t notice they need to change something, their mod would work incorrectly for a few months. When the old feature is eventually removed, it‘ll be the same situation as we have now. It‘s annoying if mods are not compatible with a new version, but at least it makes it clear something needs to be done.
The complexity is a result of trying to make the game more balanced, which is done by fine-tuning properties of buildings and policies. If you have any ideas how complexity can be reduced without it negatively affecting balance, feel free to discuss them here on the forum.

A light version isn't planned, maintaining two versions in parallel would require a lot of effort.


- Make the mod more modular
I think this is a step in the wrong direction. There's already debate about whether we should support some existing options, and it might be possible to change the DLL to make the 22-civ and 43-civ versions the same. The more versions of the game we need to make compatible, the more dev resources are spent on compatibility rather than core development (not to mention the work required in decoupling).

I'm not opposed to creating a CustomModOption for relatively isolated changes here and there that people might want to turn off, though.


- Make a patch-only installer for savegame compatible versions
We already have enough people who run into installation problems despite the autoinstaller. This would multiply that by a factor of three, I predict.



Dubious Ideas
These ones are not quite as bad but have some noteworthy problems:

- Use AI to generate higher quality artwork and leader dialogue
- Contract art assets / code
AI artwork is a very polarizing subject among artists, who tend to take a negative view of it. Furthermore, there are legal questions surrounding AI models and copyright violations. While these haven't been settled yet, I'd be reluctant to introduce something that potentially violates copyright into the mod, especially because we might then legally need to scrub it from the Git history.

Contracting someone to create art assets or code is along the same lines - bringing money into the development process is problematic for the same reasons as accepting donations would be. There's questions about whether contracting code would be legal given the license agreement, as well.


- Detach the gamecore and create a separate engine to make AI tests much faster since they have no graphics (NEW)
This idea has come up before. Unfortunately, it's a very large expenditure of resources, and I'm not sure if it would be legal to do given the license agreement. It's true that it would be great if we could pull it off, but the number of hoops to get there is extensive.


- Create a Community Patch Kit (Vaderkos)
Creating a (0) mod is dubious to me because we're trying to reduce the number of clicks, not add more. :p

The ethos of the mod stipulates that all database enhancements and Lua hooks are added in the (1) Community Patch mod. I'm definitely not opposed to integrating good quality work that makes modding easier into the main mod, though (hence why this is 'dubious' and not 'vetoed'). :)


The other ideas, while not necessarily good ones, are at least worth more discussion in my opinion.
 
Last edited:
- RE: Modmods Repository - "we need some kind of organization (sub forums for civs, QOL, overhaul-level changes, etc.) and then also moderation to archive things that aren't being maintained" (hokath)
- Suggestion that we just have the following subs: released mods, WIP mods, and archive/retired
- From there have the colored labels on each to denote civs, QoL, game rules etc. Like how the Congress threads get color labels
- Also if possible, a label that will tell the last compatible version of VP
I think maybe this should be fast tracked -- I'm no forum admin but it seems like low hanging fruit for improving the VP experience for the casual forum-goer.

Another idea that admins might look at alongside this stuff: some mods in the modmod sub use the CF "resource" structure, and many do not... For example if you click through maritime battles mod in my sig, the discussion thread has tabs at the top with "history", "updates", etc. But the recon pay mod in my sig does not. It used to be possible to create these linkages as CF user, but in recent years it seems to have changed, such that I have to request admin to do it now. Even when it worked it wasn't super obvious how to do it -- if we could ensure this structure was easily accessible to modders, CF resource pages have good feature set for reviews and version control etc... Good enough anyway, better than just a bare forum thread with uploads or links -- we should use it as much as possible, but it needs to be easier than it currently is.

My wishlist for this forum feature is:
- button next to 'attach file' for 'upload resource' that links to this more extensive upload structure, only available when creating or editing OP of a thread
- ability to designate multiple authors per resource
 
We already have enough people who run into installation problems despite the autoinstaller. This would multiply that by a factor of three, I predict.
The problem with the current installer is that while it's working just fine with VP-only use cases, it could lead to problems for people playing with mods. Because the installer wipes the cache folder, it means that the user needs to re-enable the mods they are playing. If they enable them in the wrong order, then it could lead to an ID mismatch problem within the database (some parts load first than others or the opposite regardless of the mods enable order).

While I can just grab the file from the repo and then paste it into the right folder, you can't say the same thing for regular users.

Current installation problems also occurred because of folder structure issues. People who installed Steam on a different folder (not the typical C: drive and on the Program Files folder) usually faced this. Someone needs to make a FAQ about this as it has become a reoccurring issue.

Contracting someone to create art assets or code is along the same lines - bringing money into the development process is problematic for the same reasons as accepting donations would be. There's questions about whether contracting code would be legal given the license agreement, as well.
While I know it's frowned upon to receive donations for a Civ V mod that is built upon collective contributions, and also Gazebo himself is opposed to the idea of financial incentives for the mod development. It would be a good idea to have asset ownership for the mod. Just in case someone all of a sudden asks to take their assets off the mod even if we know they were generous to lend us their creation. This happened many times in the creative industry. The creative industry is kinda notorious for making sure there's a formal agreement for asset usage because you can't just trust someone based on an informal agreement.

I don't want to give an opinion for the contracts for someone to do code for us though, as it's a different thing than just asset ownership. Asset is an individual thing that can be taken on/off a mod, code is an interconnected stuff that can't just be taken on/off a mod.
 
Is there any intention or demand to keep VP on feature parity with new generation of historical 4Xs? I played some Old World, Humankind and Millennia (demo) in my time away from VP, honestly the only one I properly played was Humankind. VP is still better (by far) in most regards, but I can name two areas where Civ5-VP formula feels underdeveloped. One is barbarians, both Civ6 (iirc) and Humankind have interactions with barbs, allowing you to recruit them either directly or indirectly. This would be a serious undertaking, but I think it is within our capabilities with what we can mod, so that probably would be my candidate for a new mechanic/feature.
The other issue with VP is the infinite city sprawl. Civ6 added a half-baked loyalty (iirc) system to discourage settling close to other civs, the newer games seem to move towards an area/region system. In VP this is discouraged by the AI aggression and yield scaling, but I still feel overwhelmed with how crowded the map gets, the AI is relentless. I don't have any suggestions here though. Oh and a wargoal/casus belli feature would be welcome, as mentioned earlier.
 
One is barbarians, both Civ6 (iirc) and Humankind have interactions with barbs, allowing you to recruit them either directly or indirectly.
BE as well -- its not a modern game but nonetheless, demonstrates that civ 5 engine can do more with barbs -- I'd enjoy this if well executed
 
I like that in Civ6 you could bribe barbarians to attack someone and after some time, a camp is transformed into a city state. It add some dynamics to the game.
 
So in Humankind (and other Amplitude games afaik) the map automatically gets divided into regions which are distinct geographical areas, and each region can only have one city, where you place the city is up to you. It's possible to found a city closer than 3 tiles to the enemy, but regions in general are much larger than the 3-tile radius we get in Civ5/VP. There's also a neat mechanic where you can claim a region with an outpost, but link it to a near city instead of founding one. It has its drawbacks, this essentially means there's a hard cap on total cities on the map (due to number of regions generated), there's also a hard-ish city cap per player instead of a proper wide/tall mechanic.

I really like the mechanic but I'll admit it isn't really suitable for VP. Something like Civ4/Civ6 would be better, such that if you're going to found a distant city very near to your rival's heartlands, you either need to have policies or traits to back it up, else it gets flipped/converted.
 
Top Bottom