[We the People] Development "diary"

but as a developer, can't you understand that as a player you want something from it too?
Obviously we do and honestly developing the mod right now is less fun than usual because all the fun stuff is after the next release. Right now it's just finishing tasks, which haven't been done yet, likely because the "fun factor" is low, but they need to be done in order to get the mod working correctly.

If the release should contain "what we want", then colopedia would likely have to be rewritten. However we agree that it is released "as is" even if it is less than ideal for understanding a lot of the new content. The issue is that a rewrite would take too long compared to how fast we want to release.

How many network games are actually played in relation to single games? 1 in 1000? Or even 1 in 10.000 played games?
I honestly have no idea because multiplayer is direct between the involved computers so no central server can count. Also there is no spyware in the mod so we don't know if people play or which type of games are played. Judging based on feedback multiplayer is a much wanted feature through. It's also something we have invested countless hours into (I recently spent 2 weeks pinpointing a specific problem) so releasing without fixing the last known problems in a day or two seems silly.

As for how much delay it will take to fix multiplayer, I will say much less than expected. It turned out that when I went through it line by line, there were less diplomacy actions with candidates for OOS issues than expected and as a result I'm already done. I found and fixed 6 bugs (8 if you include two exploits I found and fixed) and is now fairly confident that there aren't any left in that file.

I think I'm not the only one who hopes every day that the new changes will be released at some point. And it's a bit of torture when it keeps being postponed.
It's demotivating for us too to not get the release going. The plan was to have released by now, but testing uncovered some "must fix" issues. As I recall this includes crashes. The good news is that we have more or less fixed those issues so for the first time in quite a while I feel like the release is actually getting close.
 
After all, it's been years since the last release.
We modders are very aware of that and trust me we understand.
Do you think that it is not frustrating for us modders as well how long it takes?

Two years ago this project was at the brink of being abandoned. It was really really hard to get the team motivated and join forces again.
It took a lot of energy and will power to continue work and eventually regroup the team to start working on another major release.

---------

And yes, we achieved what is truly the biggest and also in my opinion considering gameplay also the best release this mod ever had.
But that is also part of the problem: It is too f***ing big! Too many todos and too few people contributing.

Closing all the todos, continously finding new issues and trying to fix this can be quite demotivating.
Especially when there are only very few people actually skilled and motivated enough to do so.

So I fully understand that from a players perspective having to wait is frustrating. But 2 years of modding without a release may be frustrating as well.
Please understand that the enormous amount of effort the mod team has put into the last release also did cost a lot of motivation and energy.

----------

It's demotivating for us too to not get the release going.
Yes indeed it is, because at some point it feels like fighting a war that will never be over.
You try again and again and there is still always something new missing or coming up ...

My personal motivation for example broke end of February completely after working on this release for 2 years.
(Mostly due to trying to continue working for a release while I was starting a new job and moving to a new city.)

Thus I am e.g. taking a break now from modding to see if and when I find motivation to mod again.
Still I am very grateful that other team members are still motivated and working on this monster of a release.

-----------

So whatever you currently may feel, please understand that being frustrated and complaining does not help anybody.
None of us is paid for this, none of us is in any way obligated to invest effort for e.g. fixing bugs, it is all voluntary contribution.

There are many more dependencies in a big and complex project and a big release like this than players may believe.
Several people need to deliever work products, lots of activities need to be coordinated, consense on balancing needs to be found ...

-----------

I from my side just hope that the current active team stays motivated to continue work and am thankful that there is still work invested.
So let us for one moment step back a second, take a deep breath and understand that the team is maybe also demotivated and just tries its best ...

I am very grateful that a skilled modder like e.g. Nightinggale invests time to fix OOS issues because we had lots of MP players complaining about them.
Or do you guys really feel it is justified for attacking modder for investing spare time to fix issues so community may have a better time with it?

-----------

Nobody in the team intends to withhold the mod from community - quite the contrary because we are proud of what we achieved.
In fact it is actually available for everybody as public dev branch since many months already and we have already 100+ players / testers.

We modders also tired sometimes, we modders are also demotivated sometimes, ... and still people invest their spare time.
Maybe being supportive and understanding their perspective may help a tiny little bit to keep them motivated.

There is often this illusion that there is a gigantic team of modders working on this project.
But honestly if you look at the commits there are weeks when only one or two modders are contributing.

This is a spare time project, not a professional project.
It is normal that people sometimes run out of energy and motivation.

----------

Personally, I prefer a mod with errors that are then corrected than waiting forever.
Oh sure, that is said easily if you are not on the other side being attacked by angry players when there are e.g. OOS issues in MP killing the game they played for hours.
Especially considering the OOS issues in MP games we have been heavily attacked already on e.g. Steam and also discord - people even became insulting.

----------

Summary:

I understand that people are frustrated. I also would have liked us to successfully officiallly release a stable new big WTP 4.0 already months ago.
But I also understand that fixing MP bugs before a release may be wiser than releasing with unplayable MP and having to go through a horsehockystorm with frustrated MP players.

It is already possible to grab the development branch, compile a DLL and play.
More than 100 people from community already did that because they also did not want to wait anymore ...

I am simply thankful to everybody that is still investing effort into the mod.
And yes I am also looking forward to have another stable public release.

But to do so, modders need to stay motivated - which is actually the hardest thing in modding.
And attacking modders for fixing bugs does not really help anything for that ...

Currently everybody is a bit wasted from the last years and additional stress does not help.
Highest priority is thus currently to stay motivated and finding fun in modding again.

Thanks for your understanding.
 
Last edited:
We modders are very aware of that and trust me we understand.
Do you think that it is not frustrating for us modders as well how long it takes?

Two years ago this project was at the brink of being abandoned. It was really really hard to get the team motivated and join forces again.
It took a lot of energy and will power to continue work and eventually regroup the team to start working on another major release.

---------

And yes, we achieved what is truly the biggest and also in my opinion considering gameplay also the best release this mod ever had.
But that is also part of the problem: It is too f***ing big! Too many todos and too few people contributing.

Closing all the todos, continously finding new issues and trying to fix this can be quite demotivating.
Especially when there are only very few people actually skilled and motivated enough to do so.

So I fully understand that from a players perspective having to wait is frustrating. But 2 years of modding without a release may be frustrating as well.
Please understand that the enormous amount of effort the mod team has put into the last release also did cost a lot of motivation and energy.

----------


Yes indeed it is, because at some point it feels like fighting a war that will never be over.
You try again and again and there is still always something new missing or coming up ...

My personal motivation for example broke end of February completely after working on this release for 2 years.
(Mostly due to trying to continue working for a release while I was starting a new job and moving to a new city.)

Thus I am e.g. taking a break now from modding to see if and when I find motivation to mod again.
Still I am very grateful that other team members are still motivated and working on this monster of a release.

-----------

So whatever you currently may feel, please understand that being frustrated and complaining does not help anybody.
None of us is paid for this, none of us is in any way obligated to invest effort for e.g. fixing bugs, it is all voluntary contribution.

There are many more dependencies in a big and complex project and a big release like this than players may believe.
Several people need to deliever work products, lots of activities need to be coordinated, consense on balancing needs to be found ...

-----------

I from my side just hope that the current active team stays motivated to continue work and am thankful that there is still work invested.
So let us for one moment step back a second, take a deep breath and understand that the team is maybe also demotivated and just tries its best ...

I am very grateful that a skilled modder like e.g. Nightinggale invests time to fix OOS issues because we had lots of MP players complaining about them.
Or do you guys really feel it is justified for attacking modder for investing spare time to fix issues so community may have a better time with it?

-----------

Nobody in the team intends to do withhold the mod from community.
In fact it is actually available for everybody as public dev branch since many months already and we have already 100+ players / testers.

We modders also tired sometimes, we modders also demotivated sometimes, ... and still people invest their spare time.
Maybe being supportive and understanding their perspective may help a tiny little bit to keep them motivated.

There is often this illusion that there is a gigantic team of modders working on this project.
But honestly if you look at the commits there are weeks when only one or two modders are contributing.

This is a spare time project, not a professional project.
It is normal that people sometimes run out of energy and motivation.

----------


Oh sure, that is said easily if you are not on the other side being attacked by angry playes when there are e.g. OOS issues in MP - killing the game they played for hours.
Especially considering the OOS issues in MP games we have been heavily attacked already on e.g. Steam and also discord - people even became insulting.

----------

Summary:

I understand that people are frustrated. I also would have liked us to successfully officiallly release a stable new big WTP 4.0 already months ago.
But I also understand that fixing MP bugs before a release may be wiser than releasing with unplayable MP and having to go through a horsehockystorm with frustrated MP players.

It is already possible to grab the development branch, compile a DLL and play.
More than 100 people from community already did that because they also did not want to wait anymore ...

I am simply thankful to everybody that is still investing effort into the mod.
And yes I am also looking forward to have another stable public release.

But to do so, modders need to stay motivated - which is actually the hardest thing in modding.
And attacking modders for fixing bugs does not really help anything for that ...

Currently everybody is a bit wasted from the last years and additional stress does not help.
Highest priority is thus currently to stay motivated and finding fun in modding again.

Thanks for your understanding.
I might be newcomer, but I think it has to be said - you're as team doing good job with this mod! It's very enjoyable to me, just as much as Caveman2Cosmos for BtS for example is. Don't feel ashamed for slow rate of release, the very thing that small team like yours can do such expansive uptades is wonderful in itself. Don't feel discouraged by players complaining either - there will be always someone who's either impatient or disatisfied and you can't really please anyone: people vary! So keep up good work and don't feel discouraged seeing some complaiments from time to time.
 
Well my dear modding team...
Time for uplifting words....
I still haven't played it, don't want to compile files, waiting (often the joy of waiting, Vorfreude, is the best of it)

Checking every two days for news.
It is not gonna be the salvation of our souls, but it is going to be glorious...
Thank you so much for giving your free time, your passion, your hearts blood. There are many of us, silently hoping and hyped...

A little bit further... And I don't mean time..... I can wait.

In the meantime, age of wonders 4 (back to fantasy setting, trailers look very cool) is coming on 2nd May. 4x is a queen among the genres.

Played fallout 2 today, first time since 10 years.

Best wishes dear team.
Your loyal Austrian
 
Oh sure, that is said easily if you are not on the other side being attacked by angry players when there are e.g. OOS issues in MP killing the game they played for hours.
Especially considering the OOS issues in MP games we have been heavily attacked already on e.g. Steam and also discord - people even became insulting.
Yeah people these days seem to expect online multiplayer in all games to such a degree that it's no longer a positive feature, it's more like a flaw if it ain't there. Have you seen games in early access? Multiplayer is a common request even in games where it makes little or no sense. Also even if it does make sense, a lot of games don't get multiplayer ever. Why? Because writing a well working multiplayer game is hard. It is even harder when using the civ4 engine as there is a lack of data encapsulation meaning the code is OOS safe and OOS dangerous code mixed together. Often OOS is caused by a mistake in a single line and it's hard to detect because it works fine while testing in single player. None of this matters to people who feel like they are entitled to get perfect multiplayer for free.

I will turn it around and say it's actually amazing that currently We The People has very stable multiplayer support. The risk of running into an OOS seems to be lower with the mod than when playing vanilla.

Having said that, I play multiplayer once in a while so to me it doesn't really matter how many other people use multiplayer. Since I do, it just has to work even if I would be the only one using it. This part matters more to me than whatever some angry mob will say.
 
Team - I am constantly in awe of what you have already achieved and what you are still doing.

As a player who has been with the game since its first release, and who logs on here every couple of days with baited breath, I salute you too.

Of course I hate waiting. But I am sure it will be worth the wait... and thank you all.
 
Hello Team,

I'm also checking regularly the progress and hope for the new release.
However, I know you are doing it in your spare time and as you did a great job in the past I will wait until you decinde that the version can be released.

Thanks a lot for the work and making this game such a gem :)
 
Hi Team,

I’ve been playing this magnificent mod that you and the team have built for years. My family live all over the country but get together online once a week to play. That’s thanks to you and the team.

I wanted you all to know as I appreciate that it must get frustrating for you as modders, especially when people take you for granted, but you guys are doing an amazing job and this is the best mod I’ve played. Take all the time you need. All your work is appreciated
 
I can no longer understand that. How many network games are actually played in relation to single games? 1 in 1000? Or even 1 in 10.000 played games? Or is the relationship even worse? But for that, again every single player has to wait and wait and wait? I can no longer understand that. I think it's nice that the mod is being developed further, but as a developer, can't you understand that as a player you want something from it too? After all, it's been years since the last release.
I think I'm not the only one who hopes every day that the new changes will be released at some point. And it's a bit of torture when it keeps being postponed. In addition, more bugs that have remained undiscovered until now will surely be discovered after the release.
I think it's great that you are developing the mod further and I'm grateful for the work, time and patience you put into it as a developer, but please also think of the people who want to play your mod at some point. Personally, I prefer a mod with errors that are then corrected than waiting forever. As a fan of the mod, that's not very nice at the moment.

Sorry for saying something, as such I'm just trying to read along as I don't have the skills to contribute. That's why I'm more of a silent reader here so as not to disturb you. But as I said before, as a player you wait for a release every day and I was just too disappointed and that's why I wrote something.

Very long time lurker of civfanatics over here... since the early day of Civ4, massive time spent on the big hits like FFH2 and its mods, Realismus Invictus, PIE...so many great others... and these days mostle WTP.

I'd just like to thank JohnDay for finally giving me the necessary motivation to create an account here and post and give a quick comment, which is :

@JohnDay : please take your complaints elsewhere. All due respect, your post is a bag of horse$. It reminds me of rich kids who complain when they get free ice-cream because they wanted chocolate but got vanilla. The people you're talking to are doing great work, on their spare time, totally for free. Do you have any idea what or how rare that is ? What they need is praise an encouragement, not childish complaints. Not sure in what fantasy world you're living when you still feel the need to complain over this project. You're a civ4 fan so am sure there's hope but take my advice : grow the hell up quickly man.
 
What is more of an issue is that failing to load our dll file means we can't use the dll file to post an error if this happens. Instead I found a place in python, which runs whenever python is initialized. The issue is that it's limited what can be seen from python at this point. The correct solution would be to add some "is vanilla" functionality to the dll, which python can ask, but since we can't mod vanilla, all I have to work with is what is available in vanilla. After pondering a bit about that one I realized vanilla has 100 widget types and we currently have 121. This means all I have to do is ask if there are precisely 100 widget types and if so, then python will know our dll file didn't load. It's a bit of a hack, but sometimes you have to do whatever you can based on the options you have available. It works.
You could also just import or call a function that wtp exposes but vanilla doesn't, and catch the error. I think this is supposed to be the "pythonic" way. :)
You could even expose an is_not_vanilla() function or something and do the same. This might be the most stable solution.

On another note, I'm quite interested in what you are doing with multithreading. I recall reading a few years ago that the C2C people tried this and it didn't improve performance enough to be worth the hassle.
 
On another note, I'm quite interested in what you are doing with multithreading. I recall reading a few years ago that the C2C people tried this and it didn't improve performance enough to be worth the hassle.
They did it in an entirely different way. I can't remember the details, but it was something about a very slow function and they tried to hide how slow it is by pushing it to another core. At some point somebody optimized the function to make it not slow instead and that improved performance more than it did to push it to another core.

We on the other hand use it not just for a few cores, but to enable all cores when there is a task where it makes sense. For instance when the AI tries to determine which profession to assign to a unit. It does so by testing the outcome of all options. In theory we assign one profession to each core and let them run through it (reality is even more complex than that). While we can benefit from optimizing the code in question, we did most of that already and it's still slow enough to justify using multiple cores.
 
I noticed after the release that multiple people complained that it's annoying to start vanilla and then restarting the game after selecting We The People in the main menu. There are ways to start Colonization directly into a mod, which are add a mod="folder name" as launch arguments or edit CivilizationIV.ini to list the mod. However even armed with this knowledge, people still struggle.

I have now coded the game so that at startup the mod writes its own name to CivilizationIV.ini. No need to do that manually. The result is that people will experience:
  1. install mod
  2. start colonization (in vanilla)
  3. select We The People in the ingame menu
  4. now forever when colonization starts, it will start We The People directly
The launch arguments still work and will take priority if given. If something goes wrong and people for some reason lose the ability to start Colonization (not sure how that should happen), the game can be recovered by simply deleting CivilizationIV.ini and it's back to starting in vanilla.

Hopefully this will once and for all solve the issue of people getting annoyed that they have to start vanilla before the mod.
 
So we finally released and that changed everything regarding modding. Until the release we had a period for what felt like ages where we only focused on fixing everything, which had to be fixed for a release. Honestly that ended up being not that fun and we have learned we need to organize releases differently.

So we released and that sure made feedback come alive. Most was positive, but there were a few bug reports about issues we missed. The first critical one was just hours after releasing so we made a new release a few days later and then all would be well. Well not really. It was after that we started seeing the game crash. This turned out to be a bug, which we have had for months, but somehow nobody noticed or nobody informed us about. I did manage to find and fix it, which could be a story in itself because it was a truly random crash, which showed up with no pattern to it. Well at least it was a rare crash since it had gone unnoticed for that long. Well no, suddenly multiple people both on the forum and in discord started talking about it. Well that sucks.

I realized we had to release a crash fix update ASAP, but at the same time it would be bad to make people download such a huge zip file over and over, like multiple times a week. I then decided to make an update, which intentionally would only update the DLL file and that way be a 1.5 MB update instead of a gigabyte. This meant not including every single fix we had made up until that point, but none of the others were critical. However now a new problem showed up. Other modders were elsewhere wasting time on silly stuff like sleeping. That's when I did what I wouldn't have expected would happen: I decided to release without a prior debate. And that's what I did. In that sense I will argue that 4.0.2 is my release as I not only did everything practical regarding the release, the only thing mentioned in the changelog, which I didn't do was the unloading with full barrack bugfix, which ray did. I bet some people woke up to a surprise. The main reason why I decided to "go rogue" was that waiting wouldn't just be waiting for next morning, but until next evening so it could be a 20 hour delay. I didn't want to wait that long while people reported having issues with crashes.

We have learned a lot about how to work with a mod, both from releasing 3.0 and 4.0. Mistakes were made with 3.0 which resulted in no more bugfix releases after 3.0.1. It also resulted in no more releases due to how our workflow worked. We also had issues with feature branches not getting merged back into some main development branch. All those issues are unacceptable so we have completely reorganized how we work. We decided to aim for 4.0 and do nothing else for a while. It took longer than planned and just focusing purely on a release while leaving most tasks as "it can wait until after the release" is not a fun way to work. It was however needed if we would ever release again.

Starting with 4.0, we now aim at a single branch approach. This seems to work and if you look up online there is a lot of debate regarding how development with branches should be done. We have essentially changed from one philosophy to another and there are fierce speakers for either approach. I will not participate in that fight, but I will say that it works better for us now.

As for bugfix releases, here is where we really stand out from 3.0. Our goal is not to release everything we have made in 4.0.x releases. Instead there is a dedicated release branch, which picks selected parts of the development branch. The end result is that we can fairly quickly make bugfix releases, which contains just the fixes and not new untested features or whatever else we have in development. We can also just skip features, which would break savegames. This has turned out to work very well, at least for the time being and this time we can make multiple bugfix releases like the plan was for 3.0, but never really worked out.

Last, but not least, modding is now way more fun than it was prior to release. There is a higher degree of freedom for what to do, but also it feels like there is more time to dig into each task. The newest bug I fixed is a good example of this change. Rather than finding some line to push a change into, which seems to work, I first spent time cleaning up the code. Now it's way more readable meaning not only is it less likely to contain bugs, the next person to work on it will have to spend less time figuring out what does what in the code and as such will be able to make whatever change is needed in less time. In fact I'm generally focusing on clean code now and code, which is less prone to bugs. I want to get rid of all the code, which I consider prone to hiding bugs.
 
Firstly, I would like to congratulate you for the release. I never understand where people find motivation for such big and long projects. Where can I find some good documentation (I am interested in AI bonuses specifically) and report minor bugs that I noticed(popup description of units gets confused, negotiation windonw can hang when re-loading and crash the game afterwards etc...)?
 
Firstly, I would like to congratulate you for the release. I never understand where people find motivation for such big and long projects. Where can I find some good documentation (I am interested in AI bonuses specifically) and report minor bugs that I noticed(popup description of units gets confused, negotiation windonw can hang when re-loading and crash the game afterwards etc...)?
For bug reporting you can use this thread in this forum:

https://forums.civfanatics.com/threads/we-the-people-bug-reporting.636760/page-72#

Alternatively you can use the github page: https://github.com/We-the-People-civ4col-mod/Mod/issues
 
"Feature development" has not really started again in the core mod.
(At least nobody officially announced that he is working on feature content right now.)

The main reason for that is, that we also still have a lot of small improvements related to WTP 4.0.x.
(E.g. we still improve balancing or fix small issues or work on Colopedia or add Achievements or ...)

Also most of us also want to have some time to play the mod or other games again.
Thus most of the team is taking it a bit slower right now and mostly just works on improvement releases.

However, almost all team members are still around and join internal team chat regularly. :)
So it is also likely that at some point we will start working on a major content release again.

------

It is good to take the time to "stabilize" again. :thumbsup:
Once most of the improvements are done, we may turn to new feature content again.
 
I'm done rewriting the error reporting system. You have likely all seen the vanilla system as it currently reports that it disables incompatible maps. That's about all it can do meaning it's not that useful. I looked into calling the window's message system directly and it turns out that it's atrocious and seems dated even compared to our 20 year old compiler. Calling it directly in our code would not only be ugly code, it would be prone to bugs. We want error messages to tell us about problems rather than the message itself causing new problems so I had to find a better solution.

I wrote a brand new class to make a human readable interface for the rest of the code. It also adds features Microsoft didn't provide, such as translation support. Now the error windows are no longer hardcoded to be English only. It will now always display the window on top of everything else, including the game in full screen mode meaning we finally have an error message window, which works the same at startup and during the game.

The message windows can now also control icon, alert sound and buttons meaning it's now possible to do something like a yes/no question.

This is currently used to detect errors at startup, such as incorrect steam version, incorrect mod placement and similar. It also has some detection of xml setup being incompatible with the current DLL file (through this can be expanded a lot). In other words the mod is now less likely to silently fail.

Semi related we now have a readme in html format (mainly done by WarriorPoet) and I added the ability to open an URL from the game. This mean that when there is an error at startup, the game is now able to open the readme file at the section telling about the issue in question and how to fix it. When opening the readme this way, it also detects the current language and will jump to a translated section if the language is available (German and Russian).

Elirector updated his WTP downloader/updater to work with WTP 4.0.2. This makes it easier to install the mod as it automates a number of the steps, which people have somehow managed to mess up (judging from feedback).

All in all while this won't change the gameplay, it should make it easier to get the mod running. It has also made it a whole lot easier to add error messages in the future and the plan is to test if xml settings are valid and that way hopefully avoid a number of future bugs. A good example of this is say a unit linking to graphics for the unit. That would make it valid to test if that specific graphic file exist and that way catch typos in filenames etc.
 
Work has started on a small additional feature, being the implementation of unique national units. About the original idea: https://forums.civfanatics.com/thre...e-units-and-unique-buildings-accepted.656205/

Each European nation will have a unique national unit, replacing a specialist. For example, Russia’s Siberian Lumberjack will outperform an Expert Lumberjack. The Dutch Timmerman will be able to produce even more than a Master Carpenter.

It is still work in progress, but a first step has been made. The Dansk Fisker has arrived in the Americas.
Adding this would not have been possible on myself, but only with the guidance of the whole team. Graphics were made of course by Schmiddie. 👍

Dansk Fisker.png



For me, working on this not just about adding a feature to the mod, but also getting into the basics of XML editing. My knowledge of modding/editing is almost nil. So I also want to take the opportunity to emphasize to everyone that it is really possible to contribute to the mod without skill but with motivation. :)
Hopefully this gives you just a push to join the modding team! :thumbsup:
 
Last edited:
Good news: adding the unique units for each nation has been completed!

The full line-up:
- Danish Colonies: Dansk Fisker, replacing the Expert Fisherman
- Dutch Colonies: Timmerman, replacing the Master Carpenter
- English Colonies: Colonial Distiller, replacing the Master Distiller
- French Colonies: French Furrier, replacing the Master Fur Trader
- Portuguese Colonies: Escavador, replacing the Expert Prospector
- Russian Colonies: Siberian Lumberjack, replacing the Expert Lumberjack
- Spanish Colonies: Explorador, replacing the Seasoned Scout
- Swedish Colonies: Järnsmed, replacing the Master Blacksmith

As said, also a huge thanks to Schmiddie for creating all the graphics! :clap:

Unique Units.png
 
Last edited:
Top Bottom