[Dev] Godot / Mono Synchronization Thread

[Quintillus]

Spinning off from a topic Puppeteer mentioned in the Dev Diary 0 thread.

We should make sure we're coordinated on which Godot version we're using. We probably discussed it in Q1, but that has been a long time.

I've checked, and I am using Godot 3.2.3, with Mono. This is probably what we coordinated on in Q1.

Godot 3.3 is out now, and adds cool features like being able to play mp3s. I propose we switch to it for now. I downloaded it and took it for a spin and it seems to be working fine, although it did add a newline to the MainMenu scene.

Godot 3.4 has a release candidate available. I haven't tried it yet.

-------

We've also talked about Godot 4, which is in previews IIRC. Earlier I mentioned I was leaning towards switching to it. But I don't really know what benefits it brings. I have seen that initially (until 4.1), it will only support the Vulkan API, which may be relevant for users with older hardware. Vulkan is supported by:

Intel Skylake (6000 series) integrated GPUs and newer, e.g. Core i5 6600K
AMD Graphics Core Next GPUs and newer, generally Radeon RX 200 Series and newer, e.g. Radeon R7 260. With notable exceptions, however, especially on the low end
NVIDIA Kepler GPUs and newer, generally GeForce 600 Series and newer, e.g. GeForce GTX 660. Again, with exceptions.

These date back to 2015 (Intel), 2013 (AMD), and 2012 (NVIDIA), and lower-end parts that don't support Vulkan would continue to be sold for a couple years after that. So not exactly bleeding edge, but not yet ancient, particularly when Civ3 works on GPUs from the 20th century.

As far as I can tell, Godot 4 will still support Windows 7, which may also be relevant to some. But that's based on the "latest" documentation showing Windows 7 support; I haven't found a hard-and-fast "yes, it will be supported" statement, and haven't tested it myself.

Of course, the more important factors may be the changes to the engine itself, which will take more time to evaluate. But I'm also less concerned about finding blockers from that front, at least at this early stage of the project.

 

[Puppeteer]

The game logic will be in C#/Mono which shouldn't really change much between Godot versions, if at all.

As far as graphics engine differences, my original thought is that we're not going to be doing anything that should be impacted by that for quite a while. We're putting 2D texture on screen and animating a few sprites at a time, not real-time ray tracing high-poly shapes at 120fps. It might make a difference in APIs here and there...don't know yet. My main thought was that 4.0 will be out and reliable before C7 is, and as we can be sure that eventually version creep will bite us, might as well start with the future.

However, you touch on a really important point: we probably need to decide what hardware & OS C7 works on. I guess my inbuilt assumption would be that we leave the past in the dust, but maybe that's not a consensus opinion.

We definitely have people running WinXP for Civ3 on WinXP-era hardware. It's just not clear to me how much of that is by necessity and how much it is because Civ3 runs "right" on it.

Our two main version creep candidates are the .NET runtime (Mono for the foreseeable future) and Godot game engine. Each of those is only going to work so far back and so far forward at any given point in time.

(Side note: Not necessarily saying we'll *never* real-time ray-trace high-poly at 120fps...who knows what the new C7 native unit format will be? )

 

[Puppeteer]

Oh, as far as Godot versions, I currently have v3.3.3 . v3.3.4 is out as of Oct 1. Which can be super confusing because 3.4 is highly anticipated and due soon, and 4.0 is on its way in maybe the next few months? And also highly anticipated.

Anyway, every time I search for Godot v4 I get confused by all the "4" releases and end up reading about the wrong thing.

OS/HW compatibility aside, I think we should target at least 3.4 (which is up to beta 6 now) and consider targeting 4.0.

That might mean more frequent updates, but switching versions should be quite easy since the difference in both dev and distribution is the prebuilt Godot executable, so if we hit a snag and are wondering if it's due to beta status we could easily just temporarily use another executable version.

I might change my mind if the 4.0 2D API is different enough to trip us up. And I'm not that heavily invested in which version we settle on. Not at all. I'm more butt hurt about needing to figure out Mono better after I feel like I'm starting to get the hang of dotnetcore.

Edit: I drug my Windows laptop into the hotel and am updating it now. Since I'm using Chocolatey on Windows and Homebrew on Mac to install stuff including Godot, my updates are pushing me to Godot 3.3.4 . (I suspect it only will push maintenance releases and not from v3 to v4....not sure about v3.3.x to v3.4.x.)

Speaking of hotels and domicile, I have a 12-month lease starting on the 12th, so I'll have all my stuff inside one place for an extended period of time, unlike the last 3 months.

 

[Quintillus]

I forgot to reply after reading this on Monday. I'm now updated to Godot 3.3.4 on both my laptop (Win10) and desktop (Win8.1). I'd been flipping back and forth, with 3.2.3 on my desktop, for 12-24 hours with no compatibility issues.

I agree that use of Civ3 on older systems may be due to the compatibility challenges that more recent versions of Windows can pose with Civ3, rather than a general preference. But I also suspect there are more tech laggards among the Civ3 audience than among the general gaming audience, simply given the age of the game; I would count myself among that group, although not quite as much as I was in the middle of last decade.

It's also not really clear when 4.0 will be out, for realzies. In a Godot blog post from November 1st (Monday), Juan Linietsky, who is one of the head honchos IIRC, wrote:

Godot 4.0 alpha is around the corner now (we have been pushing pre-alpha builds recently), so looking forward to a release soon!

But Godot 4.0 has been mentioned as early as September 2019 (maybe earlier than that), and the alpha is not out yet. Which makes me think it's earlier along in its path than what I'd thought previously. If it were beta, I'd be more comfortable saying, "let's do it!" than not being in alpha yet. At this point, it looks to me like the sensible thing would be to go forward with 3.3, and once it's out 3.4, and keep an eye on 4.0, seeing how easy it would be to switch over, and occasionally checking what the perceived stability is.

I'm also thinking it probably wouldn't be the greatest idea to jump on the first alpha when we're still new to Godot and doing things the wrong way because we don't know any better, and would be unlikely to be able to distinguish reliably what's a bug because we're doing it wrong, and what's a bug because there's a bug in the alpha.

 

[WildWeazel]

FWIW Godot 3.4 just dropped. I assume that means it's all hands on deck for 4.0 now. I doubt it's going to be ready by the end of the year though.

 

[Puppeteer]

FWIW Godot 3.4 just dropped.

Wow, and it supports the new Macs natively. (As well as the now-so-last-year Intel Macs via universal binary.) Even with Mono.

Besides that, other new features that may interest us:

• Now defaults to WebP for lossless compression instead of PNG.
• 2D viewport has its own scaling factor, although I don't yet understand how that's different or better than scaling Node2Ds.
• Shader language data structures added. It's been a while since I've tried to use a shader for civ colors, but I seem to recall that passing data might have been an issue, and it sounds like this might be helpful. I'd really like to load one of each unit texture and then use a shader to add the civ color. Godot has no concept of indexed color palettes.

I just downloaded it. I'll see if it does anything particularly annoying with C7. I'm expecting it to rearrange the project file and maybe the csproj files, but I won't commit the changes.

Edit: It's working fine and didn't change anything except adding or removing a blank newline from project.godot and MainMenu.tcsn .

 

[WildWeazel]

As always, if you want to try something different, git branch is free

 

[WildWeazel]

They're already making progress towards 4.0: https://old.reddit.com/r/godot/comments/qq0kcx/new_prealpha_build_for_godotengine_40_a_couple/

Sounds like porting is not trivial, especially GDScript. Not sure what that means for C#. But to me that just suggests getting on 4.0 before we have much to port. Which raises the question, at what point are we done prototyping? I always intended the current repo to be a throwaway, and only use it until we're satisfied we have the right technologies. Now that we're moving faster I don't want to waste good code, but also don't want to create a Big Ball of Mud.

 

[Quintillus]

Interesting. My thoughts so far is the technologies we've been using seen pretty good. .NET Core is working well, C# is pleasant to use, and most importantly, Godot 3.3/3.4 are proving quite satisfactory. I know Puppeteer has also posted that he's been pleasantly surprised, and with the progress Flintlock has been making on the map and units, it must not be slowing him down too much. Which means:

- I'm not sure I'd want to throw away the repo at this point. We've kind of been prototyping but then refining. The "always expect to throw away your first attempt" is a principle I go back and forth on. While the second attempt is always nicer, even with an experienced team, I tend to find it less relevant when you have a more experienced team, which it seems that we do. More importantly, I have concerns that starting over may be a momentum killer.
- That might mean that we should try Godot 4 quite soon to get an idea of just how big of an overhaul it will be.
- We have separated the code out somewhat, into Civ3 import code, C7 data code, and C7 engine code. Which would help for migrating. But a lot of the work has been on the Godot side just because a lot of the basic display things needed early on are in Godot (and it helps figure out if it's the right platform). Still, it means that if we decide to stay on Godot 3 for now but in a year are thinking, "It's really be nice to have that new feature in Godot 4.3", the back-end logic would not be intertangled. Or at least we'd have made an attempt to keep it disentangled.

The tl;dr is I don't think moving to 4.0 is worth killing the momentum we have at this point. Disclaimer, I haven't spent that much time learning about why 4.0 is awesome.

Update: I'm compiling Godot 4.0 locally now, will report back on the results. Might be awhile; I never did upgrade my Sandy Bridge processor, and forgot to overclock it before starting the compilation process.

 

[Quintillus]

I forgot I had Godot 4 building. From timestamps, it looks like it took about 26 minutes, being allocated 3 of my 4 cores, on a Core i5 2500k running at 3.3 GHz (turbo disabled). Not too bad compile time wise.

But that was the general build on Windows, not the Mono version. I fired it up; it couldn't do anything with C7 due to not being Mono. The menus, oddly, did not appear below the menu buttons, but several hundred pixels below them and a bit to the right. But I was able to confirm that if you clone the master branch, the About dialog will tell you you are running Godot 4.

So then I went into the Mono instructions. First, I installed Mono, the latest version, 6.12. Then I generated the mono "glue". Then I launched the Mono build of Godot, after figuring out that you need to generate the blue by running bin\path-to-mono-build.exe, rather than cd'ing into the bin dir and running it from there.

It failed, though, printing some errors about not being able to convert doubles to floats.

I recommend following the guide at https://docs.godotengine.org/en/3.4/development/compiling/compiling_with_mono.html, and scrolling down to the "Examples" area for your OS, which make it very clear what to run.

The build might have failed because there's a dotnet6 branch for the Godot engine, and maybe the master branch doesn't work with Mono 6 but only Mono 5? That's just a guess though. It could also be that the Mono build is not compileable based on the current commit; it is pre-alpha after all.

I also recommend making a release build if you want to try it for real; the non-Mono Godot 4 build was noticeably less responsive than the Mono Godot 3.4 build.

 

[Puppeteer]

A possible "middle ground" move is to have a 4.0 branch, and those of us wanting to kick the tires on 4.0 can merge the busiest branch into it and see what breaks and what doesn't.

If GDScript is changing significantly, I'm going to deduce that Mono would, too, because I can't imagine why they would stray farther from Python syntax, so I deduce the changes must be the object API of the engine which would mean it changes for Mono, too.

Huh, I didn't realize there aren't periodic official builds of 4.0. I had presumed there would be nightlies or milestone builds. There are unofficial builds available here: https://hugo.pro/projects/godot-builds/ .

But honestly the fact that the Godot team isn't making periodic or even sporadic builds of 4.0 makes me rethink my desire to target it for now. We should definitely kick the tires on it if driven to, but not in a way that throws a wrench into other progress.

I haven't really coded on a group project until now, and I'm pleasantly surprised that there seems to be no conflict so far that's making road bumps for anyone's progress, and some days 3 of us are in the same file over a relatively short time span. I thought I had goofed stuff up a time or two–and in fact I think I actually did break mousewheel zooming but caught it and fixed it quickly–but generally I think it's working well right now.

And I have to agree not to stop progress. I know the current pace is temporary, and in fact I am flagging. I seem to be at a tail end of one of my high-focus streaks where it devolves into rapidly shifting my attention around and then running out of steam. It's a very typical pattern for me. Also, I'm moving into an apartment Friday, so I have a lot of things to busy myself with over the next few days at the very least, and really a bunch of business to catch up on. (I have lots of ideas that intrigue me, so I will get back around to it, and probably sooner rather than later.)

But back to not-me, Quintillus and Flintlock are making tons of great progress, so I think we should do nothing to slow that down. I expect sooner than we like we'll love a longer quiet, slower period where maybe one person is picking at the code occasionally, and we can refactor some then.

Then again, managing a project or people is *not* my strength, but again this is a for-fun project, not for-compensation.

I am not actually opposed to internally forking our project internally. I can think of ways that could be bad, but if one of us is just on a different trajectory than others, I think it would be better to fork and borrow back and forth than for one of us to not code. That might be messy if we start ballooning our repo count too much, but again I think it's a compromise.

And I'm throwing too many ideas around lately which is its own distraction, including this post. Ignore this if it helps. At least I'm enjoying myself.

Edit: Cross-posted w/ Quintillus...catching up

 

[Quintillus]

I'm inclined to agree with Puppeteer. There aren't any official builds (and I think the Hugo ones are the non-Mono versions, though I haven't verified that... at any rate they don't say they are Mono), the build I did locally had an obvious UI bug, and the local Mono build didn't work. I don't really have the motivation to figure out why it didn't work (well, data type conversion, but why that happened), especially since it could be a long investigation.

Once there's an official Mono build, even if it's an alpha, I think kicking the tires would be good, on a side branch at first.

Also, good luck with the move-in on Friday @Puppeteer !

 

[Quintillus]

Moving a discussion from the Performance thread to this one. I'm starting to realize we may not be as clear-cut on .NET/.Net Framework/.Net Core versions as I'd thought. At least, I've found it confusing.

This post is 90% de-confusing my mind over .NET versions, so feel free to ignore it. Or chime in if you've also found it confusing/think there's a way to make it less confusing.

Yeah, it's really messy jumping between SDKs, but I keep going to dotnetcore/dotnet because I have it at least a little bit figured out. I know `nuget restore` for the Mono side but not really anything else. I like simple command line stuff for a lot of my tasks.

I should really just figure out how to Mono on the console and make sure everything is on ... if not the same SDK then mono 4.72 or whatever Godot Mono 3.4 is using. Right now the non-Godot libraries are on netstandard2.0 for compatibility between the dotnetcore and Mono lines.

But now that I have the file opener in C7, we could manually open the file...oh we don't have a save function yet. Hmm...

Admittedly I've found the terminology around different versions of .NET to be confusing. I was pretty much with it through .NET 4.8, but the .NET Core/.NET merger threw me off. Checking Wikipedia, I see that I've been using some incorrect terminology. Particularly:

- What I've been calling .NET 4.8 is technically .NET Framework 4.8.
- .NET Core (the cross platform one) is now just called .NET, which I've used as shorthand for .NET Framework for about a decade.
- .NET Core goes through version 3.1 before being renamed to just .NET with 5.0.
- Confusingly, the .NET Framework and .NET Core have overlapping versions (1, 1.1, 2, 3, maybe some others). I think this was actually less confusing before .NET Core got shortened to .NET, since when there was always a "Core" in the name, it could be distinguished that way.

So I guess since .NET Framework and .NET Core merge with 5.0, then when I switched the target framework from 5.0 to 4.8, that probably is equivalent from switching from the merged successor to the .NET Framework to the last pre-merged one. Which at first I thought might mean it only works on Windows, but it looks like Mono 6.6 supports the .NET Framework 4.8, so that wouldn't be true either.

It looks like C7 is targeting the .NET Framework 4.7.2. Which Mono 5.18 (and later?) support.

C7Engine and C7GameData target .NET Standard 2.0. Which according to the table at Wikipedia, means .NET Framework 4.7.2, but also .NET (Core) 2.0. I'm honestly not sure which one that means we are using, or whether there is any practical difference between .NET Framework 4.7.2 and .NET (Core) 2.0.

It also looks like we're using the Godot.NET.Sdk/3.3.0 as our Project SDK in C7, though we're running Godot 3.4.

It probably makes sense to be consistent, and even if .NET Framework 4.7.2 and .NET Standard 2.0 are the same, we should probably use one of the other consistently.

 

[Puppeteer]

You're on the right track.

History: Microsoft back in their full-blown proprietary "embrace, extend, extinguish" mode made .NET, somewhere along the way becoming .NET Framework. It was not open source at the time.

Simian (I think) made Mono to be an open source version of .NET. as much as they could, they copied the APIs, so Mono tracks .NET Framework aside from some Windows-specific stuff.

At some point (post-Balmer, I think, probably Satya Nadella) started embracing open source more (and surely that has nothing to do with their entry into the cloud business, but never mind my cynicism), and they wanted their own cross-platform .NET, but for reasons they couldn't just open it as-is, so dotnetcore was started.

Now Microsoft owns Mono, and dotnetcore and Mono are merging at v5.0, as far as language API specifications.

However, there are at least two things to consider: The APIs specified by the versions, and the runtime. Before the v5 merger, there were some API virtual versions you might call them (there is a correct name for it, but this is noob to noob here) so libraries targeting netstandard2.0 and another version or two was reliably usable by both Mono and dotnetcore. Oh, and aside from the runtime/SDKs there are the tooling differences which is where I'm stuck.

Anyway, so my command line stuff was using the dotnetcore SDK and toolset, and so far Mono tooling...I don't even know what it is. I mean there is nuget and csc, but I have no idea how to create a csproj or add a reference or package without hand-editing the files in Mono. So when I want to do something quick outside of Godot I reach for dotnetcore which I think is just called dotnet now as of v5 and later. But Microsoft and branding/marketing....just wait a few weeks and it will change names again.

So, yeah, the only reason the non-Godot-using libraries are dotnetstandard2.0 is because of my inability so far to grok or even know about Mono command line tools.

On the other hand, since netstandard2.0 is really a language API set (not sdk, runtime, or tooling), I'm not sure it's causing any actual problems. I've yet to run into a language or API feature I need in .net472 that netstandard2.0 doesn't have, although I think I've come close a time or two.

It *does* cause some confusion, though, like System.Text.Json is included it the dotnetcore 3+ or so, but in Mono .NET472 it's a nuget package. And on the flip side, dotnet v5 dumped the old character encodings into a nuget package, so when my dotnetcore upgraded to dotnet5 my QueryCiv3 string reads quit working until I added the nuget package, but .NET472 still has them.

 

[Quintillus]

I don't know how to create a .csproj other than by hand, either. Still, my understanding of Mono/.NET is less ambiguous after this discussion. It's good to know that Mono is merging at 5.0 as well, I hadn't learned that before.

Tooling in C#/.NET is a weakness for me as well. I'm conceptually aware of nuget, but not csc.

It sounds like we're probably fine with .NET 4.7.2/.netstandard 2.0. I'll try again in a few days to see if I can get .NET 5.0 working locally, but for now changing the stand-alones to 4.8 locally is working. Which, incidentally, thanks for posting the map (.sav). I thought I had a matching map, and it was close, but there were some differences. Probably I had the barbarian setting set differently, and maybe something else as well.

 

[Puppeteer]

Huh, actually it seems to specifically reqire ".NET Core" which should be styled dotnetcore but is just dotnet as of v5.

https://docs.godotengine.org/en/stable/getting_started/scripting/c_sharp/c_sharp_basics.html

Godot bundles the parts of Mono needed to run already compiled games, however Godot does not include the tools required to build and compile games, such as MSBuild. These tools need to be installed separately. The required tools are included in the .NET Core SDK. MSBuild is also included in the Mono SDK, but it can't build C# projects with the new csproj format, therefore .NET Core SDK is required for Godot 3.2.3+.

In summary, you must have installed .NET Core SDK and the Mono-enabled version of Godot.

Edit: This adds a lot of possible confusion back into the console utility platform question. Whee.

Edit 2: If I'm reading this right, we're using dotnetcore v3.1 or dotnet v5 to compile code that is run by Mono emulating .NET Framework 4.7.2. Oh, and the csproj specifies Godot SDK. WTH Microsoft?

This part of this post seems to belong here. We seem to be–by design as prescribed by Godot–using dotnetcore 3.1 and/or dotnet v5 to compile the code which runs in Godot's Mono environment.

I had been thinking that Godot's Mono was the actual compiler, but apparently not. This suggests to me that it should be possible to build the project outside of Godot and distribute it with the redistributable precompiled Godot executable. Not that we should do that, but it means running unit tests and at least some integrations should be possible independent of Godot.

More specifically it would seem to mean I should be running unit tests in dotnet instead of mono? I'm really not sure on that one. Or maybe compile with dotnet and run with mono? Not even sure that's a thing I could do.

Edit: Ok, apparently ".NET Core" is the correct styling after all. Not sure if that changed or if I've always been wrong.

 

[Quintillus]

I see the confusion now. It's definitely odd that those docs seem to say to use .NET Core, whereas earlier today I found a Godot thread (second thread) that says they aren't moving to the unified .NET Core until Godot 4.1. Unless I totally misinterpreted those threads, which is possible.

My vote is we stick with 4.7.2. It seems like the confusion is wasting a lot of time and causing issues, and the simplest way to avoid that is standardizing on the safest option.

(In other news relevant to this thread, Godot 3.4.1 is out now. Haven't tried it yet)

 

[Puppeteer]

(In other news relevant to this thread, Godot 3.4.1 is out now. Haven't tried it yet)

Don't bother. 3.4.2 is out today!

https://godotengine.org/article/maintenance-release-godot-3-4-1
https://godotengine.org/article/maintenance-release-godot-3-4-2

3.4.2 was quick-released to fix a problem on MacOS causing screen flickering in 3.4.1 . There are a handful of other fixes.

3.4.1 has quite a number of fixes, but none of them would appear to impact C7. We should probably go to 3.4.2 as a matter of course, but I don't see a rush, and we should probably make sure everybody is back to working order on 3.4 before making other changes.


As far as .NET vs Mono, I am currently broken. I think I understand what's going on, but I thought so before and that understanding has changed a bunch today alone.

 

[Quintillus]

Well, I'm glad I didn't rush headlong into 3.4.1 then.

There is a 3.4.1 fix that might be relevant - well, two, the rid cleanup ones. I've found if I leave C7 up and in game while I'm AFK, I'll get hundreds of thousands of errors that report something about an rid, in the C++ layer. No noticeable effects in-game though, and I haven't found a pattern as to why beyond leaving the game up. Since noticing those fixes in the 3.4.1 RC fix logs, I've suspected that may be what's going on, especially since I don't recall seeing them before I upgraded to 3.4 (although that could have been coincidental).

If I notice that after being on 3.4.2, I'll bring it up on our GitHub. But I wouldn't be surprised if 3.4.2 fixes it.

 

[Puppeteer]

Mine says "dotnet CLI".

Which is different than what Puppeteer's says, which discomforts me a little.

(From another thread.) So we have a setting not tracked in the repo, whee. And I later realized I have no idea what the automated export uses to compile. I'll have to look in the docker container and see.

Edit: The godot-ci:mono Docker image we're using for the auto build is based on the latest mono Docker image. I'm not sure if or how often the godot-ci Docker image is refreshed, so I don't know if the mono version changes over time. But good to know I guess. I may want to echo the mono version in the build script or perhaps even stuff the info in a text file in the artifact.