[Dev] 0.0 "Aztec" Progress Thread

[Quintillus]

Creating the first (and last) of the ABC version threads, for Aztec. It consists of:

• Project infrastructure
• Web presence
• Launcher application
• Download & run binary
• Skeletal architecture
• Prototype tilemaps

Link to document: https://docs.google.com/document/d/1Y_Vwnewfi78nOot8znd9Jt9X75DlS_KtuTUCINi6fd8/ Section 5.3 covers the Roadmap.

--------------

Status:

I'm trying something new by trying to summarize the status of the objectives in this area. This is subject to interpretation, and will change with time, but having it in one place might help build consensus on where we are milestone-wise. Last updated on November 9, 2021.

DONE

• Web presence, for now at least. Will be iterative but we have the forum and a thread in the main C&C

ADEQUATE

• Project Infrastructure. We could set up a CI pipeline and so forth, but we're making progress with what we have.
• Download & run binary. We can create these from Godot easily.

IN PROGRESS

• Skeletal Architecture. This has been making good progress in early November. I think the only big area we haven't added a skeleton for is the AI... and maybe that's adequate for the first release.
• Prototype Tilemaps. This is rapidly progressing; I think the remaining item for getting it to Adequate is having a little bit of unit animation. Maybe centering on a selected unit, and selecting a unit based on clicking on it

TODO

• None

------------


My reading as of 11/1/2021 is that we have prototype tilemaps (although we may want to experiment with those more?), web presence on CFC (do we want outside web presence yet? it feels a bit early to me), and at least some of the skeletal architecture. "Project infrastructure" might be done too; does that cover the GitHub project and the general Godot setup?

Notably, "Launcher Application" and "Download & run binary" are missing, to the best of my knowledge. My initial thoughts are "Launcher Application" likely means a bare-bones way to tell the project where to find the Civ art files, so it doesn't crash (although maybe it already loads that via the registry on Windows? I think I saw that somewhere in the code? Wouldn't work on Mac though, maybe or maybe not on Linux via Wine). Download & run binary should indeed probably be an early priority, so the audience can participate.

 

[Quintillus]

Well, the first step towards Download & Run Binary proved to be easy enough. To create one for Windows, I went to "Project -> Export..." in Godot. Then "Add...", and "Windows Desktop" from the drop-down. The defaults seem to be okay, as long as you're fine with 64-bit. I then did "Export Project" and "Export "PCK/Zip", to the same directory. Then you just double-click the name of the .exe file you created, and it runs magically.

It looks like this is stored in your local Godot configuration, as I didn't see any changes in the repo in Git after adding the export type.

I packaged it up into a .zip file, which you can download here.

I'll see if I have any luck creating versions for Linux and macOS from my Windows box.

 

[Quintillus]

Well, it looks like I can export artifacts for Linux and macOS from Windows. Whether they work, I am not sure yet. The Mac one winds up being a .zip file with a C7.app folder inside of it, so I'm a little dubious whether it'll work nicely with the the PCK file. The Linux one looks potential promising, however.

Links (for all versions):

Windows
Linux
macOS

The macOS one is for x64, not M1, as far as I can tell. In theory that shouldn't be a problem.

The Windows one does indeed auto-detect the Civ3 install location from the registry. I'm kind of afraid the other ones are going to crash due to not being able to do that, although maybe they'll just fail to load any graphics and only have the text buttons. I'll fire up a Linux VM and see what happens there.

-----

Update: Fired up my Bionic Beaver, and tested the Linux version. I had to chmod +x it at the command line (there has to be an automatic way of doing that for distribution, right? although maybe that requires building it on Linux). But after that, it fired up Godot successfully. It then crashed, for the predicted reason of not being able to find the artwork:

Unhandled Exception:
System.ApplicationException: Media path not found: Art/title.pcx
  at Util.Civ3MediaPath (System.String relPath, System.String relModPath) [0x000a1] in <156fe06a03ab405e8e602bb1bf3c78f7>:0

I'd consider that a success nonetheless. It shows we need to have handling code for the art if it isn't on Windows (maybe WildWeazel has thought about that? I think he is on Pop!OS), but I'm pleasantly surprised that I can build it for Linux from Windows, even if I have to chmod it.

It'll be a bit before I can test the macOS version; it might also make sense to leave that to the resident Mac expert, if we have one.

 

[Puppeteer]

I would call our current tilemap "placeholder" and not really "prototype". But that may not be a showstopper for the milestone. We can display tiles in a map display on Godot which is an achievement.

Well, the display code is at least prototype, but the tile structure/logic is a cardboard stand-in until some game logic schema emerge.

I'm almost certainly overthinking it. That's my kind of thing.

Yeah, Godot exported projects are really just an actual Godot executable sans editor code and the project folder packaged up. That was an interesting discover. I mean I think the mono gets compiled into assemblies, but the scene files and resources are just bundled up as-is and shipped with the executable.

It's possible to compile and optimize the executable, but I'm not even sure what that would gain us if we even cared to do that in the far future.

The upside of all of this is that it makes DLC (modpacks in our case) really easy for the end user. I think.

Incidentally, if I recall correctly, PCK is just a non-compressed bundle of file blobs, so basically a .tar archive analog. For speed of access. For our purposes the zip format probably makes more sense as C7 won't be constantly reading resources/textures for a 3d FPS, for example. Our use case will almost entirely be one-shot loads and then everything is in the heap while the game is active.

I'll try the Mac link. Yeah, I've recently been lusting after M1 Macs (even though I have no particular need for such), so I looked at Godot and M1. Godot *without Mono* now builds x64/ARM bundles for Mac, but for some reason the Godot Mono version doesn't natively make M1 code yet. I'm sure it will. I'm curious what the holdup is given that Mono is cross-platform, but not enough to go researching it. However, if I understand correctly, the M1 Macs should be able to run x64 code with their Rosetta thingy. (Not sure if it's emulation, JIT transpiling, or what...but it's something that reportedly works well.)

 

[Puppeteer]

MacOS download worked...sorta. I'm actually not all that good on Mac yet and am probably doing it the hard way. I had to unzip the download, then go into that folder and unzip the executable. Then I had to control-click it to open it to get the option to open anyway (else it just gives you cancel & move to trash options because developer is untrusted).

Then I had to go back to my terminal and launch the app with
~/Downloads/C7_Initial_Binary_macOS_x64/C7.app/Contents/MacOS/C7 . This last part is because right now our UI crashes to desktop (oops) if it can't find the Civ3 install folder and the art files, and if it's not in the registry which the Mac doesn't have it looks for the CIV3_HOME env var for the path, so I have to launch it from a session with that var set.

So we may need some default placeholder art and/or exception handling. And a UI to configure the Civ3 install folder location. And a local config file to store that. But not in the near term, probably.

I think it's ok for the executable to be zipped on Mac; I just don't really know the "right way" to install it. But that's a me problem not a Godot problem.

 

[Puppeteer]

The Windows one does indeed auto-detect the Civ3 install location from the registry. I'm kind of afraid the other ones are going to crash due to not being able to do that, although maybe they'll just fail to load any graphics and only have the text buttons.

[...]

Update: Fired up my Bionic Beaver, and tested the Linux version. I had to chmod +x it at the command line (there has to be an automatic way of doing that for distribution, right? although maybe that requires building it on Linux). But after that, it fired up Godot successfully. It then crashed, for the predicted reason of not being able to find the artwork:

Yeah, for non-Windows, just set CIV3_HOME what would be the top-level Civ3 install path, with Art/ , PTW/, and Conquests/ under it. There is also a way to emulate the registry because dotnet, but the method is different between the dotnetcore and Mono SDKs, of course.

Zip files don't store permissions, but tar files do (and tar.gz, tgz, tar.bz, tar.xz, etc.). If Godot won't tar the output we could automate a repackaging I would think. It is feasible to CI/CD pipeline a Godot build with a headless Godot executable.

 

[Puppeteer]

Can we get a link to the outline document in these milestone threads? I'm scrambling to find the link right now. I know it's around here somewhere. If I find it, I'll edit it into this post.

 

[Quintillus]

Ah, I'd forgotten about CIV3_HOME. I actually would put a UI to configure the Civ3 install folder, and either local config or setting that environment variable to save it, as a near term thing, at least if there is a non-technical audience on Mac/Linux. Otherwise it's a bit too technical for the average user to figure out how to set an environment variable to get it to run.

Ongoing term, I think compiling placeholder/substitute art is a great task that non-technical people can help with. As a community, we've probably created custom art for almost everything, although maybe not things like the Hall of Fame. But I know Balthasar did some great interface work, Kyriakos among others has great city sets, there are great unit creators and terrain sets. But we should support pointing to Civ3 art as well, since that will take time to compile.

I did the .zip files manually with 7-Zip, not through Godot (though there may be a way to do that; I didn't think to try). It's interesting that tar files keep permissions. I wonder if that could work from Windows? It may make more sense to just build the Linux one on Linux though, particularly if it isn't easy to figure out on Windows.

I'm glad to hear the macOS download sort-of worked; that is more than I expected given I created it on Windows. It doesn't sound quite as straightforward as would be ideal yet though; in particular I'm a little confused on the double-unzip. Is the "unzip the executable" unzipping C7.app, since .app files are usually self-contained? The developer being untrusted issue is also unfortunate.

It also occurs to me that I don't have a good feel for what version of Civ most Mac users are running. IIRC, the original Aspyr version had a different file structure than the Windows one, and than what you describe. But I'm not sure that the Aspyr version even runs on modern Macs, as IIRC it's 32-bit. So there are questions about what we should be targeting when it comes to Mac. I know in my editor, my code to find Civ3 assets uses a different code (and directory) path on Mac than Windows/Linux, though I'd have to look up what the difference is.

 

[Quintillus]

Can we get a link to the outline document in these milestone threads? I'm scrambling to find the link right now. I know it's around here somewhere. If I find it, I'll edit it into this post.

It is at https://docs.google.com/document/d/1Y_Vwnewfi78nOot8znd9Jt9X75DlS_KtuTUCINi6fd8/; I have also added it to the first post of this thread.

I am pretty sure you already have access, but if not WildWeazel should be able to add you.

 

[Puppeteer]

Thanks! Yeah, I found the link in one of the earliest email chains, and now that I see it, I'm not sure it was intended for more than 3 of us at the time. I'll let @WildWeazel reveal that document's current visibility status and readiness for public consumption.

I'm not terribly worried about a MacOS version in the short term. There really isn't a Civ3 available for Mac that we should target. The original vanilla has a native Mac version, but that was for PowerPC Macs, not Intel. Otherwise it's various hacks to try to run Windows software on Mac, and recent Macs more or less refuse to run 32-bit code.

I mean I definitely want it to work on Mac, but I think we have to target a modern native Mac environment which at the moment means ideally executables with both x64 and ARM/M1 machine code, but the latter isn't even feasible yet with Godot Mono. I guess I'll be the Mac "expert", but even though I've been using one for two years I'm still really really new to how app installs are supposed to work, much less how to properly deploy an app without signed code. (Or more typically the expected download-and-install experience for an average Mac user.) I've just been treating my Mac as if it's a really nice Linux-y command line environment with a really good GUI. And homebrew instead of apt or yum for a package manager.

I guess the "good news" for Mac systems is that they should have the same expected files available somewhere if they're making C3C work at all.

Also, I think I'll wait and see what other Mac users say about the downloadable and take my cues from their broken expectations. If any want to try these extreme early prototype builds.

I'm not sure it's sensible for native Windows to set unix permissions in a tar file, but it's certainly doable from WSL/WSL2 (Ubuntu on Windows). But I imagine at some point–and maybe soon to keep the newly-interested engaged–we can automate the build process, and probably on a Linux cloud VM or even a cloud function.

Like I say, building the "binary" is really just making the Mono assemblies, packaging the folder, and shipping it with a prebuilt platform-specific Godot executable. The Mono runtime is in the Godot executable, so it's pretty darn portable.

 

[Quintillus]

That all makes sense, particularly that we should focus on a native Mac environment and that at this point, that likely means the same files in the same structure.

I have a Mac Mini (2010) with High Sierra IIRC, that I can also use to test as needed, but it's not hooked up most of the time and is a bit slow (spinning hard drives aren't really what High Sierra expects as a boot drive), so it shouldn't be something to be counted on very often.

It would be nice to have an automated build process. That isn't the sort of thing that tends to get me excited, so for now I'll probably stay where the momentum is, and perhaps toss up a manually compiled build every so often, which was easier and quicker than expected.

@WildWeazel can also chime in on when it would be appropriate to post binaries publicly. Maybe having it coordinated with dev diary updates would make sense? But we can figure out a balance between "often enough to have a feedback loop" and "not so often there aren't any noticeable changes".

 

[Puppeteer]

FYI, merged my QueryCiv3 updates into the C7 repo, and I'll use the C7 repo as the origin / source of truth for the code now that it's public. The main API should be pretty stable, maybe, but the convenience getters will continue to be expanded. (This will make more sense in the future.)

 

[Puppeteer]

Um, what is the naming convention for a locally-scoped variable name? Is it camelCase or PascalCase?

 

[Quintillus]

Um, what is the naming convention for a locally-scoped variable name? Is it camelCase or PascalCase?

I think we're supposed to be using PascalCase. I've been very inconsistent, however, since I've been using camelCase for my entire career (except occasionally SCREAMING_SNAKE_CASE); sometimes I remember that this project is supposed to be in PascalCase, and sometimes I forget.

I didn't see it in the proposal doc, but it might be in an old thread here somewhere. I think we did discuss it, but am not sure where.

 

[Quintillus]

Putting my updates here as I think they fall best under "Skeletal architecture", specifically, signals, but also learning more about scenes and how those work in Godot.

The updates I've made tonight have no end-user effect whatsoever. Well, I guess they fix the Domestic Advisor, which was getting zoomed with the map. But there are no new features, just new arrangements of things.

-----

Previously, all of the advisor logic was in Game.cs. Game.cs is a popular file. Now, there are only two items related to advisors in Game.cs:

[Signal] public delegate void ShowSpecificAdvisor();

...
public override void _Process(float delta)
{
        ...
        if (Input.IsKeyPressed((int)Godot.KeyList.F1)) {
            EmitSignal("ShowSpecificAdvisor", "F1");
        }
}

Our global key handling is, for now, in Game.cs, and handled in _Process. We check for F1 being pressed, and if it is, we send out a "ShowSpecificAdvisor" signal, with "F1" as a parameter.

This signal is wired up in C7Game.tscn:

[connection signal="ShowSpecificAdvisor" from="." to="CanvasLayer/Advisor" method="OnShowSpecificAdvisor"]

CanvasLayer/Advisor is a new CenterContainer in the scene hierarchy:

We find the event handler there:

    private void OnShowSpecificAdvisor(string advisorType)
    {
        if (advisorType.Equals("F1")) {
            if (this.GetChildCount() == 0) {
                DomesticAdvisor advisor = new DomesticAdvisor();
                AddChild(advisor);
            }
            this.Show();
        }
    }

If there aren't any children yet, it creates a Domestic Advisor. Then, it shows itself.

There is another Signal associated with Advisor.cs as well, also defined in C7Game.tscn:

[connection signal="hide" from="CanvasLayer/Advisor" to="CanvasLayer/Advisor" method="_on_Advisor_hide"]

We see its code in Advisor.cs:

    private void _on_Advisor_hide()
    {
        this.Hide();
    }

Pretty straightforward - when it receives this signal, it will hide itself. It is invoked by the DomesticAdvisor (DomesticAdvisor.cs):

    private void CreateUI() {
        ...

        TextureButton GoBackButton = new TextureButton();
        //set up texture, etc, then...
        GoBackButton.Connect("pressed", this, "ReturnToMenu");
    }

    private void ReturnToMenu()
    {
        GetParent().EmitSignal("hide");
    }

An alternative would be to have a signal on each of the advisors, which would all be linked to the same method on Advisor, but one GetParent() jump seems like a more reasonable alternative.

Finally, there's one more signal from the AdvisorButton (in the top left of the UI) to the Advisor area:

[connection signal="pressed" from="CanvasLayer/ToolBar/MarginContainer/HBoxContainer/AdvisorButton" to="CanvasLayer/Advisor" method="ShowLatestAdvisor"]

ShowLatestAdvisor is substantially similar to the previously mentioned OnShowSpecificAdvisor, except that it won't set a specific advisor before displaying (other than creating the Domestic one as the default one should it never have been displayed before).

-----

Through this I've learned a decent amount about signals in Godot, and how we can avoid having all of our signals live in one place, or go through the same place. I haven't figured out a way to make them quite global yet - in the sense of, anyone can send an event without having to grab the node that it is "from" first - but we now have effective communication that doesn't depend on Game.cs.

 

[Quintillus]

I've made some significant updates in the direction of Skeletal Architecture, specifically putting my thoughts and proposed engine/data/Godot separation into place.

You'll see there are now two more top-level projects at https://github.com/C7-Game/Prototype, namely C7Data and C7Engine. They each have short readmes, but in summary the C7Data is intended for storing the game state, and C7Engine is intended to be where the mechanics of the game are implemented, and perhaps the AI, although perhaps it will call out to another project for that.

I've added some very basic basics to these, and now when you create a new game via Godot, it calls a createGame method in the engine. For now, that creates a dummy hard-coded map, with one terrain type, with three units on it, and that's within a game object that also has a turn variable. Super boring! But I've added just enough unit characteristics that I think it should be possible to use it for starting to implement the most basic of functions - the Hold (skip) action and Fortify action. The Engine has a method to get the next unit, and to fortify the unit, and to move the unit (and I guess I forgot to add one for the Hold action, that'll be next). It also has one to advance the turn.

Combined with my other small update, adding all the "basic" (bottom row) unit action buttons, I think it should be possible to show some interactivity and updates of game state based on those unit action buttons, as well as the lower-right info box, which can display which unit is currently selected, even though it won't show up on the map yet.

I expect a lot will change in the C7Data and C7Engine projects, but it felt important to get something out there in terms of data and mechanics.

@Puppeteer , in CreateGame.cs, there's a call to gameData.createDummyGameData, which makes a hilariously simple map. Right now it generates it in C7Data, and it should probably be in C7Engine when it isn't hard-coded. But that createGame method would be the place to plug in a less hard-coded map (although of course we also would have to get it back to the front end, which I haven't thought through yet, so it may be a bit premature for that).

 

[Quintillus]

I learned a bit more about Signals. It seems that we can't send just any old object over a Signal; I tried sending one of the shiny new objects I created in the C7GameData project via a signal, and got null on the other end. My suspicion is that we can only send serializeable objects, aka flattenable objects (I'm not sure if .NET uses the same "serializeable" term that Java does; it might have a better one; I know it took me years to reliably remember what "serializeable" meant). We'll probably want to serialize the objects so we can write them to a file (or whatever we use for a save game) anyway, so this isn't necessarily a big problem, it's just something that doesn't exist quite yet.

While I was at signalling, I finished moving LowerRightInfoBox out of Game.cs, and it's now entirely managed by GameStatus, including the new script GameStatus.cs. This involved getting rid of the famous GetParent().GetParent().GetParent().EmitSignal, and the end of our "just call GetParent() all the way up to Game.cs" days. I'm a little bit sad, that epitomized the "we don't really know what we're doing yet, but we're having fun adding shiny things and making them work" phase - which might still be the overall phase, we just know slightly more than a few days ago.

I also refined our turn end signals a little bit. The TurnEnded signal in Game.cs is emitted by OnPlayerEndTurn, and is now the one that announces to all the other UI components that it is connected to - you can connect a signal to multiple recipients - that the turn has ended, so they can do whatever they need to do. Right now, GameStatus listens to that signal, so it can tell the blinky light to stop blinking, but it could be all sorts of other things - UI controls that need to disable themselves, a music player that plays an end of turn sound, etc.

Going the other way, GameStatus emits a BlinkyEndTurnButtonPressed signal, which is listened to by OnPlayerEndTurn in Game.cs. This is the same method that will emit TurnEnded, which GameStatus listens to, to stop blinking. But it illustrates the event driven nature of things, and also how there is one central area that handles the turn ending - OnPlayerEndTurn in Game.cs. Eventually, hitting Enter or Spacebar when all units have been moved will also emit an event that connects to OnPlayerEndTurn, but the TurnEnded event will still be emitted, and the light will still stop blinking, without having to code that for each way that the turn could end. The same applies if the turn ends because the preference "Always Wait and End of Turn" is disabled, and the turn ends that way.

------

I have not yet completed the wiring up of the game data all the way through to the GameStatus area. I still plan to do that in the near future, though.

-----

I'm also thinking it may be a good idea to have a Godot Tutorials thread. We're all pretty new to it, and there may be new contributors in the future as well. Or we might take a break, and come back, and know there was something written up about how those pesky signals work, but not remember where it was. Another candidate tutorial would be how exactly all those margins and anchors work (I'm still figuring that out). And I'm sure we'll run into more as we go.

 

[Puppeteer]

Yeah, we have a lot of good reference notes in these threads, but it's all going to get lost quickly. I'm scrambling to find stuff already. But lots of notes in a forum are far, far, better than none or less.

You're probably way ahead of me on signals now, but one thing I think is particularly important to keep in mind is that Godot signals are compiled C++ code, not C#. Mono is bolted on to Godot, so when we're using Godot objects we're really accessing the unmanaged code through a translation. (Another side effect of this is that you can't necessarily just drop references to Godot objects and get them garbage collected, but if you explicitly delete stuff from a scene via prescribed methods it's supposed to clean up after itself.)

My takeaway from that is that we *only* want UI/display info being passed to/from the Godot environment and the C#/CLR environment and not a core CLR game logic object.

We may very well decide to use a signaling pattern pattern in game logic. If we do, we might end up with a C# native signaling method in addition to the Godot signaling, because there's UI and then there's core logic.

Edit: Huh, we might want to organize our code that way: *this* folder had Godot references and manage the UI, while *these* folders aren't allowed to know Godot exists. Maybe Godot code is only allowed to import a list of interfaces? Not sure how practical that is, just brainstorming. Nah, I guess we'll want some code that can speak logic and UI. And I think I just reinvented a model-view-controller pattern.

 

[WildWeazel]

Thanks for getting these threads started. The roadmap is a rough first draft so take the requirements with a grain of salt, but I do think the items in these first three collectively, do need to be addressed before much else.
A bit more on what I was thinking for each:

• Project Infrastructure
  • GitHub org/repo setup, with whatever features we're going to use (permissions, project boards, wiki, CI, etc.), a minimal development environment, and any other 3rd party services that are immediately needed. I've enjoyed using Pivotal Tracker in the past, but GitHub may be sufficient on its own these days.
  • Also it's time to start figuring out what kind of process and standards we'll follow. We should at least nail down iteration timing, when and how to talk planning and status, a git/task workflow, and some kind of coding standard (I think Godot has one of its own; C# certainly does), by the first milestone.
    
  • This will help answer the problem of how to keep track of stuff- cut and dry info can go into a wiki/readme, to-dos and open loops into issue tracking.
• Web presence
  • Whatever medium we're using for public discussion and promotion, whether that's opening up this forum, a Discord/IRC server, and/or something else.
  • By the time this milestone is met it should also include an official release site (probably GitHub Pages) with downloads, release notes, roadmap, and so on.
• Launcher Application
  • Not strictly necessary, but since we're iterating quickly and want to make it easy on users it would be good to have a single click-and-run tool to list and download versions, locate/configure Civ3, etc. Very similar to the old Minecraft launcher.
  • I actually made another repo called GodotForLaunch (because pun) to start looking into this and practice networking and UI, but never did anything with it.
• Download & run binary
  • Specifically, a CI product for every supported platform that you get from the release page and run with no setup. If using a launcher, this applies to both executables.
• Skeletal architecture
  • Partly done, also pretty open ended. Since it's hardly a playable game at this point, it should at least demonstrate the technologies, overall structure, and design patterns that we intend to use.
• Prototype tilemaps
  • The idea of the "prototype" bullets at the end of each milestone is to start working out the design and integration of a major or complex feature of the next release without necessarily exposing it in the game yet. It might involve internal or external tools, standalone demos, stubs, etc.
  • For tilemaps, the major things to figure out are scalability, smooth scrolling, wrapping edges, and layering.

I can happily confirm that your Linux download worked after a chmod and pointing $CIV3_HOME to my Steam game directory!

I'm fine with passing the proposal document around at this point. It's link-accessible.

 

[Quintillus]

@Puppeteer - Interesting, I had not realized Signals were using C++ behind the scenes. Does it also follow that all Godot objects are C++ under the hood then? That mixed model could definitely be a learning area for me; I don't think I've ever worked on a mixed managed/unmanaged project, or at least one where the unmanaged part used enough memory and lived long enough that the interactions needed to be considered from that viewpoint.

@WildWeazel - Good to hear the Linux download works! What are your thoughts on when the right time would be for the next dev diary, and potentially sharing builds... actually it looks like there might be a relevant thread for that.