[Dev] 0.0 "Aztec" Progress Thread

Nice. Thanks for that.

I planned ahead for this so this code doesn't have to be so horrible. The Player object corresponding to the human is passed all the way up through createGame to the UI and gets stored in Game.controller. Also, barbarian units can still be grabbed by the unit auto-selector, the only reason it skips them right now is because they're fortified. IMO the best way to solve this is to move unit auto-selection out of the engine into the UI layer. The alternative is to pass the controlling player to the engine so it can cycle appropriately but ideally the engine shouldn't know which players are local vs remote or even human vs AI (though that might be impractical for implementing diplomacy).

 

On the topic of PCX vs ImageTexture, which is relevant to both what @WildWeazel and @Puppeteer wrote:

In addition to what Puppeteer wrote about having code to turn a PCX into a Godot texture (ImageTexture), a relevant detail is that PCX -> Godot Texture is a 1:Many relationship for some PCX files (and 1:1 for others). For example, many advisor leaderheads are in one PCX file.

I added caching and dictionaries (C# dictionaries, to be precise, basically the same as a Map in Java) for both; the PCX dictionary uses the relative path (e.g. Art/popupborders.pcx) as a key; for ImageTexture it uses a composite key that includes both the relative path, and the width/height/upper-left-cropping-anchor for the texture (e.g. if it's a 20x20 texture from Art/someFile.pcx, and we crop starting at 60, 80, the key would be something like "Art/someFile.pcx-20-20-60-80").

I haven't seen any problems with using a C# dictionary to map Godot ImageTextures. And while they may use slightly more memory, I think it's the better of the two caches - creating the textures seems to be relatively expensive compared to loading from disk, so caching them will save us more time than caching the PCXs (although since some PCXs result in multiple ImageTextures, we save additional time by caching both).

It is true that for many PCX, FLC, and likely other files, we may well only need to reference them once per session, and caching e.g. the FLC may not make sense. But I think caching-by-default (opt-out) makes it easy to get the performance benefit without a lot of effort. Another potential side benefit is that it isn't necessary to keep track of references to textures that are re-used across the application, e.g. instead of a bunch of areas having to know where the advisor leader heads are, or the cancel/confirm buttons, you just say, "give me this texture", and if some other area already has, you get it really fast.

Or maybe we'll conclude that it only makes sense for things that are used in a bunch of places, such as the popup backgrounds, often-used buttons, and advisors. Everything gets more complicated when things like civ color, shadows, and smoke come into play, it's the non-civ-colored PCX realm where I'm most confident that caching is a useful default.

 

I believe the engine will have to know which player a unit belongs to, at least. It probably also will have to know which players are AI, not because the AI needs to treat humans differently, but because that way it knows which players should not have their turns simulated by the AI.

It might be thinking too far ahead, but one of the reasons I was thinking auto-selection in the engine made sense is that if we do add multiplayer someday, and there is simultaneous turns like in Civ, then the next-selected-unit might not exist anymore, due to combat or bombardment. The engine would know about that; the front end may know about it if is synchronized quickly enough, but ultimately the engine would be a more reliable source of truth.

Perhaps it isn't relevant, as the player might click on a unit that also just got bombarded to death in the UI, or an already-selected-but-not-moved unit might get bombarded to death. So perhaps my conclusion is that for multiplayer to work well, the engine will have to confirm/deny if a move is valid. And that multiplayer adds a lot of cases that aren't concerns in single player.

I still am favoring letting the engine handle unit selection, though. I'm concerned from an architectural standpoint that if we move too much logic to the Godot side, we'll lose a lot of work if we ever want to move to Godot 4, Unity, or some other engine, and also that we'd be giving the front end too many game logic responsibilities. It can be tough, because the front end does need to know some thing (which units are there, are they the player's?), but I'm not sure that just because a mechanic may not be relevant to the AI, it should go there.

Regardless, having the player object and/or its guid on the front end is a lot better for the selection logic than the color! Good reminder about the barbarians being fortified, too. That means that regardless of where we put it, we'll have to make the auto-selection logic smarter to account for there being multiple players as time goes on.

 

The AI is part of the engine so at some level the engine must know which players are AI-controlled and which are not. I was thinking that the portion of the engine that handles incoming player actions shouldn't have to know. The original Civ 3 wasn't written that way, instead the AI can modify the game state directly, and as a result there are some bugs where the AI is allowed to do things the human player isn't.

We should write the engine and UI to be maximally pessimistic from the start so the game doesn't end up plagued by subtle desyncs and crashes. Maximal pessimism for the engine means every incoming action must be fully validated (even more than latency or transmission error, assume it's coming in from a player who's trying to cheat). And for the UI it means assuming that the game state can change underneath it in any way at any time. This happens automatically if the UI is written to take a snapshot of the state of game, the most current state known to the client, and work with that whatever it happens to be. That isn't how it works right now since the terrain sprites are stored in a Godot TileMap, which keeps its own state, but it's what I'd like to do. I expect the most challenging part will be dealing with animations since they have their own states that unfold over wall-clock time, unlike everything else in the game world for which time only exists in the form of the turn counter.

IMO it's strange to have unit selection be part of the engine. For example, it means the engine has to know something about the players' game preferences because there's an option to disable unit auto-selection. It also means a mod to add Civ 4 like unit controls couldn't just be a custom UI thing, it would also have to modify the engine and the interface between it and the UI. Besides, there's also the matter of latency in multiplayer. If you want all engine interactions to go over the network then unit selection will become laggy, and my experience with other games is that dealing with a laggy interface really grates on the nerves. The easy way to avoid that lag is to make unit selection be a special part of the engine that doesn't get wait for the server's OK to do its thing, and as long as it works it doesn't really matter, but I don't like that since it makes the code less conceptually neat.

 

It sounds like we're in agreement that the engine and UI should be pessimistic, and should validate actions (although it does sound like a somewhat tall task given all the other tasks). You may be right that the part of the engine that handles incoming actions won't need to know which players are humans, as the rules should apply equally. I can't think of a case off the top of my head where that part of the AI would need to know.

For unit selection, potential network lagginess is my top concern with having it in the engine. But if the player disables unit selection, the Godot UI would simply not call the engine to select the next unit; the preference could remain in the UI. Or perhaps it would only call it to see if it returned that there were no units (indicating the end turn button should start blinking), and ignore the result otherwise.

I think one of the early non-developer tasks should be conducting some user research on the relative priority of multiplayer, and what types of multiplayer (PBEM? hotseat? simultaneous turns?). I don't want to box us into a corner where we can't support any of those modes, but simultaneous turns in particular could result in a lot of extra thought and time spent, and since we already have a large task, if only 10% of the audience plans to use that feature, I wouldn't want to increase the time spent by 50% upfront to support it. It would make more sense to spend 10% so we don't box ourselves in on future support, and spend that additional 50% later if we get across the finish line on the main game, even if that means more overall effort to get SP + MP.

(But I'm also looking at it from the perspective of someone who has played 98% single player... albeit I probably would have played somewhat more MP if the Civ III MP code were better)

 

I was imagining the engine would be in control of when units get auto-selected, for example by sending an event to the UI when the selected unit runs out of moves. But that just makes it even stranger that the UI is in control of when units get auto-selected but not which unit gets selected even though it has access to the entire game state. Anyway I'll stop harping on about this.

Same, I never got deep into Civ 3 MP and the last time I played was something like 15 years ago. Multiplayer is such a large part of the remaining Civ 3 community that it would be a shame if we didn't support it, but you're right it's only worth so much effort. I believe if we plan accordingly it's totally doable.

 

As for unit selection, I'm not sure that has any meaning for the game engine. I don't really expect the engine to ask, "what do you want to do with this unit in particular right now?" for every unit the player owns. An AI can loop over the units itself and then command them by ID. Instead I imagine the engine handing an AI or the human UI a game state in which all the units and such can be listed and given orders.

On the other hand, some UI preferences will need to go in the SAV file which means going through the game engine. Civ3 seems to remember your unit turn order, so it must be in the SAV file. (Huh, hadn't thought about that before...not sure where to go looking for that...ok I have some ideas on that now.) But my first thought for implementing that would just be to have a "UI settings" JSON object that the UI and engine pass back and forth just so the engine can stuff it in the SAV.

 

Removed some cruft as discussed somewhere around here. Also renamed the directory and filenames from Legacy to Civ3 but did not change the code as it's a small bother and I think it's going to be heavily revamped shortly. There is now a jim-cruft tag to the last commit before it was deleted for later reference.

I also took out the skiing (?) warrior as I think everyone has had a chance to see it, and it doesn't need to be there, but the hack temp code was at the end of the _Ready() function of Game.cs and surrounded by "hack" comments if you were using it for something.

 

Oh look, an actual Aztec task: We have a web page now: https://c7-game.github.io/

Repo is at https://github.com/C7-Game/c7-game.github.io . Pushing to main should trigger the page to rebuild, but it may take a while.

The linked title at the top confused me at first, but I'm pretty sure that's just the theme header. If we hate that, I think changing layout from page to none will get rid of that, but I think it's possible to change the site title from c7-game.github.io and thus that big link at the top. Another layout type is post. I've used Jekyll a lot on my own, but I haven't done more than poke at GH Pages before. It's Jekyll behind the scenes, but I'm used to *being* behind the scenes, so I'm not totally sure what to expect.

Edit: In the repo, Settings -> Pages has a theme chooser and some other info.

 

Aztec target, sort of: executable, sort of: https://github.com/C7-Game/Prototype/actions/runs/1512435558#artifacts

GitHub Actions CI build works for Windows exports now. I'm having to have CI move the json save into the build folder alongside the .exe and .pck files, but that works. Maybe there's a better way to handle it, but it works for now, and the "hard-coded new game" is not a permanent feature, anyway.

You do have to unzip the Windows.zip file and then unblock C7.exe, but short of app signing which I assume costs money and effort, I don't know how to get around that manual action.

The automated builds just CTD with no trace dumps when they crash, so whee. There is probably a flag to change that, but I think I'm done hacking away for tonight.

Edit: I don't recall what I typed here or elsewhere, but for now the CI builds are manually triggered under the Actions tab in GitHub. It can run against any branch with a drop-down choice. It appears all the results are public as I could follow the link in a non-logged-in browser. We can trigger on all sorts of stuff in the future, but for now feel free to go play around in Actions and export your feature branches on Windows. Mac and Linux CI builds are probably trivial to add, but I'm really, really tired right now. And for now they would still need a Civ3 file tree, an env var or fake registry to work, anyway. But I imagine a dev or three wants that ability, anyway.

 

Sleep is for the weak: https://github.com/C7-Game/Prototype/actions/runs/1512485359#artifacts

Edit: The Mac automated build doesn't work. (Well, the build succeeds, but I can't make the output work; I did have a manual export working earlier.) The Windows still does. Can someone try Linux? I don't have X11 handy anywhere.

The archive names are much more helpful now.

When you run a build, it will build for all three platorms.

Seriously going to bed now. Maybe.

Moar edit: It's possible to have inputs on the workflow trigger, so I'm pretty sure I can have a name field input so you can run a build and have it be e.g. C7-Windows-myTestBranch.zip or whatever. Oh yeah and once I figure out debug vs release, that could be an input.

 

Looks great! I took a look at the build when it was producing an error and I was tired, and had no idea how to fix it. But now when I download the Windows build, I just download, decompress, and run. No need to unblock it. Not sure what the difference is. I'm on Windows 8.1, don't have UAC enabled, and have obviously run things named C7.exe before (but you probably have too?). I do occasionally get the "this application comes from an untrusted source on the Internet" notice when downloading other programs, but for whatever reason my computer seems to think you are trustworthy.

I'll fire up by Bionic Beaver and see how it runs there.

Beaver update:

Got it downloaded and extracted. It's great that Ubuntu had a good GUI program for decompressing now. Ran it. Got an error:

I guessed that chmod +x would fix that, and sure enough it did. At that point, the splash screen would appear, but then it would disappear and crash. Running it from the console confirmed that the error was not being able to find the main menu mp3 file, which must mean I don't have Civ3 files copied over to this Beaver.

Which means two things:

1. It basically worked, we just have the chmod problem that I also had when running the Linux build on Windows. I wonder if this is as easy as adding the chmod step at the end, since GitHub Actions is probably running on Linux? Although GitHub is owned by Microsoft now, so maybe it's on Windows or a legacy Xenix box that they were looking to repurpose.

2. It might not be a bad thing that I have a VM without the Civ3 install files set up; that makes it a good test bed for avoiding crashes if files are missing. IIRC, I did set up a try/catch around the main menu background, but didn't think of that when adding the music. Update: Pushed a small fix for the mp3 crash, verified by renaming my Civ3 folder on Windows.

Oh and (3) I couldn't download without being signed in. Tried on Firefox on Bionic Beaver, and verified on SeaMonkey on Windows. Took a look in the Settings, but didn't see anything obvious, although I did see that it keeps the builds for 90 days. Which is fine for automated builds, but if we plan to use GitHub Releases to host releases, we'll have to figure out how to yank a build for a release and make it stick around a bit longer.

 

Cool! Yeah, I can add a chmod +x before uploading the artifact. Actions can run on various platforms I think, but the CI script is using Ubuntu latest and a docker image based on Ubuntu Focal.

Oh, I wonder if the same thing is needed for Mac? I don't recall having to set permissions on Mac before, but if I `ls -la` I can see Unix permissions on each file. But then again this was my first cross-platform export attempt I think. Oh, right, zip files don't preserve permissions.

Ok, so I can see if the artifact upload can be configured for tar.gz instead, or with Mac include its little metadata-in-zip file.

 

I just pushed a fix for the mp3 crash (on development), so the next go-round won't have that issue.

Maybe that's why Linux-focused software always uses .tar.gz? I always wondered why it did that, and was always annoyed when software came in a .tar.gz archive for the Windows version, since it isn't natively supported.

 

Yup, exactly.

As far as "we need to catch more exceptions," I'm realizing that just enabling debug mode accomplishes the same thing, just not in as user-friendly a manner. But it's less work, and this is an alpha or pre-alpha product at the moment.

 

I think anyone with push access can trigger the build, so you can do it yourself (if desired) at any time and not have to wait for "next go-round".

So...apparently zip *can* support permissions and does by default, but it doesn't on Windows? https://github.com/actions/upload-artifact/issues/38 But the real problem is that the upload-artifact action transfers raw files without permissions, and it's the archive hoster that zips the files for download.

There is a workaround provided...not sure if that's going to give me a tar inside a zip or if it will show the raw tar file as a download if I only upload one file in an archive. Will see soon I guess.

Edit: Actually, though, the artifact is really intended to be an intermediate step, anyway, and not a user downloadable. We would have a release workflow that attaches the exact files we wish to the release I think. I was thinking this is annoying for download, but that's not really its purpose. I mean we still need to preserve permissions, but actually I think the zipped bundle is just what you get when you download the artifact; I think the artifact exists as a folder somewhere if we use a workflow to deploy from it.

 

Is Aztec done? @WildWeazel

Over the past 24-48 hours we have on-demand exports which are as close as we're getting to "download and run" without signing applications or however that works.

The website exists. https://c7-game.github.io/

The one thing we don't have for sure is the launcher app, but I'm not really sure how that's gonna work, especially at this stage. I guess we *could* deploy the PCKs and binaries separately which could save a lot of download bytes. (The main binary is literally just Godot unmodified by us, although I think without the editor functionality...or maybe that is in there, not sure.) Actually that might be a solution to my Mac issue. Instead of letting Godot try to export a .app we can just toss the binary, a PCK, and the json file in there and it should work as well as the other platforms and as the editor does.

 

I'd like to have direct links on the website to download the official Aztec binary for each platform, like @Quintillus did here earlier. That way in our update we can just say "go here, download your version, set CIV3_HOME if necessary, and run"
Since we're pretty much there, I think we should create the first Aztec tag with release notes and zipped builds, link it on the site, and open this forum all at the same time as the dev diary. I'm not worried about the launcher right now.

 

Ok, cool. Yeah, the automated build and grabbing the artifact is not the user delivery system, and I've been mentally thinking of next steps there. Of course for Aztec we can just manually download the artifacts and attach what we want to the release page on GH. The whole artifact for Windows, the tgz inside the zip for Linux, and for Mac...maybe a little more manipulation.

Oh, I just figured out what I want to do for Mac, at least in the immediate term. Cool.

Point being, we can do it now by manually attaching the most user-friendly bits to the release instead of the artifacts which are too often archives inside archives. But we can also at some point have a GH workflow to create a release for us and attach the files in an automated fashion. We would certainly want to manually edit the release title and notes, but the rest can be automated.

 

https://github.com/C7-Game/Prototype/releases

A draft release I've tagged v0.0-aztec as of right now, just after Quintillus' merges/pushes this morning. I'm running the build now and will start attaching files.

Is this what we have in mind? Anything needs done differently, or someone else wants to give it a go? I won't get my feelings hurt if someone else wants to do it or do it differently.

Edit: Files attached, and the Windows and Mac ones work. (The Mac, -ish...can't 'start new game' without crash, but can load.) Of course you have to work hard to tell Win10 and especially MacOS that this is not evil radioactive waste after downloading and before running. I still don't have an X11 box to test the LInux one.

Edit 2: Changed the link to the releases page...apparently the draft link changes every time you safe draft? Or I did something wrong.