[Dev] 0.0 "Aztec" Progress Thread

[Puppeteer]

Does it also follow that all Godot objects are C++ under the hood then?

That is my understanding. But I am far from always correct.

I don't think it is something the average Godot Mono user has to worry about, and I don't think we're going to have to think much about it, but I presume that our ambitious goal will have us hit an oddity of that interface sooner or later.

And I tend to want to optimize everything, but truly I doubt we're going to tax Godot and its bolt-on Mono in any way with a turn-based game. Godot might actually be overkill, but it scratches itches, will do the job, and makes everything pretty easy. Besides, if we want to add 3d units, buildings or terrain, the engine is ready for it.

And the unmanaged part should look after itself mostly.

 

[Quintillus]

So far, I'm really liking Godot and it doesn't seem like overkill. Compared to doing everything without a game engine in my editor, it's really nice having the structure it provides, and like you said, makes things pretty easy. A definite part of why I never tried to convert my editor into a replacement game is that I didn't even want to try adding animations to it and trying to get what worked passably for displaying an editor-style map into something that could display a living, breathing, changing game world at 30+ FPS without requiring a Core i9-9900K to do so.

 

[Puppeteer]

Agree. When I thought of turning my Civ3SAT code into a CivAssist-alike that's why I went with an HTML interface with a command-line local server spawning a browser. It's kinda dumb, but I had no interest in learning a desktop UI framework. (GTK, WPF, Qt, WinForms, WinUI, Cocoa...any of it), but I had experience with and potential future needs for continued HTML experience.

Godot has been a pleasant surprise, and I'm at least interested in playing with it. And comparing it as a UI framework, I think the distributable is actually smaller than a bundled Chromium which is funny.

 

[Quintillus]

I've wired up some events such that now, when you click the "hold" button, the next unit is selected, and it updates the lower-right info box. Once no more units are available for selection, it displays the end of turn message there, and hides text areas as appropriate to match Civ III.

Architecturally, the important call-out is ParameterWrapper.cs, which is a work-around for wrapping C# objects in Godot, as suggested by a commenter on one of the Godot bug reports about not being able to pass C# objects. It works quite well, and as far as I know there aren't any monsters lurking underneath it, although I'm not 100% sure it's totally memory safe. But it seems to be much better than having to dig out individual strings, ints, etc. from an object to pass them around, and is less work than manually creating wrapper types. It seems roughly equivalent in overhead to using GSON to JSON'ify and de-JSON'ify objects in Java, which having done that before is a pretty low-overhead way to pass things over the wire, and thus it's a solution that I'm happy with so long as we don't find out it's losing memory. I also wrote a JavaDoc for the class, with detailed examples of how to use it (and now know that there's a slightly different syntax for C#Docs, I'll use that going forward).

This also means the blinky light now only blinks when you're out of units to move, not after five seconds no matter what.

I plan to continue down this path of turn/unit selection mechanics a little bit more, since those are still pretty fundamental tasks; for example I'd like to have Fortify working in terms of a unit not coming up for selection again the next turn (though not defense bonus, since we have no concept of that yet), and the Enter or Spacebar to end turn ought to work, since we need more ways to end the turn. Maybe I'll experiment with having different unit action buttons per unit, which is quite common in Civ, even at the beginning of the game.

Agree. When I thought of turning my Civ3SAT code into a CivAssist-alike that's why I went with an HTML interface with a command-line local server spawning a browser. It's kinda dumb, but I had no interest in learning a desktop UI framework. (GTK, WPF, Qt, WinForms, WinUI, Cocoa...any of it), but I had experience with and potential future needs for continued HTML experience.

Godot has been a pleasant surprise, and I'm at least interested in playing with it. And comparing it as a UI framework, I think the distributable is actually smaller than a bundled Chromium which is funny.

I also had little interest in learning a desktop UI framework, at least not without a good reason. Maybe C7 is providing that reason? But Godot is not making me regret trying it out.

I have dabbled some with JavaFX, which is a desktop UI framework, since I have a good amount of Java experience. It's somewhat similar in being scene graph oriented, although I haven't dug into that enough to really compare it adequately. It kind of doesn't suck, and has cool charts, which is what I've mostly used it for. It's also cross platform by default, which outside of the game engines is rare, or requires extra work (bundling KDE on Windows, for example). But also no one is really using it, so it's a dead end. Godot has life!

 

[Quintillus]

A little more progress on the turn loop. Units now become selectable again for the next turn.

I also re-organized the UI elements I've added over the past week, so they are now located in sub-folders of the top level UIElements folder. None of them are at the top level anymore. I expect our organization to evolve organically over time, and it was starting to feel like it was time to group those things.

Slightly more "Babylon", but @Flintlock 's map coordinate update is pretty nifty! Definitely a significant step that now we get the Civ III coordinate from clicking on a map tile.

 

[Quintillus]

Aside from the graphics part of it, the "hold" action and turn loop is now pretty much ready to go. Well, maybe replacing "4000 BC" with a turn number, so the user can see that progressing. But Enter or Spacebar for end turn now work, and Spacebar works for the "hold" action. So you can cycle through all your units doing nothing just like you can in Civ III. It's been kind of cool having Civ III vanilla and C7 up side by side doing the same actions and seeing the same results.

Worth noting is that I'd been using the key handler in _Process(float delta) before. However, if the user presses a key for more than one frame, that triggers every time. Not really a problem for bringing up an advisor (maybe performance-wise, but not functionally), and for games like racing games that would be desirable. For us, however, it would be really bad if you pressed spacebar to skip a unit, and it skipped five units instead. So I've moved the new handlers to the _UnhandledInput function where our mouse handlers already live. That only fires once per key press. There is an IsActionJustPressed/Released method for _Process that in theory can do the same thing, but I couldn't find documentation on what to pass it as the string action parameter for keys before deciding to just move to _UnhandledInput.

This also got me thinking about how to handle keyboard shortcuts for actions more generally - I'm leaning towards a Map, which I think is Dictionary in C# parlance, as well as the fact that we'll eventually need to have spacebar not do its usual actions when e.g. the player is renaming a unit or city. At a basic level, we could probably add a check around all our top-level keyboard shortcut handlers to say, "if popup_displayed = true, don't check these", but there may be something more elegant. (And we also don't want spacebar to end turn when e.g. the domestic advisor or city screen is up, so there's a lot of times when it shouldn't cause its "usual" action. Actually, the fact that it isn't supposed to when the Domestic Advisor is visible, and I added a prototype Domestic Advisor screen, means there's a great opportunity to try out a solution to this and see how it works)

---

Edit: I also got rid of the original End Turn button, which was the left of the two in the toolbar and the one that became disabled when the turn ended. We were up to five ways to end the turn, which seemed a little bit excessive. There's still one End Turn button in the toolbar, and now the game status area says "Please wait..." during the AI turn, so there's still visual feedback.

 

[Puppeteer]

Oh, I left out a mention of 3.4 features about input : https://godotengine.org/article/godot-3-4-is-released#input . Because at first glance I didn't think it relevant, but I don't think I've messed with keyboard input in Godot at all, so I don't actually know if it's relevant or not.

Hmm..if popups are Control nodes, can they not just handle all input or selectively choose not to pass input to now-lower control nodes? I'm not really sure what I'm talking about, but at least for mouse input the foremost control node can prevent mouse input from going to background nodes....although I'm not sure if that's only in its are or over the whole screen, but then again we could make a transparent full-screen Control Node and the input handler node could be its child, maybe?

In any case, input handling is something almost all Godot game creators use, so there should be TONS of info on input handling in particular, compared to all other Godot topics.

Yeah, for me at least, code & scene organization kind of emerges once we start trying to do things.

 

[Quintillus]

Oh, I read about that earlier tonight, in the blog. I don't think the axis/vector part would apply to us (unless we want to support gamepads for map movement), but the physical key mapping is kind of interesting. For Civ III, I doubt it would be useful, as the keyboard shortcuts aren't really based on physical proximity, but for the WASD example they give it makes a ton of sense as a feature. Caveat though, I only have the English versions of Civ; I don't know what the keyboard shortcuts are in other editions. Does "F" still fortify a unit in German Civ III, even if that might not make sense given what the German phrasing is? But it seems most likely that either the shortcut is always "F", or it's based on the word in each language, not that it's always at the same place on the keyboard.

Interesting thought on whether higher-level nodes could block input to lower level ones. Ummm... maybe? I'm not sure if it makes as much sense as it does for visual, mouse based input, but I'm not sure it doesn't make sense, either. The thing that could get a bit weird is what if you have two nodes at the same level of the scene graph hierarchy, and one of them says "ignore this key" and the other says "do something with this key"? With mouse events, you also have a "which one was clicked on?" to disambiguate that, but you don't with keyboard events.

I could still see something along that line potentially working. But it might be tricky; I'm thinking what would happen if I bound "F" to the fortify button and it was 4 levels deep in the hierarchy, and the Domestic Advisor was 2 levels deep somewhere else on the hierarchy? Of course, I suppose we might have issues with the fortify button appearing over the domestic advisor in that case, too. But if the top-level handlers are registered at the top level where they are now, overriding it seems potentially plausible.

Which leads me to agreement that a solution will likely emerge once there's code to try some of these options out. Alas, I got sidetracked reading up on JimmyH's IndieCiv, and downloading it to refresh myself on where it is, and heading over to DesignerNotes and realizing I really need to try Old World. I've been waiting for it to land on Steam or GOG, but that's still going to be about six months out, so maybe I should break down and buy it on EGS after all.

(Edit: Does anyone have a copy of the latest IndieCiv release? I might have it somewhere but it isn't in the obvious place on my HDD, and the link is dead. I got the code, but it's throwing errors about not finding Mono or XNA, and I'm probably not motivated enough to sort those out when I could be working on C7. Thought it might be neat to see where IndieCiv wound up though)

 

[Quintillus]

Hmm..if popups are Control nodes, can they not just handle all input or selectively choose not to pass input to now-lower control nodes? I'm not really sure what I'm talking about, but at least for mouse input the foremost control node can prevent mouse input from going to background nodes....although I'm not sure if that's only in its are or over the whole screen, but then again we could make a transparent full-screen Control Node and the input handler node could be its child, maybe?

This seems to be working. Open a game, and see how Spacebar changes the unit selected. Then open the Advisor (F1), maybe expand the window a bit so you can see the selected unit, and press Spacebar, and nothing happens. The Advisor will detect spacebar and enter and stop those keys from propagating. Close the advisor, and the keys work again.

I'm probably going to explore making this more general purpose, so we don't have to add the same logic in every type of pop-up. But that's not going to be ready tonight.

 

[Puppeteer]

Cool! As far as mouse input, maybe the controls only block that for their defined area, and I previously had invisible parts of control nodes blocking visible ones.

I rearranged "my" buttons, and both the old buttons and new work, but they didn't at first when I had buttons behind an invisible part of the toolbar.

The new side toolbar with "my" old buttons is animated! It slides in and out of the screen with a button click.

I also added pinch zoom. I thought I had done that before, but maybe I was thinking of my other browser-based projects. Pinch now works on both my Mac and Windows trackpads. Interestingly my touchscreen laptop won't pinch zoom it, so I guess that's a different gesture API. (It does work for touch drag which seems to be an equivalent gesture API to click and drag.)

Funny, now that I have pinch zoom working, I'm not sure I need those buttons anymore. It was the only way I could zoom on my Mac before.

 

[Quintillus]

I saw in the Git log that @Puppeteer wrote:

Added bg color and label to main menu
For if we don't have civ3 files available
Needs a lot more to make it useful,
but it exists now

I'd been wanting to center the background when the application is > 1024x768, so I did that, but also added an exception handler on the background scene so it doesn't crash if the art is missing, and changed the label so it provides some hopefully-helpful diagnostic info about the lack of art and how to fix it. Tried it out by renaming title.pcx so it didn't exist, and it seems to work.

Definitely not an airtight solution, but it will avoid the most predictable "the application doesn't do anything when you start it" problem.

I see @Flintlock is also doing some cool work with units! I'd started looking at Puppeteer's old GodotCiv3 repo with thoughts along a similar line, and came back and saw it was already added. That may be one small step for a Settler, but it's one giant leap for Settlerkind.

 

[WildWeazel]

Really great progress, everyone. So far it's coming together faster than I expected, and I feel bad that I haven't been contributing code. I've just been following everyone else's developments and fiddling with GitHub administrata. I'm looking for a good place to jump back in, and have a few ideas.

Firstly, given my self-appointed architect hat and lack of involvement in current features, I have more than half a mind to just go nuts with declaring and stubbing out interfaces, components, and event types based on all of our reference material. (I've already brainstormed quite a few of the latter) Then they're in place, at least as a draft, for us to use or flesh out as we work through game features. In an ordinary iterative project that would be a gross violation of YAGNI but given that we know almost exactly what features we'll need in a remake, I think we can afford to plan ahead. (Oz will also be pleased by its proximity to a functional spec)
I'm imagining this as all pure C# code, but I'm not sure how closely those things will need to be coupled to Godot (especially events, as those should be emitted through the script API). The general advice to beginners with game engines is to do as much as you can in the editor, but (again with the exceptional circumstances) since we want it to be data-driven and mod-friendly, I'm not sure that advice holds up. Plain Old Objects are easier to generate.

Secondly, I'm thinking about making a branch for porting to Godot 4 and seeing how much work is involved as it and we both evolve.
Being woefully inexperienced with the engine I may be the best positioned of us to pick up 4.0 with fresh eyes.

Thirdly I had mentioned wanting to have a launcher UI to manage downloads and configuration, which is something that could be done in parallel at my own pace. I'm not sure that's the best use of my time right now, though it would get me up to speed with the editor.

Finally it may be best to just focus on understanding our code and looking for improvements and refactoring opportunities, to get a better handle on the question of what to do with this "prototype". My main concern is the design, whether we have good separation of concerns and clean interfaces to build on.

Caveat: I'm about to go traveling for a week with no computer, so I won't be doing any of this for a bit. I'll be online, but availability TBD.

 

[Puppeteer]

I like the concept of lots of structures and interfaces. I see counterarguments to that, too, but I feel like we're a bit too tied to thinking in Civ3 right now instead of converting from Civ3 to a new native C7.

As for signaling, I am not at all opposed to using Godot signals for UI/display only and otherwise shunning them in favor of native C# for game logic (the "model"). It would be a bit weird to have two signaling mechanisms, but in my mind it makes sense to use the UI (Godot) where we need to and keep the rest native C#.

As for dev-time UI (scene tree and object inspector UI) versus creating objects in code...I'm not saying how I think is definitely right, but I tend to think that static elements like the various UI screens will be a bit more self-documenting if they are built in the scene tree, but more dynamic elements (map tiles and units especially) done in code.

That said, I'm not even saying it's a good idea to redo the existing advisor screen as its own scene (tscn file). And truthfully it's easier to run into ordering issues if some elements are done in the scene tree and others in code (such as the top toolbar and the UI buttons). So I'm kind of contradicting my previous paragraph here. I guess I would like there to be a skeleton of static elements in the scene tree to which dynamic content is attached, but that thought doesn't necessarily translate well to reality, and it creates dueling sources of truth for the UI ordering.

As far as how it runs, it doesn't matter. A dev-instantiated object and scene-tree/inspector-defined object are indistinguishable while executing; they are equivalent. So aside from having a root scene for dev convenience–it's possible but I think a bit tricky to skip the scene tree entirely–it really comes down to us devs and whether the scene tree helps visualize things better or whether it just creates unnecessarily two places to look for Godot object definitions.

I think a launcher is eventually a good idea, but I think we probably want to manually handle code plus media plus add-on media to understand what it's going to look like.

 

[Quintillus]

I think it would be helpful for WildWeazel to stub out interfaces as described, as it would help demonstrate what the approach described in the reference docs looks like in practice. I've tried to put some structure around it being an event-driven architecture in place, but haven't really known whether it's similar to what WildWeazel envisioned or not. Then you get into areas like interfaces where mods could hook in, and not having worked on a project where modules could be loaded dynamically (especially modules in another languages, such as Lua), I'm much less certain how to approach it.

(In other words, all those enterprisey projects technically use dependency injection, but they always inject the same dependencies in practice. I'm also thinking of interfaces as positive in the general sense, with Flintlock's recent tile layer interface being a good example. But not in the "everything must have an interface" sense, which I've seen on some enterprise projects. Might be hard to describe the difference, but basically interfaces for structure or where you might actually swap out an implementation are good, interfaces where you know there's a 99% chance you'll only ever have on concrete type, and you basically are adding the interface because you always have one, aren't)

I've found the scene graph helpful for structure/knowing where things are at a high level, and the static/dynamic split Puppeteer mentions makes sense to me. Perhaps the bigger benefit will be to newcomers; on large projects (which this will eventually become), one of the hardest parts starting out can be understanding the structure and knowing where to even look for the code you'd need to update, and the scene graph could help with that, at least on the front end. Events... that's tougher, I hear there's a C#-y way to do them that could likely apply on the back end, and the script interaction thing is another area where I'm a bit hazy on how to do it. I also have some concerns that it might be easy to introduce spaghetti code by having too many events tying too many things together all across the application, but I'm not sure how realistic that concern is.

I also didn't realize at the time that you could embed scenes within a scene when I added the advisor area. It may be a good idea to make that its own scene that can them be embedded. (I'm also thinking of the "advisor" incorporating all the F-key areas in Civ, including F7-F11)

But regardless of your choice of in-code tasks, enjoy your travels!

 

[WildWeazel]

I think I'll do a combination of the first and last: start with what we have and see what interfaces and layers I can extract, then expand on those to stub out a skeletal architecture.

As for signaling, I am not at all opposed to using Godot signals for UI/display only and otherwise shunning them in favor of native C# for game logic (the "model"). It would be a bit weird to have two signaling mechanisms, but in my mind it makes sense to use the UI (Godot) where we need to and keep the rest native C#.

and for events that cross the boundary we should be able to use the ParameterWrapper for the payload

Then you get into areas like interfaces where mods could hook in, and not having worked on a project where modules could be loaded dynamically (especially modules in another languages, such as Lua), I'm much less certain how to approach it.

My current project uses plugins in Python, and I've done Minecraft Forge modding which is based on event callbacks, so I have some experience with the pattern but I don't know yet how closely those ideas carry over to Godot and especially Lua. It would be great if we can find a working example of runtime Godot plugins.

(In other words, all those enterprisey projects technically use dependency injection, but they always inject the same dependencies in practice. I'm also thinking of interfaces as positive in the general sense, with Flintlock's recent tile layer interface being a good example. But not in the "everything must have an interface" sense, which I've seen on some enterprise projects. Might be hard to describe the difference, but basically interfaces for structure or where you might actually swap out an implementation are good, interfaces where you know there's a 99% chance you'll only ever have on concrete type, and you basically are adding the interface because you always have one, aren't)

Yup, I think that's an anti-pattern but I don't remember the name. Interfaces and DI certainly have their place, and it's not "literally every type". I'm a bit more liberal with events/signals, since you can always just ignore the ones you don't care about.

But regardless of your choice of in-code tasks, enjoy your travels!

Thanks! I'm flying in and out of CMH. If you're in town maybe we could grab a drink!

 

[Quintillus]

and for events that cross the boundary we should be able to use the ParameterWrapper for the payload

There's already an example of two of that going on. I had no idea whether that was the "right" way to do things, but it seems to be quite functional.

All the other things sound , no specific comments, except...

Thanks! I'm flying in and out of CMH. If you're in town maybe we could grab a drink!

Sent you an e-mail with my availability. It would be cool if the timing works out, I remember a small group of us was thinking of an Ohio meetup a few years ago, but it didn't wind up working out.

 

[WildWeazel]

So right away I'm hitting a design question re: event types. With being so heavily event-driven, naturally there are going to be a lot of event types and a lot of event callbacks. I see a few ways to organize events:

1. Many very specific event classes and/or signals (CombatStart, CombatHit, CombatLoss...)
1a. No relationship between types
1b. Inheritance and/or interfaces to relate types
2. Fewer broad types with variable content (CombatEvent, CityEvent...)
2a. Enums indicating what happened and what content to expect
2b. Polymorphic payloads similar to 1b

There are drawbacks all around: Option 1 means many similar registrations and callbacks, and likely a bunch of boilerplate code. Option 2 seems to break OO with kludgy type checks/casts and/or extraneous parameters. 2b is kind of a compromise but still needs type checks.

Maybe generic classes a la ParamaterWrapper would help? I'm not sure if Godot signals, or even C# events, get in the way of any of these concepts. This must be a solved problem though. Time to consult the internets.

 

[Quintillus]

Some progress that doesn't fit nicely on any milestone:

- I fixed the transparency issue with barbarian camps, which may be of interest to @Flintlock . The issue was that we were only treating the last PCX index as transparent, when it should be the last two. This was just the first PCX to use that second-to-last index.
- I made the barbarian warriors not-selectable. This code is horrible because we don't have enough concepts of players yet to make it proper. But after I'd clicked on them and realize it was selected, I wanted to fix it.

And the main area I'm investigating now is performance. After @Puppeteer added the victorious warrior, I noticed that the UI paused for a bit when I brought up the disband popup, specifically the warrior paused. So, I have:

- Creating a PcxPerformancePolishing branch
- Added a Stopwatch object for Game.cs and DisbandConfirmation.cs, which starts in the _EnterTree() method (called when a Godot object first enters the tree), and stops at the end of the _Ready() method. This lets us see how long it takes to load something, and I'm about 82% sure that it is being started and stopped in approximately the right place (although in the case of Game.cs, it would be even better if we could start it when you pressed the main menu button).
- Starting adding some caching for PCX and ImageTexture loading in Util.cs, suspecting that may be a good part of the load time. So far, this is the all-arguments version of LoadTextureFromPCX, and the LoadPCX method.

So far, the game scene doesn't load any quicker, but the DisbandConfirmation has gone from 225 ms, to 205 ms with only PCX caching, to 145 ms with texture caching. Which is still really slow for loading a popup, but if you told me upfront that 35 minutes could save me 35% or more on car insurance load time, I'd be pretty happy.

There's still a lot of room to explore that way too, notably a lot of PCXs and textures are not going through the caching load method and are loading from disk and/or recreating the texture every time.

Performance specs are with Civ3 loaded from a 7200 RPM, 2 TB hard drive, with a 3.6 GHz Core i5 2500k processor. I found it interesting that, based on the data so far, spinning rust is not the main limiting factor despite having egregious amounts of disk loads. Most likely the loads are close enough together that either the HDD cache is caching the data, or we're loading things very close to each other on the disk so it's close to sequential load times, or both.

(I also have not yet dealt with cache invalidation with this new cache, as that is a hard problem. But more relevantly, we probably don't need to invalidate it until the player goes back to the main menu, or we figure out a way to consume way more memory)

 

[WildWeazel]

Sounds like a winner! We should think about how to cache media in general, since it will surely be needed as we scale up. (What's the difference between PCX and image texture, and their respective dictionaries?) There could be a dedicated media cache class that maps relative paths to whatever data is loaded from whatever file it resolves to. Maybe even do conversions (when we get into new format support) and/or return a usable placeholder if there's no valid data there (solving the missing file errors). Then any code that needs media can just call MediaCache.GetMedia<Format>(path) and not care about files and formats.

I started thinking this should be a Component, but maybe not: I imagine all media loading will be in the Godot/UI layer, which may not be "allowed" in the game logic layer where ComponentManager will be.

 

[Puppeteer]

We should think about how to cache media in general, since it will surely be needed as we scale up. (What's the difference between PCX and image texture, and their respective dictionaries?)

I had been assuming that since C# objects are by-reference that we can reuse the reference to existing textures. e.g. We should only have to read a Warrior FLC or a map tile image once per session, no matter how many warriors or grassland tiles we draw. (This has been part of my challenge in figuring out how to do civ color in a way that does't require duplicating textures; I still think it's doable with shaders but haven't messed with it in a while.)

But I haven't tested it to date, and for UI popups we'll probably be dropping the texture references when we exit the popup scene? I also hadn't considered that Godot textures are actually C++ objects with C# references and whether that may impact whether or not a texture is assigned by reference or by copy in dual-environment reality.

If it were just C# we could just keep a collection of references to textures as a "cache" I think. (I haven't looked at what Quintillus has done.) By the time we get to Godot there is no difference between a PCX texture and a non-PCX texture. That *may* change with civ colors, but yet to be seen...we can either key the shader on a particular alpha value or maybe have a secondary mask layer or texture on which the shader operates. But anyway, the PCX code I wrote turns the image data into byte arrays for the palette and image data (by palette index), and the PCX to Godot code turns that into a Godot texture (represented by RGBA values of 0-1 float). Same with the FLC code, although add a dimension to the byte arrays for multiple images.

*If* the Godot textures can be tracked and reassigned by reference like C# objects can, then just keeping a reference to the Godot texture object is probably the easiest thing, although technically less memory efficient than keeping the palette and indexed byte arrays around.