[Dev] 0.1 "Babylon" Progress Thread

[Quintillus]

On the topic of "Do you guys think it's worth it to block game logic while waiting for the player to respond to popups?", and the associated Civ III exploits.

I'd propose we finish applying all the in-between-turn (IBT) calculations to the game before showing the player any popups, and set city production to "Not Selected", and have the previous turn's economic results already applied before the player makes any action. The player can still get city production pop-ups so it's easy to choose what's next, but all the things that are supposed to happen between turns will have actually happened by the time the next turn's interactions start.

Related to that, if the player for some reason doesn't choose production for a city, we can block end turn with a notice if they try to end it that, "Leeds doesn't have any production assigned", or something of that sort.

---------

On the original razing example, I think Civ IV made an improvement on Civ III by letting you examine the city before deciding whether to raze it. No more, "Oh, that was the city with Smith's Trading Company?" after razing it.

 

[Puppeteer]

Good points.

I presume it *is* possible in C# to run code synchronously. Looks like there is a wait method.

About interturn processing...oh yeah. I would propose we allow both. A lot of folks really want "Civ3 with limits removed" but otherwise exactly the same, which I presume includes things like age change popups where you can then switch to the trade advisor and do some trade shenanigans with any free techs earned. Same with free techs on wonder completion or first to Philosophy.

Maybe that could be a flag that mods could set. Or possibly each processing phase (research, tech, production, etc.) could have it's own "interruptible on interturn" flag.

I'm really hoping we can design this so we're not limited to 3 factors of production, and if we have pluggable fops then each fop will be heavily moddable in behavior.

 

[Quintillus]

I've wired up the Wait and Fortify buttons so they do what they're supposed to. For now, a unit that is Fortified is forever un-interactable, but that will be changing Soon(TM) now that units display on the map. Wait works just like you'd expect, using a queue behind the scenes so that Wait'ed units show up again in the order they are wait'ed - including if you wait a unit more than once on a turn, and if you end the turn without interacting with some Wait'ed units.

I've also switched out the hard-coded 4000 BC for a turn counter in the UI, which provides feedback that turn processing is going on. That area will evolve a lot over time (don't forget the month/week turn options!), but until the concept of years is present, I think this makes sense.

Most recently I've been adding keyboard shortcuts, and fine-tuning how the Advisor overlay interacts with keyboard shortcuts (found a gremlin I missed last night), as well as UI overlays. In the latter, my increasing knowledge of Godot means that I see the road to having buttons outside the Advisor area not be interactable while the advisor is open, just like in Civ; I also have wired up Escape to close the advisor. But I have to change how I did the centering to make that work, as the way I'd done it wound up being the "wrong" way, even though it did work for a time.

I keep getting distracted by singing sea shanties, though. At least they're good for morale.

Good points.

I presume it *is* possible in C# to run code synchronously. Looks like there is a wait method.

About interturn processing...oh yeah. I would propose we allow both. A lot of folks really want "Civ3 with limits removed" but otherwise exactly the same, which I presume includes things like age change popups where you can then switch to the trade advisor and do some trade shenanigans with any free techs earned. Same with free techs on wonder completion or first to Philosophy.

Maybe that could be a flag that mods could set. Or possibly each processing phase (research, tech, production, etc.) could have it's own "interruptible on interturn" flag.

I'm really hoping we can design this so we're not limited to 3 factors of production, and if we have pluggable fops then each fop will be heavily moddable in behavior.

I still think we could probably work age change shenanigans and bonus techs into the "after the turn is finished" side of things. Usually, tech would finish based on research accumulated, but the game engine could still auto-finish current research based on buildings, tell the UI 'the player needs to select free techs', and receive that update on the player's turn, validating that the player is eligible for a free tech when it gets that response (that's the network security part of me writing). Trade shenanigans might be a little tougher, but if we stick to "UI interactions are after the turn is finished", then the "Choose next tech" would only appear on the subsequent turn after entering an Age. Thus, we could similarly have a call to the engine for "give me my free Age tech", which the UI would call after the user has closed the "choose next tech" dialog, including potentially gone into the advisors and done some trade shenanigans. It might be a bit of a pain to work around, but it's still probably preferable to inconsistencies around what happens during the IBT.

Edit: I think different phases being interruptible is a lot of what got Civ III into trouble with e.g. the science slider exploit. The exploit came about because of the interruptions and not being able to anticipate how the user would use the UI to change the inputs into the next phase of the turn calculation, before the interruption ended. We could try to do the same thing but with more foresight, but I suspect we'd be setting ourselves up for missing edge cases just like Firaxis did.

---

By three factors, you mean food/shields/commerce, right? If so, it's an interesting thought, which crossed my mind the other night as well. I was reading up on GalCiv III on Explorminate last night, and saw today that they recently did an interview with Illwinter Game Design, whose games have a zillion types of production compared to Civ. I wonder if they code in each one or have a more general system? I feel like supporting more, particularly more than just "one new hard coded one" could be a challenge, though it could also open a lot of doors to expansion. I also wonder how configurable that is in Civ IV. I kind of want to say some mods (Fall From Heaven) had more types of production, but I am not sure about that. Some third-party games which use the Civ IV/V engine also appear to have new types of production (Warlock Master of the Arcane, Fallen Enchantress... though I haven't played either enough to speak with confidence), but if they licensed the engine they may have added their own types at the code level within the engine.

The game that comes to mind where adding more resources would (theoretically, famous last words) be easy is Victoria II, which has a whole production chain, and techs that modify the production rates of specific resources. They clearly thought about that upfront. But they also have a UI that's designed to accommodate that they might have had 6 resources on day 1 and 44 at the end. In all likelihood we'd need a new UI for more factors of production. The right click on a tile could handle arbitrarily many fairly easily, but what about city screens and other areas that would have to know about the new means of production?

Which is why I'm thinking that going beyond the Civ three means of production might be making it into another game. I could see an argument that by abstracting the means of production to one level, we could make it easier to add more types to the engine in the future. But I'm not convinced it's something we should try to make moddable before 2027, or that it would "play like Civ" if we did that. It almost reminds me of Jon Shafer's "At the Gates", which is tile based and has a bunch of types of resources, and is another game I should fire up again.

 

[Quintillus]

Please pardon the extra buttons that you'll see in C7! I've started adding a bunch of them, and plan to soon add the concept of enabling buttons based on the unit selected (and maybe the tile it is on, and maybe whether there's a boat to load onto, but probably not the latter two and almost certainly not the latter one right away). I'm also thinking about the best way to add unit button shortcuts in a less duplicitous fashion, and will be trying out multi-key shortcuts (Ctrl+C to chop forest, for example).

 

[Puppeteer]

Nah, keep it comin'!

 

[WildWeazel]

I presume it *is* possible in C# to run code synchronously. Looks like there is a wait method.

Yup. It's something I've never quite gotten around to understanding, but it's definitely a language feature. Back in my day we had Thread subclasses and explicit semaphores and we liked it that way avoided concurrency like the plague. You do have to be careful about it with engines though, sometimes you really need to use a built-in mechanism to keep Bad Things from happening.

I'm really hoping we can design this so we're not limited to 3 factors of production, and if we have pluggable fops then each fop will be heavily moddable in behavior.

I went down this rabbit hole and wrote up an analysis in the proposal on how we could plug in a public works fund (not a FOP but a commerce bucket) if the components are decoupled enough. With SOLID enough code you can change anything....

 

[Quintillus]

The Unit Action Buttons now will show/hide based on what is appropriate for the unit selected, as returned by the engine. You can see this as the buttons available change as you select different units. This is implemented in a general-purpose way, where the engine returns the keys of the buttons that should be visible - "road", "buildCity", "pillage", etc., so it should be easy to add the rest of the buttons as we go.

The more complex side will be the engine logic to determine which buttons should be visible. For now, I've hard-coded it to return a set based on unit type, but there will be a lot of factors that go in to it, including BIQ properties, terrain type (you can't build a city on a mountain in the default Civ3 rules), other units on the tile (you can only Load if the appropriate ship/helicopter/land transport is there), terrain improvements (you can't pillage if there's nothing there), etc. I fully expect that to be an iterative process.

I'll have to go over the Public Works Fund in the proposal document.

 

[WildWeazel]

I've always been morbidly curious what that combinatorial logic looked like, where some of them are configurable and some are not. (action:terrain, overlay:terrain, etc)

 

[Quintillus]

I've always been morbidly curious what that combinatorial logic looked like, where some of them are configurable and some are not. (action:terrain, overlay:terrain, etc)

We might have our own equivalent someday!

@Flintlock - I made a small change to the map view drawing code. I noticed that it was reaching directly into the engine to get all the units:

foreach (var vT in mapView.visibleTiles()) {
            int x = mapView.wrapTileX(vT.virtTileX);
            int y = mapView.wrapTileY(vT.virtTileY);
            foreach (var unit in C7Engine.EngineStorage.gameData.mapUnits)
            {
                //do awesome things

The problem is this leads to a hard tie between the Godot front end and the engine, which is problematic if the engine winds up being on a server somewhere. So, I added a couple methods to the EntryPoints folder in the engine to expose APIs for grabbing the map, and the units. I also made the "gameData" object on the engine "internal", so it cannot be accessed outside of that project.

It does raise some interesting points about how much the Godot side will need to know. It makes sense why it needs the whole map, and with three units, all the units makes sense. But it would be easy to stash those units somewhere on the Godot side and start modifying properties, and have the Godot side and the engine become out of sync.

I've brushed that under the rug so far, but it had occurred to me that the engine should really be returning unmodifiable objects to the Godot side, or at least making it slightly harder to accidentally wind up out of sync. How exactly that works in C#, I'm not sure, but for example in Java there's an UnmodifiableList class where if you try to add/remove items, it'll throw an exception, but you can still view them. Whether there's a good way to enforce that locally for objects (versus collections) is another question.

It might make sense to set up a network proof of concept in the not too distant future as well. I've been thinking of things in terms of the engine could be in "local" mode, for single player, where you still access it via APIs, but then it just accesses and returns the data locally and super quickly. Or, if it's in "remote" mode, you'd call the local APIs, but they'd see they were in "remote" mode, so they'd in turn call the equivalent APIs on a remote server, and return what they got back.

That might not be the ideal way to do things, just what I've thought of when trying to think how could we do it to combine both low latency locally (and not having to set up a local web server), and reliable network communication, without having to rewrite everything for the network version.

 

[Flintlock]

I've been thinking of things in terms of the engine could be in "local" mode, for single player, where you still access it via APIs, but then it just accesses and returns the data locally and super quickly. Or, if it's in "remote" mode, you'd call the local APIs, but they'd see they were in "remote" mode, so they'd in turn call the equivalent APIs on a remote server, and return what they got back.

This is more or less how I was thinking of implementing multiplayer except not for reading game data, only modifying it. We definitely do need a layer separating the UI from triggering changes to the game state. In a single player context we could feed state changes triggered by the player (AKA actions) directly into the engine, but in a multiplayer context those actions would have to be sent to a server where they would be checked for validity and if valid sent to all players. The client then would incorporate an action into its own game state only once it's been received back from the server. So all clients would have local copies of the entire state and they'd all stay in sync as long as the updates are deterministic, which they have no reason not to be.

The only disadvantage that I can think of to this approach is that players could cheat the visibility rules by inspecting the program's memory, but the Civ 3 MP community is small enough that that shouldn't be a problem. The alternative would be for clients to have only partial game states and update them periodically with partial updates from the server, but I don't think that would be worth the added complications. Complications like a new set of types to describe partial states, the fact that the partial state updates would have to be accurate otherwise the server would desync from all the clients, and the clients would need logic to request data they need but don't have.

And in any case, the display portions of the UI should be shielded from these concerns. The UI should display a snapshot of the game state and not worry about how that snapshot is being maintained. And for simplicity, the data types used by the UI should be the same as those used by the engine.

I don't know what's the best way in C# to prevent the UI from modifying the state directly, maybe we could do something clever with protected members? I think the official name for what I described above is the model-view-controller pattern so we might be able to find something by reading up on how that's usually done in C#.

 

[WildWeazel]

Just throwing this out there, FreeCiv used the same client-server model for single and multiplayer. So IIRC starting a single player game means spinning up a local server and setting all other players to AI. The neat thing about that is you could easily swap out AI, local, or remote players mid-game, and (but?) your AI is forced to play by the rules.

 

[Quintillus]

It probably does make sense for each player to have a local copy, at least for responsiveness. I think we're in agreement on the general approach.

I might have to look into how FreeCiv does it a bit more. The network (firewall, really) implications of starting up a server locally make me a bit hesitant. It's probably a somewhat outdated concern; at least on Windows the system is generally pretty good about giving a Windows Firewall prompt for programs that start servers, giving sensible defaults, and making that prompt visible (which older versions of Windows were not always good at). Presumably on Linux it's usually pretty lenient if it stays local?

Being able to swap players out mid-game is a cool feature. IIRC, Civ IV supports that in multiplayer as well. I'm not sure exactly how they implement it, but I recall players joining mid-game when I was playing multiplayer Civ IV Vanilla back in the summer of 2006, shortly before and after Warlords came out. Much less inconvenient if someone's connection temporarily dropped than having to go back to the lobby, like in EU III, and a lot of other games, really.

(By the way, did we get feedback at some point on the audience's desire for SP vs MP? That would help in knowing the relative priority/how much effort we should put into planning it, and at what stage. I don't want to paint ourselves into a corner, but in the absence of data, I'd favor getting SP up and working before putting tons of effort in MP. Although it would be cool to play each other in MP as we add new features)

 

[Quintillus]

It was pretty cool seeing the ability @Flintlock recently added to move units with the num pad! I see the "movement points indicator" has been added too. Silly question, what option do I check in Civ for the green indicator to show up? I see the red one when a unit has already moved, but not the green one (although I have Vanilla fired up currently, in an attempt to convince myself to focus on the more basic features first). I've moved Movable Units into the "In Progress" category for this milestone.

I have merged in the first batch of work on popups, and the Disband functionality, including the confirmation. There's still some extraction-of-common-popup-functionality work to do, but as Disband was the slowest-progressing bit of functionality I've done this fall, and one of if not the largest, I think I'm going to switch it up and do something else first, and then come back to it. Maybe see how that Godot 3.3 mp3 functionality works and add some main menu music.

(Although it is really nice compared to the huge chunks of work I've had on some project teams to have the largest chunk in one block thus far still weight in at less than 250 lines net, and that with some de-duplication still on the table)

 

[Quintillus]

It's been more than 24 hours without a post in this forum!

I've made a couple updates, including adding a Chariot with two movement points, and consolidating/changing some code so the status area updates properly when a unit moves but still has movement points left. I also extracted the popup background code so it's reusable and should be more efficient (far fewer children added to the scene graph).

I'll be traveling to Maryland for an outdoorsy vacation this week, which will likely result in fewer updates. Although the sun is setting awfully early, so I might not disappear entirely. I'll also be offline Saturday. Which isn't unusual, as it's college football season, but this Saturday is also the weekend I'm going to my friend's annual Independence Day celebration, which doubly guarantees I won't be on CFC.

 

[Quintillus]

Making some progress on "Place cities". There's now a popup where you can enter a name, press Enter to confirm the name (or Escape to cancel), and if you go ahead and build a city, it gets added to the game, and your Settler is no more.

Still have some UI polish to do on the popup, and the city does not yet appear on the map.

I'm pleased with how relatively easy this third popup was to add. It had some new twists (the text input area, for example), but I'm seeing dividends for having added a couple before and refactoring them a bit (though there is still some opportunity there as well).

Screenshot of the WIP:

 

[Quintillus]

@Puppeteer - I was pondering unit animations, since playing the build city animation would be cool. But I'm a little bit confused about the status of GodotCiv3 (in your example repo), versus the Civ3UnitSprite in ConvertCiv3Media (in the C7 repo).

Specifically, in GodotCiv3, there's this file/section, which seems like a fairly straightforward place to start from when understanding animations.

In C7, we have this file/section, which appears to (maybe?) be the same Civ3UnitSprite that Civ3Unit in GodotCiv3 uses. At a cursory glance, at least, they appear to be the same. The readme (for ConvertCiv3Media in C7) states:

This reads the unit INI file and loads a Flic[] property so that all of a unit's animations are in one object. It is intended to be inherited by Godot or Unity objects that translate the byte arrays into the native image formats.

So I'm thinking that Civ3Unit in GodotCiv3 does that translation project? Should we add that Civ3Unit class to C7?

Since it seems to be working well enough in GodotCiv3, I figured I'd ask rather than dig deeply/reinvent the wheel, although the latter can be fun in a game of Civ.

I see the notes about them being hacks, too... to me that's still a future problem, at least insofar as making sure C7 has all the GodotCiv3 functionality we want to carry over comes first.

(This also means I've started exploring the map code a little bit. I've also been starting to consider when we should start using Puppeteer's ConvertCiv3Media functionality to read in initial rules, and for now am thinking, "not quite yet, but probably fairly soon." The proposal puts "Save/Load" in Carthage, though it can be debated whether that includes load from scenario/biq, or save/load from save game. But I suspect we'll read the rules incrementally, just as we've been adding mechanics one at a time. For now, a few more mechanics (can't move a unit onto water, for example), a little bit more UI (animation), and perhaps a rudimentary AI opponent, seem slightly more important to me. We just have to be careful not to code too much around "unit name is Warrior", and to add proper data fields as needed)

 

[Puppeteer]

Hah! I'm reading this after pushing a hackish unit animation to the repo. Yep, using that code from the new LegacyUnit/LegacyUnit.cs file with the class Civ3Unit, pretty much lifted straight from GodotCiv3.

It worked a bit surprisingly well, but I quickly realized it's not tied to the map at all, by scale or positionally. Aside from copying and pasting that just now, I haven't touched the code in months, but I recall the idea was to have a unit class with Animate and Move methods with Enums for Direction and Action.

Since I'm kind of tossing the unit in there and not really banging on it right now, I was going to have him just loop the victory animation. But once I realized he doesn't magically pan or scale with the map I gave him movement as an example. Apparently the code to wrap the unit back around is in place, so he just skates around the screen doing the victory animation. Wheee.

Civ color is going to be handled differently. I seem to be passing it an integer for color now, but offhand I don't recall what the integer refers to. It defaults to 0 which seems to be white. OOoooooohhh, I think I remember now...I think it actually accesses the color definition files somewhere, and that's the index for it. Maybe that part of the API won't change after all, but I still someday soon want to take another crack at using shaders to handle civ color so we don't need to duplicate the texture for every player.

Reading the BIQ rules should be trivial, although work needs to be done. All the rules follow a pretty rigid structure unlike, say LEAD or CITY in the SAV. I have scaffolding for them in https://github.com/C7-Game/Prototype/blob/Development/QueryCiv3/BicSections.cs . When filled out it should look more like https://github.com/C7-Game/Prototype/blob/Development/QueryCiv3/GameSections.cs#L62 .

Actually there is one current error situation re: BIQ. I can detect if a BIQ is present at all, but it crashes on media-only BIQs, so I need to find the part where it says it has custom rules and then use the default BIQ if it doesn't.

Edit: Oh, I don't really have the SAV/BIQ reader in the 'mainline' code right now. It's in TempTiles under UtilScenes. I had been avoiding stuffing that into the prototype game because it really needs to be done a little differently; instead of drawing the map from save it should create a "native C7 map" and then render that, and map generation will populate the same structure.

(The current map generation is also a cardboard cutout as it only uses plains, grass, and coast. I feel like I need to figure out some more things about the Civ3 saves before I can progress on making the map generator make more terrain types.)

Edit 2: In short, the C7 map needs its own format/API, and then draw from that, and the legacy SAV reader and new generator will write to that format/API.

Edit 3: I guess I'll have to see if we have milked the GodotCiv3 example from my repo for all the useful code. I *think* the unit animation was the last bit missing, but I'm not entirely sure. Hmm, maybe the FLC to PNGs non-Godot example might be instructive in animating e.g. leaderheads. It's how I made the following–well, that and ffmpeg I think to make the frames into a video. But nah I guess it's really just a carbon copy of the unit FLC reading only we skip the .ini file and only have one animation per file for animated leaderheads.

 

[Puppeteer]

Some further explanation from memory on how units work. Wherever what I say differs from reality, reality is correct. Going to cover the basics as I'm not sure where everyone is on unit animation knowledge.

Each unit has an .INI file which specifies FLC animations for a number of predefined (hard-coded, I presume) actions. Not all units have all possible actions, because warriors don't build cities, and settlers don't attack, for example.

FLC is an ancient animation format made for when the time taken to read the image data from the file might slow your animation down, and the whole thing wouldn't fit in memory, probably. But Civ3 uses a modified version of it with 8 animations in each file. Same action, 8 different directions. (Real quick FLC description: run-length encoding, after the first frame only the differences are encoded between frames, and there is an extra difference frame as the last one to make it match the first full frame to allow looping...again, from a time when such saved disk-read and draw time that actually impacted animation speed. I mean I seriously remember unironically running FLC animations from floppy disk. Not for Civ3, but it was a very early format.)

So somewhere in the ConvertCiv3Media folder code it can parse the INI file and slurp the FLC animation frames into byte arrays, and then the Godot-side code turns the byte arrays into textures for sprite animation frames, 32-bit color, each frame uncompressed in-memory. Yay. But that's how Godot does animated sprites. And the animations and directions are referred to by predefined Enums for dev convenience. (That last bit is me, not Godot. In Civ3 there are only 8 directions.)

Animation speed and movement speed are hard-coded and eyeballed by me, but there are timings in the .INI file which can and should be used at some point. And I have the levers to animate in a loop and move continuously, but nothing so far for discrete timed events like "move from this tile to that one then go back to the default/idle animation", much less "advance half the distance while walking then do attack animation, etc.".

Incidentally, the .INI file also specifies sound files, but I haven't touched those yet. That will need to be thrown in with the "discrete timed events" mentioned above.

What's there works, but I won't claim that it's pretty or ideally coded.

 

[Puppeteer]

Overnight I realized I'm almost certainly overthinking unit pan & zoom. They're positional and scaled in Node2D space just like tiles are, so I need to either attach them as a child of the existing zoom/pan target Node2D or make a separate unit Node2D and have the zoom/pan controls adjust it, too.

The only trick will be having them wrap when the terrain does, but that's probably trivial?

And, of course, at some point we won't be saying "draw a warrior here" but instead have a unit instance referring to the BIQ analog, and the display code will know where to get all the pieces itself based on the references and game rules.

Edit: The ByteArrayToImage function is kind of duplicated between the terrain reader and unit reader because the special indexes (transparency, shadow, smoke, civ color) are handled differently. I put in code comments that they should probably be combined and refactored, but probably not in the near term. Because I think civ color and the other special indexes will be handled differently, and we'll probably have some classes or interfaces to pass that can tell the byte array to image converter how to handle the indexed color. (Of which Godot has absolutely no concept of, so we have to handle indexed to true color translation at conversion time, or keep the byte array in memory and repeatedly regenerate textures...which actually now that I think of it may be a viable option, but it's not my first choice.)

 

[Flintlock]

Overnight I realized I'm almost certainly overthinking unit pan & zoom. They're positional and scaled in Node2D space just like tiles are, so I need to either attach them as a child of the existing zoom/pan target Node2D or make a separate unit Node2D and have the zoom/pan controls adjust it, too.

The Node2D to attach to would have been MapView.unitView, but I went ahead and implemented my idea for loose map layers since I saw Quintillus working on cities and figured we'd need a city layer soon. So if you just want to draw something on the map easily, the way to do it is to create a class that implements ILooseLayer then add an instance of it to the list of layers. Right now the MapView constructor is hard-coded to create one layer for units but eventually we'll want to make that process more flexible for mods. ILooseLayer is very simple, it only has one function:

drawObject(LooseView looseView, Tile tile, Vector2 tileCenter);

"looseView" is the Node2D that you actually draw on and it holds a reference to the MapView object in case you need it. "tile" is the tile whose contents are to be drawn. "tileCenter" is the location to draw to. It's not a simple function of the tile coords due to edge wrapping and in fact the same tile might need to be drawn multiple times at difference locations. When drawing the contents you don't have to worry about the camera location or zoom because those are already applied to the looseView node by the MapView.

Edit: I also put an explanation of how ILooseLayer is supposed to work in a comment in the code.