One Turn Deserves Another III

[Puppeteer]

The Civ3 Clone thread got me to thinking about how that might work. History tells us I'm highly unlikely to have enough follow-through to get nearly complete, but I might try a few ideas.

For the game logic, Civ3 is a fairly basic loop:

• Calculate harvests, builds, research, maintenance, improvement progress, etc.
• Player moves
• Repeat for each player, including AI and barbs

The graphics are irrelevant to game play. I mean, sure, they're the UI, but this is a game of tiles, what's on the tiles, moving between tiles, and a bunch of settings and rules.

The key to making such a game is the data tracking and manipulation. I've been looking for an excuse to try out Reactive Programming, and this may be it. The facts of the game--tiles, units, cities, settings--might all be observables. The game rules would be implemented in filters, and then the game update loop and UI view would observe and update based on the rule filters.

For example, when calculating each worked tile's commerce harvest, the tile would have a base commerce value, and then depending on the government, civ traits, road presence, building modifiers, etc., that value would be modified by those stacked in-effect filters to produce the final modified commerce income for the tile.

There would certainly be a need for mod scripting, so I'd like to have the filters/rules include e.g. lua to allow 3rd-party logic. But that's way, way down the road.

For now, I grabbed the SVG map display parts from my Civ3 Show-And-Tell, stripped out d3, whipped up a little AngularJS scaffolding, and have a very basic view model that can display isometric tiles based on 3 variables: height, width, and an array of tiles. (Very boring screenshot attached.)

Next I'd like to start playing around trying to get maybe a unit or two moving around and/or see if I can get any observable/filter/observer logic going.

Eventually, if this grows I'll check into graphics frameworks that could at the very least use Civ3 graphics, and preferably allow much more. PixiJS looks promising. Or I might try out a non-browser front end. The back end could end up as Go, because I kind of have a browser/Go fetish the past couple of years, and Go has a Lua implementation and is cross-platform.

-----

Otda3 GitHub repo: https://github.com/midnightfreddie/otda3
civ-viewer Godot thingy GitHub repo: https://github.com/midnightfreddie/civ-viewer

 

[Puppeteer]

Oh, and this new putative game's title is "One Turn Deserves Another III" (Otda 3).

Not enough code to upload to GitHub yet, but here's some bits of what I added to the linked html page:

Spoiler :

<polygon ng-repeat="tile in game.mapTiles track by $index" transform="translate({{ game.x($index)}},{{ game.y($index)}})" points="0,32 64,0 128,32 64,64" class="terrbase2"></polygon>

(function(){
'use strict';

angular
.module('otda3', [])
.controller('Otda3', Otda3);

// Otda3.$inject = [];

function Otda3() {
var vm = this;
var width = 20;
var height = 20;
vm.mapWidth = width;
vm.mapTiles = [...Array(width/2*height)];
vm.x = function(i){
// tile-width * map width, also add half-tile offset to the right for odd rows
return 128 * (i % (vm.mapWidth / 2)) + 64 * ( Math.floor(i / (vm.mapWidth /2)) % 2);
}
vm.y = function(i){
return 32 * Math.floor(i / (vm.mapWidth / 2));
}
}
})();

 

[Puppeteer]

I'm currently in one of those phases where progress actually regresses because I'm refactoring/rebasing the code from AngularJS to Angular. Not that there was much "progress" to begin with. And I'm not redoing code so much as learning the differences between the Angular versions. (AngularJS I tended to use for everything-in-one-page quickies where Angular spews files and directories all over the place for "Hello, world!", and I need to learn my way around.)

But I've been thinking a lot about how things could work, and as I'm working on map view / UI I'm thinking:

• I think I'll do like Civ IV where you don't know your location in the world or even the world size. Your world map is only as big as what you've explored. I should be able to make this a per-game setting to make it work either way
• It occurred to me that there should be a cartography ability--surely obtained by Map Making in a CivIII tech tree--where if you don't have it, your knowledge of the map fades over time. Line-of-sight from units and cities/culture wouldn't fade, but beyond that it would until you know how to make maps. Again, this should be easy to set per-game.
• I'm wondering if resources should work like improvements. If you don't have line-of-sight, then what your map shows is what was there last time you had line-of-sight. The way I play, I try to scout as much and as early as possible, and then resources magically appear when you get the tech. But it's always bugged me a little...I haven't been there in 2000 years, so how would the map let me know there's iron there when I discover iron working? This could enhance the use of map trading.

Due to the way I'm planning to filter per-player info, any AI should also be affected by any visibility limits the player has. Any future AI should only have the same information to work with that a human player would. Hmm, or maybe there could be cheat-like concepts such as enhanced advisers that can give advice based on knowledge the player doesn't have, even if it doesn't show that detailed knowledg. And then difficulty levels could award the enhanced advisers to either a human player or an AI.

Real AI at this point is pretty much a pipe dream, but I'm already thinking the early iterations will necessarily develop a very rudimentary AI that may move completely randomly or perhaps be aggressive and attack anything it sees. And we're talking unit-movement-only here. Strategic play is not near-term.

 

[Quintillus]

A very good point that the game is fundamentally the loop you describe; the graphics are the presentation that makes the game digestible, but perfect graphics with nothing else would still not be the game we know and love. Although I'm also familiar with the concept that the best way to get someone non-technical to buy into your idea is to focus as much as possible on the UI part of it first. That's become my tactic for new ideas at work; it's always easier to gain traction if I make a neat semi-interactive UI mockup in a couple hours versus if I do all the complex stuff first.

I like all three of the ideas in post 3 (and the OTDA III name, for that matter). Fading LOS has always been on the "would make sense" list, and resource knowledge requiring re-scouting perhaps even more so. Of course I can't say for sure if I'd really prefer it in practice - it may be a bit tedious - but it would add another element and use for things like explorers, map trading, and scouting with airplanes. And the small minimap is one area where I must admit to liking the Civ4 version of things.

Do you have extensive Angular/AngularJS experience? I spent a good chunk of 2016-2017 working on a large AngularJS (1.4) project at work. Certainly learned along the way, sometimes the hard way in areas such as performance. I haven't moved to AngularJS yet, in part due to work projects changing, but also because of all the new things to learn (Typescript, npm, new Angular concepts, etc. - I admit it'd be sort of nice to be able to start with just the Angular 2 stuff and add the rest incrementally). I recently started a (non-Civ-related) side project myself, and for now have it in AngularJS. I probably will move it over at some point, but there were enough new HTML5 technologies I wanted to use that were more relevant to the problem I was trying to solve than Angular 2 was, that I started with those first.

If you do have a chance to move forward with this, I'm curious how you will find Angular to be for the purposes. Perhaps the single biggest challenge my team at work experienced was performance slowdowns caused by the cascades of chain reactions in calculations, and I can see that being a problem with a Civ-like game, too, probably even more so from a conceptual standpoint. Admittedly, a lot of our problems were due to the entire team being new to Angular at the start of the project, and consequently using it in a number of non-recommended ways due to not knowing any better. Things like excessive reliance on 2-way data binding, calling functions inline in the HTML ({{ something.calculateResult() }} , not taking advantage of Angular features that help with modularization (services, breaking up the scope, etc.). I know some of Angular 2's changes were specifically targeted at avoiding some of these problems. But I'll still be interested to hear how the performance holds up in practice. And if it seems like it's doing well enough in Edge or Firefox Quantum, there's always IE 11 to give it a dose of reality from a performance perspective. I've spent more than a couple days fixing "you might as well take a coffee break if you load the page in IE" issues .

 

[Puppeteer]

Well, there really isn't much practical progress since the first post. I've spent a lot of time reading about Angular 2 or whatever version I'm on...Angular 5.2.9, apparently. And moving my 2-file AngularJS blank map into a 39-file, 6-folder Angular 5 project. o_O But I think I see my way around Angular 2/4/5 now.

I couldn't resist getting a domain name (edit: I let the otda3.com domain go but updated links to a new location), and here's the blank map with a "person with ball" unicode character as a stand-in for a unit. The "unit" is hard-coded and has no meaning yet, but the map tiles have meaning and structure behind them in the code, and they're even being passed and consumed as observables. I also uploaded the code to GitHub.

https://lib.bigmoneyjim.com/otda3/alpha/1/index.html

https://github.com/midnightfreddie/otda3

A very good point that the game is fundamentally the loop you describe; the graphics are the presentation that makes the game digestible, but perfect graphics with nothing else would still not be the game we know and love. Although I'm also familiar with the concept that the best way to get someone non-technical to buy into your idea is to focus as much as possible on the UI part of it first. That's become my tactic for new ideas at work; it's always easier to gain traction if I make a neat semi-interactive UI mockup in a couple hours versus if I do all the complex stuff first.

Well, my audience is mainly me, but even I find myself losing interest if I'm not making it visually more appealing or more interactive. Heck, it might even be fun to play if warriors can move around the map and fight, and no other features are done.

I like all three of the ideas in post 3 (and the OTDA III name, for that matter). Fading LOS has always been on the "would make sense" list, and resource knowledge requiring re-scouting perhaps even more so. Of course I can't say for sure if I'd really prefer it in practice - it may be a bit tedious - but it would add another element and use for things like explorers, map trading, and scouting with airplanes. And the small minimap is one area where I must admit to liking the Civ4 version of things.

Thanks!

Do you have extensive Angular/AngularJS experience? I spent a good chunk of 2016-2017 working on a large AngularJS (1.4) project at work. Certainly learned along the way, sometimes the hard way in areas such as performance.

I've hardly touched Angular 2+. I wouldn't say I have "extensive" AngularJS (<2) experience, but it's one of those things that just clicked with me, and I've used it several times for quickie interactive web utilities. I like some of the ideas behind Angular 2+, but it's not the type of "pick it up and poke it" languages I tend to like. You really have to commit to learning *all* the opinions and conventions.

I haven't moved to AngularJS yet, in part due to work projects changing, but also because of all the new things to learn (Typescript, npm, new Angular concepts, etc. - I admit it'd be sort of nice to be able to start with just the Angular 2 stuff and add the rest incrementally). I recently started a (non-Civ-related) side project myself, and for now have it in AngularJS. I probably will move it over at some point, but there were enough new HTML5 technologies I wanted to use that were more relevant to the problem I was trying to solve than Angular 2 was, that I started with those first.

Yeah, it's really hard just to incorporate a piece at a time. I think I'm finally to the point where I've nibbled at the edges enough that I'm ready to go with it, at least for this project. In fact, I'm starting to think I'll just stick with all Angular 5 and skip the Go and Lua. (I might try JS mod scripting, or I may just have an editor UI that can handle most automation needs.) I have no idea how things will perform, but finding out is much of the point of doing this in the first place.

If you do have a chance to move forward with this, I'm curious how you will find Angular to be for the purposes. Perhaps the single biggest challenge my team at work experienced was performance slowdowns caused by the cascades of chain reactions in calculations, and I can see that being a problem with a Civ-like game, too, probably even more so from a conceptual standpoint. Admittedly, a lot of our problems were due to the entire team being new to Angular at the start of the project, and consequently using it in a number of non-recommended ways due to not knowing any better. Things like excessive reliance on 2-way data binding, calling functions inline in the HTML ({{ something.calculateResult() }} , not taking advantage of Angular features that help with modularization (services, breaking up the scope, etc.). I know some of Angular 2's changes were specifically targeted at avoiding some of these problems. But I'll still be interested to hear how the performance holds up in practice. And if it seems like it's doing well enough in Edge or Firefox Quantum, there's always IE 11 to give it a dose of reality from a performance perspective. I've spent more than a couple days fixing "you might as well take a coffee break if you load the page in IE" issues .

Angular 2+ seems to default to one-way binding for most things. The way this game looks in my head, I think the UI will be almost entirely one-way bound except for display settings like grid lines and map layers. I want the back end to control all rules; the UI shouldn't have to think "can this unit move West?" Either the back end will provide a set of valid moves, or the back end will return an error for invalid move requests. I think that will help a lot with those issues. But we'll see.

I think my next steps are to get a dumb move-randomly AI service and get 1 or more 1-unit-per-player units running around randomly, and then I can start working on having them attack when moving into an occupied square, fog of war, and so forth. Perhaps have simple cities that pop out a unit every X turns. Perhaps even implement start-of-game settlers and workers, but all workers could do is move and build a generic tile improvement that doesn't do anything yet. That will take quite a bit of doing, and then I'd know where I want to go from there.

Edit: Goofed up quote tags had hidden much of this post. Think it's fixed.

(edited for updated links)

 

[WildWeazel]

Good stuff here Puppeteer. Sounds like a good approach. Are you thinking about a web/online game, or just using a web renderer as the GUI?

For the game logic, Civ3 is a fairly basic loop:

• Calculate harvests, builds, research, maintenance, improvement progress, etc.
• Player moves
• Repeat for each player, including AI and barbs

The graphics are irrelevant to game play. I mean, sure, they're the UI, but this is a game of tiles, what's on the tiles, moving between tiles, and a bunch of settings and rules.

True enough, and especially so for turn-based games. From my prototyping attempt though, I think "how does the map work" may be the big question. It needs to be large, performant, scrolling, wrapping, and with each tile a complex unique object. The game loop is then relatively straightforward.

The key to making such a game is the data tracking and manipulation. I've been looking for an excuse to try out Reactive Programming, and this may be it. The facts of the game--tiles, units, cities, settings--might all be observables. The game rules would be implemented in filters, and then the game update loop and UI view would observe and update based on the rule filters.

Interesting. I've been wanting to use more ReactiveX but I hadn't thought about it that way. It should be especially useful for the various UI screens where you need to group and filter content.

There would certainly be a need for mod scripting, so I'd like to have the filters/rules include e.g. lua to allow 3rd-party logic. But that's way, way down the road.

Not sure how a scripting language would fit into the web architecture. Have you thought about how to integrate it?

I'm currently in one of those phases where progress actually regresses because I'm refactoring/rebasing the code from AngularJS to Angular. Not that there was much "progress" to begin with. And I'm not redoing code so much as learning the differences between the Angular versions. (AngularJS I tended to use for everything-in-one-page quickies where Angular spews files and directories all over the place for "Hello, world!", and I need to learn my way around.)

I've used AngularJS for exactly one large project. It's not my thing, but if you're into it more power to you.

It occurred to me that there should be a cartography ability--surely obtained by Map Making in a CivIII tech tree--where if you don't have it, your knowledge of the map fades over time. Line-of-sight from units and cities/culture wouldn't fade, but beyond that it would until you know how to make maps. Again, this should be easy to set per-game.

Neat idea. It could be as simple as keeping the "fog of war" layer disabled until the tech is discovered.

I'm wondering if resources should work like improvements. If you don't have line-of-sight, then what your map shows is what was there last time you had line-of-sight. The way I play, I try to scout as much and as early as possible, and then resources magically appear when you get the tech. But it's always bugged me a little...I haven't been there in 2000 years, so how would the map let me know there's iron there when I discover iron working? This could enhance the use of map trading.

Agreed. This bugs me too.

Due to the way I'm planning to filter per-player info, any AI should also be affected by any visibility limits the player has. Any future AI should only have the same information to work with that a human player would. Hmm, or maybe there could be cheat-like concepts such as enhanced advisers that can give advice based on knowledge the player doesn't have, even if it doesn't show that detailed knowledg. And then difficulty levels could award the enhanced advisers to either a human player or an AI.

I don't know if AI-to-AI advisor would be useful. Wouldn't this effectively be an omniscient AI feeding information to a limited AI? Seems like that effectively the same as a cheating AI.

Real AI at this point is pretty much a pipe dream, but I'm already thinking the early iterations will necessarily develop a very rudimentary AI that may move completely randomly or perhaps be aggressive and attack anything it sees. And we're talking unit-movement-only here. Strategic play is not near-term.

I'm in the same boat. Do we have anyone here who can do real AI? I'm not even sure what technologies games use, now or back in this era.

 

[Puppeteer]

My last post was half-missing due to goofed-up quote tags, but it's fixed now.

https://lib.bigmoneyjim.com/otda3/alpha/2/index.html - Now interactive, barely. Click on / touch the unit, and it will teleport somewhere else on the map. Not very game-y yet, but the unit is now data-modeled with health, rank, and location, and there can be many of them. Next I can start thinking about movement, wrapping when moving off-map, detecting invalid moves, attacks, etc.. Oh, I guess I need to add a civ ID to the unit class, too.

(Edit: For the fun of it, here's the same thing but with 10 uniits: https://lib.bigmoneyjim.com/otda3/alpha/3/index.html . Although more than one can occupy the same tile at once.)

Good stuff here Puppeteer. Sounds like a good approach. Are you thinking about a web/online game, or just using a web renderer as the GUI?

Thanks! The short answer is that I want it to be able to run local-only with no Internet connection necessary.

The long answer is that my true goal is to tinker with technology, so I'm prone to radically changing things mid-stream. I originally thought I'd end up with a Go back end and maybe a browser front end, but I'm already curious how far this can go in JavaScript only. There are deep learning and therefore fast math libraries implemented in JS with in-browser hardware acceleration, so I'm wondering if a 2018 browser JS can perform as well as 2000 C++. I think most of the problems with Civ3 slowness were due to trade route calculations, and if I get that far I imagine the modern math libraries can help with that.

As it is now, it makes a lot more sense to show my progress online than to expect anyone to download and try it out themselves.

True enough, and especially so for turn-based games. From my prototyping attempt though, I think "how does the map work" may be the big question. It needs to be large, performant, scrolling, wrapping, and with each tile a complex unique object. The game loop is then relatively straightforward.

As long as I'm working in SVG I'm counting on existing libraries and math to only put the necessary objects in the DOM, and I think the RxJS can help with filtering appropriate units, cities, improvements, etc..

Once I try going to canvas/PixiJS, all bets are off. From what little reading I've done so far it may involve frequent redrawing of the whole canvas. But there are a number of other things I want to do before worrying about that, and even if I go that route it might be possible to maintain the SVG version as an option, too.

Not sure how a scripting language would fit into the web architecture. Have you thought about how to integrate it?

Well, again, I originally thought the back end would eventually be in Go, and there's a lua VM available for Go. If the back end stays JS, it would probably make more sense to use JS for scripting, but there are challenges sandboxing that.

The idea is morphing now. As of last night I was thinking instead of freeform scripting the two mod filters needed would be to alter a number (e.g. +50% bonus to something) which could easily be done with some sort of unscripted editor, or if/then logic to key on or trigger complex actions involving several factors.

But now I'm thinking an event/messaging bus--or observables--could handle this type of logic without exposing scripting. Any mod filter would list all the events/facts needed to be in effect, and mods could emit their own events, and I think although it might be a little messy it would allow a lot of versatility in radically altering game rules.

Neat idea. It could be as simple as keeping the "fog of war" layer disabled until the tech is discovered.

From my map project I know that Civ3 improvements are stored in the map per-civ, but everything else comes from the real map, so you won't see if improvements have changed, but you'll see if forests are cut or planted because those are terrain overlays, and you'll see if cities are plopped and if cultural borders grow without LOS. So I'm starting to think each civ will have its own version of the map, and all LOS tiles are updated every unit move, and when culture expands. And then those tiles will have a turn count or date stamp and gradually fade when out of LOS unless map making is known, and if MM is known then they won't update when out of LOS. Depending on how it comes together, to allow Civ3 behavior...well, too many possibilities to go over; need to get some real code/data in place before figuring out the approach to how to change behavior.

I don't know if AI-to-AI advisor would be useful. Wouldn't this effectively be an omniscient AI feeding information to a limited AI? Seems like that effectively the same as a cheating AI.

I want to avoid omniscience, but perhaps e.g. the enhanced military adviser would know relative civ military strength like in Civ3 which could inform (well, add a modifier to) a filtered-visibility AI whether starting a war of conquest against a neighbor is more or less wise than usual. So not unit type & location or city garrison, but an accurate assessment of overall civ strength.

I'm in the same boat. Do we have anyone here who can do real AI? I'm not even sure what technologies games use, now or back in this era.

Anyone who can do real AI is probably too busy making bank at a pharmaceutical or financial company. I'm intrigued and have dabbled with some ML tools, and I've got some vague ideas on how I might attempt such a thing on a limited scale, but the whole big strategic picture? I can't even fathom that from here.

I am starting to think that I can get this to be a playable toy that is all-war and not too complex. With limited options and only kill everyone, defend territory as goals, I might even be able to make that happen. Training a per-unit offensive model I think is doable. For defense might just hard code to prioritize a garrison size and health.

Stacked attacks and use of defensive map tiles? Hmm...would have to see what a limited contrived training could come up with.

(edited for updated links)

 

[Nanuk]

I really like your proposed game title. So much so I felt compelled to post how much I like it. Very creative.

One Turn Deserves Another II - the sequel
One Turn Deserves Another III More - the third game
One Turn Deserves Another IV - the fourth game

 

[Puppeteer]

https://lib.bigmoneyjim.com/otda3/alpha/4/index.html - Now instead of teleporting, they move to an adjacent space and can wrap at all edges of the map. Three units take turns auto-moving every half second, but you can still click on them to make them move in a random direction.

Enabling combat shouldn't normally be difficult, but I'm making it so because of how I want to implement the game rules/logic. I might have to tinker around a lot before making progress on any actual game mechanics.

Fog of war is more a data model and view thing, so I might tinker with that without having to think too hard about rules and mechanics.

I'd kind of like to arrive at a point where units can attack each other, and perhaps have rudimentary cities and improvements. The cities would initially act like unit-generator wonders, popping out a generic battle unit every few turns. The improvement wouldn't do anything for now, but give maybe a starting worker something useless to do just to have non-combat units modeled. Once I get to that point it might be fun to try out AI models against each other. A game with a single combat unit type and a goal of destroying the opponents' cities could be something to train a neural network against.

I really like your proposed game title. So much so I felt compelled to post how much I like it. Very creative.

Thanks! I'm also enjoying the name far more than anyone should, and I'm glad others like it.

(edited for new url)

 

[Quintillus]

Well, there really isn't much practical progress since the first post. I've spent a lot of time reading about Angular 2 or whatever version I'm on...Angular 5.2.9, apparently. And moving my 2-file AngularJS blank map into a 39-file, 6-folder Angular 5 project. o_O But I think I see my way around Angular 2/4/5 now.

I couldn't resist getting a domain name, and here's the blank map with a "person with ball" unicode character as a stand-in for a unit. The "unit" is hard-coded and has no meaning yet, but the map tiles have meaning and structure behind them in the code, and they're even being passed and consumed as observables. I also uploaded the code to GitHub.

https://www.otda3.com/alpha/1/

https://github.com/midnightfreddie/otda3

I've hardly touched Angular 2+. I wouldn't say I have "extensive" AngularJS (<2) experience, but it's one of those things that just clicked with me, and I've used it several times for quickie interactive web utilities. I like some of the ideas behind Angular 2+, but it's not the type of "pick it up and poke it" languages I tend to like. You really have to commit to learning *all* the opinions and conventions.

Golly gee, 2 files to 39 files is quite the file explosion. I must admit to liking that I can do things like I did last Thursday - swing down to the company coffee shop for 75 minutes, and put together a (literally) 2-file AngularJS program that provides legitimately useful information to address a problem my team was having. Not saying it would necessarily take longer in Angular 5.2.9 - and of course most of that 75 minutes was processing the practical data - but it sounds more complex.

I must also complement you on the default encryption on your website, it's quite strong, at least when viewed with Firefox. I've been doing way too much research on that sort of thing recently.

I'm in the same boat. Do we have anyone here who can do real AI? I'm not even sure what technologies games use, now or back in this era.

Anyone who can do real AI is probably too busy making bank at a pharmaceutical or financial company. I'm intrigued and have dabbled with some ML tools, and I've got some vague ideas on how I might attempt such a thing on a limited scale, but the whole big strategic picture? I can't even fathom that from here.

Real AI? Not for a game, at least. I did some AI work in college, but it was in a completely different part of AI, more academic than game, machine learning, or big data. I have read some on multi-level game AIs (where the AI considers things strategically, then in theaters, then at fronts, then individual units, or something like that), but it's all theoretical, no practical experience or hands-on with the technologies.

And while a financial company is paying my fairly well, the work I'm doing doesn't have anything to do with AI or ML (though a bit with big data), and they aren't smart enough to assign me to do what would give them the most benefit from my skills.

https://www.otda3.com/alpha/4/ - Now instead of teleporting, they move to an adjacent space and can wrap at all edges of the map. Three units take turns auto-moving every half second, but you can still click on them to make them move in a random direction.

Enabling combat shouldn't normally be difficult, but I'm making it so because of how I want to implement the game rules/logic. I might have to tinker around a lot before making progress on any actual game mechanics.

Fog of war is more a data model and view thing, so I might tinker with that without having to think too hard about rules and mechanics.

I'd kind of like to arrive at a point where units can attack each other, and perhaps have rudimentary cities and improvements. The cities would initially act like unit-generator wonders, popping out a generic battle unit every few turns. The improvement wouldn't do anything for now, but give maybe a starting worker something useless to do just to have non-combat units modeled. Once I get to that point it might be fun to try out AI models against each other. A game with a single combat unit type and a goal of destroying the opponents' cities could be something to train a neural network against.

That was pretty cool to click the new link and see the units moving around before reading enough to know they'd do that. I like the idea of a minimally viable game, where either you could have the AIs play each other, or allow someone to play "civ with only warriors as units". Of course there's no guarantee that the AI model that does well with that would hold up in a more complex simulation - though elements of it may prove useful - but having even a trivially playable game would be a significant milestone.

 

[Puppeteer]

Golly gee, 2 files to 39 files is quite the file explosion. I must admit to liking that I can do things like I did last Thursday - swing down to the company coffee shop for 75 minutes, and put together a (literally) 2-file AngularJS program that provides legitimately useful information to address a problem my team was having. Not saying it would necessarily take longer in Angular 5.2.9 - and of course most of that 75 minutes was processing the practical data - but it sounds more complex.

Yeah, that's why it took me so long to actually try the newer versions. It's just not pickup-friendly. There's a method to the madness, though. I just don't like having files in my repo unless I know what they're for and that I'm using them, and that's only the first learning curve.

Each component has 4 files - template, css, script, and spec (unit test). And then each service has a script and spec, and then there's configs for the TS transpiler, unit tests and end-end tests, and so on and so forth which the CLI "helpfully" generates for you whether you're using it or not. Then there are modules, and so on and so forth. Lots of files for the most basic app. Of course it all ends up bundled, optimized and minimized for deployment.

I must also complement you on the default encryption on your website, it's quite strong, at least when viewed with Firefox. I've been doing way too much research on that sort of thing recently.

Thanks! I'm going for encrypting everything now. For self-managed servers there's LetsEncrypt and Certbot, but I've also managed to use AWS S3 with CloudFront for TLS which seems like overkill, but it gets you AWS certs and scales down really well, so it's cheaper than a VM. (Actually I kind of hate Certbot's requirement to run as root and try to manage everything, but I run it in a container and route all my sites' /.well-known/ to the container and use http verification. Then I copy the certs to their final destinations.)

That was pretty cool to click the new link and see the units moving around before reading enough to know they'd do that. I like the idea of a minimally viable game, where either you could have the AIs play each other, or allow someone to play "civ with only warriors as units". Of course there's no guarantee that the AI model that does well with that would hold up in a more complex simulation - though elements of it may prove useful - but having even a trivially playable game would be a significant milestone.

Yeah, I should probably aim to make it minimally playable earlier rather than later. I haven't had a chance to work on it lately, but the mental gears are turning. I'm thinking a minimally-playable game would be fun to mess with a rudimentary AI that might even be fun to play against. I guess at worst that can be the barbs' AI.[/QUOTE]

 

[Takhisis]

/subscribes to thread quietly

 

[Puppeteer]

I've been very busy at work the past couple of weeks, but I've opened the project several times intending to do something, then simply overthinking everything and not accomplishing anything.

I probably need to focus on my original intent: get game logic implemented with Rx, and get something playable if only in a simple generic unit-vs-unit bash fest. But I keep getting distracted by graphics possibilities and future complexities.

 

[Quintillus]

Yeah, that's why it took me so long to actually try the newer versions. It's just not pickup-friendly. There's a method to the madness, though. I just don't like having files in my repo unless I know what they're for and that I'm using them, and that's only the first learning curve.

Each component has 4 files - template, css, script, and spec (unit test). And then each service has a script and spec, and then there's configs for the TS transpiler, unit tests and end-end tests, and so on and so forth which the CLI "helpfully" generates for you whether you're using it or not. Then there are modules, and so on and so forth. Lots of files for the most basic app. Of course it all ends up bundled, optimized and minimized for deployment.

Interesting... I can see the point, those 4 files would make the component modularized, so you could easily re-use it in other projects. And it encourages good practices. Still, a bit overkill if you're just going for a prototype. Of course, sometimes prototypes wind up becoming the real deal...

Thanks! I'm going for encrypting everything now. For self-managed servers there's LetsEncrypt and Certbot, but I've also managed to use AWS S3 with CloudFront for TLS which seems like overkill, but it gets you AWS certs and scales down really well, so it's cheaper than a VM. (Actually I kind of hate Certbot's requirement to run as root and try to manage everything, but I run it in a container and route all my sites' /.well-known/ to the container and use http verification. Then I copy the certs to their final destinations.)

I need to look into /.well-known/ a bit more. All I really know about it so far is that it's where you put the security.txt that tells people who to contact in the event of a security problem.

I wound up going with Comodo for my site's certificate, but only because I couldn't find a nice guide on how to integrate Let's Encrypt with Java web servers, and couldn't be bothered to spent too much time on it when the certificate was $9 for a year. I suppose I could've put nginx or Apache in front of my Java web server, but oh well, I have it all configured with Jetty now and $9/year isn't going to break the bank.

Yeah, I should probably aim to make it minimally playable earlier rather than later. I haven't had a chance to work on it lately, but the mental gears are turning. I'm thinking a minimally-playable game would be fun to mess with a rudimentary AI that might even be fun to play against. I guess at worst that can be the barbs' AI.

I can identify with the overthinking + work problem. Sometimes I find my habit of keeping a list of tasks helps with that, as I can pick a task and focus on it. Other times it just gives me too many ideas and I can't decide on one.

 

[Puppeteer]

I've spent much of the time since my last post battling pink eye. Whee. So still no accomplishments, but some divergent thoughts:

The original aim and most productive focus is for me to get more logic implemented using Rx and--for now--Angular 5. One likely-dead-end distraction is to implement hex tile maps and square maps with all map types extending a base map class. I was also thinking of an isometric map--the current map I call "fake isometric" but have seen it referred to as "staggered isometric"--but then realized the non-view parts of a true isometric map are identical to a square map. But the exercise gets me experience using Typescript inheritance, so it's worthwhile as my primary motivation is tinkering an learning as opposed to a final product.

A second distraction is Godot. After it was mentioned in the clone thread (linked in OP), I hadn't heard of it so went to check it out, and I'm intrigued. I have no current interest in coding Otda3 logic in Godot, but I'm considering it as a view engine for my logic, ideally (well, stubbornly) passing data over websocket or similar messaging. In addition to being free and multi-platform, it can even compile to webassembly and work online, too. But run locally I think it might be one of the easier ways to both use existing Civ3 graphics *and* allow for alternate mod graphic formats, even mixing them if desired.

Some Godot ideas, although I don't know how practical these are:

• Read existing Civ3 files natively
• Use shaders to translate native Civ3 file transparency and civ colors
• Shaders to play with / animate unit civ colors and UI indicators
• Use particle effects for fun
  • WLTK day
  • Some attacks
  • Special events like tech or GL
• Main gameplay UI
• 3d models intermixed with 2d graphics

Some downsides to spending time in Godot:

• It doesn't really boost my desired skill sets; it's entirely transient
• It's kind of silly to insist on keeping the game logic out of the engine (another "me" problem, not a Godot problem)
• It might distract from the other, more-likely-to-survive development project, or even worse attract effort away from it. (The project is a broken toy until the logic is finished, and I'm not likely to finish it, and it's not a design/tech that other potential coders would pick up and run with if abandoned.)

But, it's shiny and distracting, so I might kick its tires more. (I compiled an iso-tiled example project to HTML5/WebAssembly, and it just worked. -ish. But it doesn't translate well as-is to my needs.)

Here's the in-browser Godot example (warning: 20mb download!): https://www.otda3.com/godotpoc/foo.html . Should work with browsers supporting WebAssembly (Chrome and Firefox). Might need to click in the "game" then use arrow keys to move the troll around. (Can't do anything but move around and bump into walls.) This isn't what Otda3 would look or play like, but for me it's a proof of concept for isometric tile maps built into Godot. I wouldn't use the collision or movement code, just display tiles and various overlays and units, but with all the graphics oomph of Godot.

 

[Puppeteer]

Some Godot ideas, although I don't know how practical these are:

• Read existing Civ3 files natively
• Use shaders to translate native Civ3 file transparency and civ colors

I exported the popheads pcx file as an indexed png which Godot can read, made the whole thing a sprite, and tried a shader to implement the transparency. The first screenshot looks good at first glance, but as can be seen better in the second 2x screenshot, even the first screenshot has some pink blended into the edges. (Edit: Added a 3rd screenshot where I threw an image behind the popheads to demonstrate the transparency.)

But the bigger problem seems to be that the shader only sees the 32-bit color and not the 256-color palette version, so I can only key on the color which is problematic for a number of reasons.

I started thinking about coding Godot's gdscript to natively read the files, but it's just not worth the effort. It will make a lot more sense to use existing working tools to batch-convert Civ3-native files into a format more easily consumable by Godot/gdscript. I'm thinking of stuffing byte arrays of image data and palettes into a key-value store or a SQLite database...or maybe even just files readable by the binary serialization API. I just need the raw palette and image data to refer to and use either gdscript or shaders to construct the sprites on the fly.

At first I worried that using batch conversions would make it harder to be cross-platform, but now that I think of it, I probably only need ImageMagick/GraphicsMagick and ffmpeg which are both cross-platform tools.

Spoiler Shader code :

shader_type canvas_item;

render_mode unshaded;

void fragment(){
    vec4 transparent = vec4(1.0, 0.0, 1.0, 1.0);
    vec4 shadow = vec4(1.0, 0.33, 1.0, 1.0);
    vec4 mycolor = texture(TEXTURE, UV);
    if (mycolor.rgb == transparent.rgb) {
      COLOR.a = 0.0;
    } else {
        if (mycolor.rb == shadow.rb && (mycolor.g - shadow.g) < 0.1) {
            COLOR = vec4(0.0, 0.0, 0.0, 0.5);
        } else {
            COLOR = mycolor;
        }
    }
}

 

[Puppeteer]

Wow. ImageMagick doesn't seem to offer a way to simply dump the palette. Grabbing it from pcx files should be easy enough: just grab the last 256*3 bytes of the file. But the image-as-palette-indexes isn't that easy because it's modified-RLE-encoded. If I do this, I'm going to have to go lower-level, but I expect it will still be faster/easier to go find an old pcx reader library probably in C or C++ than to implement one in gdscript or one of my preferred languages from scratch.

For flc I had vaguely figured a workflow of flc -> ffmpeg -> series of frame image files -> convert them as I do the pcx files. But now I'm concerned that flow will be problematic, too.

Then again, pcx/RLE looks pretty simple, and Civ3-file-compatibility is all I care about, so that might be trivial to implement a reader from scratch.

But, a very busy week of work is ahead of me, so I probably won't have a chance to touch this again for at least a few days.

 

[Puppeteer]

Well, I touched it some. I found if I remove the "Filter" flag and reimport the texture, much of the blending goes away. In the screenshot if you zoom in you can still see some magenta blending, especially with the ladies' hair in the lower-right area. I expect if I tried long and hard enough I could eliminate the remaining blending.

But using the shader is still not the right move, so I'm planning on trying to generate a sprite from code. Well, actually I did that already: The little Godot robot icon in the upper-left is a saved scene that was preloaded and instantiated by gdscript on the parent node script.

The next step will be to hard code mini "image" (probably 4x4) and "palette" arrays and prove to myself that I can make an image from code and sett it as the texture on the instantiated sprite. After that I may start looking into decoding actual pcx and flc files and convert them into the image/texture and replace the special palette entry colors in that conversion instead of a shader.

Spoiler gdscript that instantiates a sprite :

extends Node2D

const TESTSPRITE = preload("res://testsprite.tscn");

func _ready():
    # Called every time the node is added to the scene.
    # Initialization here
    var mysprite = TESTSPRITE.instance();
    add_child(mysprite);
    pass

 

[Puppeteer]

The next step will be to hard code mini "image" (probably 4x4) and "palette" arrays and prove to myself that I can make an image from code and set it as the texture on the instantiated sprite.

Success! The sprites are scaled 10x so we can see them, and the scaling is trying to smooth them, so there are colored fringes.

The first image was my 4x4 hard-coded test. The next two are 8x8 pixels built from a 64-byte array referencing the palette array which is a 256-element PoolColorArray (essentially RGBA color vectors, 8 bits per channel). The middle one has some weird unexpected transparency thing happening...oh, I see now what happened there: there are different ways to build a color value, and apparently when using hex string encoding the order is ARGB not RGBA.

So basically if I can decode the pcx and flc files' palettes and the image data I know I can build sprites from them using this same method. Cool.

Spoiler gdscript palette array to sprite :

extends Node2D

var mypalette = PoolColorArray()

func _ready():
    # Set color palette size
    mypalette.resize(256)
    mypalette.set(0, Color("000000"))
    mypalette.set(1, Color("ffffff"))
    mypalette.set(254, Color("00ff00"))
    mypalette.set(255, Color("ff00ff"))
    var mywidth = 8
    var myheight = 8
    var myimagedata = [
        0, 0, 0, 0, 0, 0, 0, 0,
        0, 0, 0, 0, 0, 0, 0, 0,
        0, 0, 0, 0, 0, 0, 0, 0,
        1, 1, 1, 1, 1, 1, 1, 1,
        1, 1, 1, 1, 1, 1, 1, 1,
        1, 1, 1, 1, 1, 1, 1, 1,
        254, 254, 254, 254, 255, 255, 255, 255,
        254, 254, 254, 254, 255, 255, 255, 255,
    ]
    # sprite from code only
    var sprite_from_code = Sprite.new()
    sprite_from_code.position.x = 50
    sprite_from_code.position.y = 50
    # create an image from code
    var image_from_code = Image.new()
    # create/format image data
    image_from_code.create(mywidth, myheight, false, image_from_code.FORMAT_RGBA8)
    # assign pixels
    image_from_code.lock()
#    image_from_code.set_pixel(0,0, Color("ff00ff"))
#    image_from_code.set_pixel(1,0, Color("00ff00"))
#    image_from_code.set_pixel(0,1, Color("808080"))
#    image_from_code.set_pixel(1,1, Color("00000080"))
    for i in range(myimagedata.size()):
        image_from_code.set_pixel(i % mywidth, i / mywidth, mypalette[myimagedata[i]])
    image_from_code.unlock()

    # Create texture from image
    var texture_from_image = ImageTexture.new()
    texture_from_image.create_from_image(image_from_code)
    # apply texture to Sprite
    sprite_from_code.texture = texture_from_image
    # Scale up test sprite
    sprite_from_code.apply_scale(Vector2(10.0, 10.0))
    # add sprite to scene
    add_child(sprite_from_code)
    pass

Edit: Added 4th and 5th images. The 4th is what the 2nd was meant to be, the stand-in colors at half transparency. The 5th image has the logo behind to show the transparency.

Edit 2: I'm realizing something in the engine is expecting the sprite texture to be continuously tiled, so it's blending the edges as if they'll be next to the opposite side. I'll have to figure out how to turn that off or otherwise work around it, although I'm sure the problem is greatly exaggerated here by the 10x scaling.

 

[Puppeteer]

Wow. In one evening I got the PCX reader code working! Most of it just worked as I coded. The big hangup turned out to be that Godot didn't like it when I was reading from a file and writing to an image object a byte at a time. I read to a byte buffer, closed the file, then write to the image object, and it's happy now.

Well, I touched it some. I found if I remove the "Filter" flag and reimport the texture, much of the blending goes away. In the screenshot if you zoom in you can still see some magenta blending, especially with the ladies' hair in the lower-right area. I expect if I tried long and hard enough I could eliminate the remaining blending.

The magenta fringes are still there; it turns out they're in the original. There must be more than two transparent values.

But using the shader is still not the right move, so I'm planning on trying to generate a sprite from code.

This is still in effect. Even though I might have been able to make one graphic import look good with shaders, it wouldn't have worked universally. The new method where I read the PCX file will work on any PCX file Civ3 uses.

The popheads don't look any different from the quoted post, so I won't put a redundant screenshot. The one I'm attaching was my first almost-full success but is also amusing. It's distorted because on an extra junk pixel at the end of each row, and it's incomplete because only the lower-right quadrant is showing, but that was just a "centered" property I had to disable.

This is pretty significant. All I need to do is figure out the rest of the special palette entries, substitute them, and this routine should be able to read any Civ3 PCX file. Breaking the composite images into regional sprites should be trivial. The bigger trick will be tying it all--or any of it--to game logic.

Also, I'll start trying to decode FLC files next. Getting those to animated sprites would be nice.

Edit: Oh, since I *can* read other files now, I should do so. Done. Added a couple of terrain screenshots; Godot icon added to show transparency.

Spoiler gdscript - create sprite from pcx file :

extends Node2D
var mypalette = PoolColorArray()
var image_from_code = Image.new()
func _ready():
    # Set color palette size
    mypalette.resize(256)
    readpcx()
    # set shadow and transprent color values
    mypalette.set(254, Color("80000000"))
    mypalette.set(255, Color("00000000"))
    # sprite from code only
    var sprite_from_code = Sprite.new()
#    sprite_from_code.position.x = 50
#    sprite_from_code.position.y = 50
    sprite_from_code.centered = false
    # Create texture from image
    var texture_from_image = ImageTexture.new()
    texture_from_image.create_from_image(image_from_code)
    # apply texture to Sprite
    sprite_from_code.texture = texture_from_image
    # add sprite to scene
    add_child(sprite_from_code)
    pass
func readpcx():
    # not a generalized pcx reader
    # assumes 8-bit image with 256-color 8-bit rgb palette
    var file = File.new()
    file.open("popHeads-ORIG.pcx", file.READ)
    # seek to margins
    file.seek(0x4)
    var leftmargin = file.get_16()
    var topmargin = file.get_16()
    var rightmargin = file.get_16()
    var bottommargin = file.get_16()
    var width = rightmargin - leftmargin
    var height = bottommargin - topmargin
    print(width)
    print(height)
    # seek to bytes per scanline; assuming 1 color plane
    file.seek(0x42)
    # this is always even, so last byte may be junk if image width is odd
    var bytesperline = file.get_16()
    print(bytesperline)
    var imagelength = bytesperline * height
    print(imagelength)
    # seek to palette, 256*3 bytes from end of file
    file.seek_end(-(256*3))
    var palettebytes = file.get_buffer(256*3)
    var palette = PoolColorArray()
    palette.resize(256)
    for i in range(256):
        palette[i] = Color(
            palettebytes[i * 3] * 256 * 256 * 256 +
            palettebytes[i * 3 + 1] * 256 * 256 +
            palettebytes[i * 3 + 2] * 256 +
            255
        )
    # set shadow and transprent color values
    palette.set(254, Color("80000000"))
    palette.set(255, Color("00000000"))
    # seek to image data; assuming RLE-encoded
    file.seek(0x80)
    var i = 0
    var b = 0
    var runlen = 0
    var imagebytes = PoolByteArray()
    while i < imagelength and not file.eof_reached():
        b = file.get_8()
        # see if first 2 bits are 1s
        if b & 0xc0 == 0xc0:
            runlen = b & 0x3f
            b = file.get_8()
            for j in range(runlen):
                imagebytes.append(b)
                i += 1
        else:
            imagebytes.append(b)
            i += 1
     
    # create/format image data
    image_from_code.create(width, height, false, image_from_code.FORMAT_RGBA8)
    image_from_code.lock()
    for x in range(image_from_code.get_width()):
        for y in range(image_from_code.get_height()):
            image_from_code.set_pixel(x,y, palette[imagebytes[y * bytesperline + x]])
    image_from_code.unlock()
    file.close()