WildWeazel
Carthago Creanda Est
Discuss it here!
Summary TBA
Summary TBA
Last edited:
Analysis: New Game Engine
- Thread starter WildWeazel
- Start date
The forum is up! Yay!
I'll leave the first post for the "executive summary", and start by stating that at the time of this forum's creation, a debate was going on about the four (five?) various options, namely:
One of the primary topics of discussion at the time of the forum's creation was scenario compatibility, and how a new game engine would affect that. Vuldacon had made the good point that even small differences could have significant impacts on some (potentially many) scenarios.
With that baseline established, my further thoughts after catching up on the latest (final?) e-mail updates):
--------------------------
There are some valid points that some scenarios may be more tied to the peculiarities of the engine than others, and it may be that some scenarios, particularly as they are now (without modifications), may simply play better in C3C. I suspect it will be somewhat like Play the World vs. Conquests - at the time Conquests came out, the Double Your Pleasure mod was also well-tuned to PTW, and while you probably could just take it and play it in Conquests, with some quirks due to the changes, it was eventually overhauled as Rise and Rule, with changes that adjusted it to play better with Conquests. (Note this is my understanding of events that occurred prior to my joining CFC, but I went with it as the best example off the top of my head)
Thus it may be better to approach it like that; the goal is to be compatible as much as possible, but if you already have an existing mod that's working perfectly well with Conquests, it might make sense to play it in Conquests. The new engine could open up new possibilities for both future iterations of existing mods, and for altogether new mods.
Vuldacon also raised a valid point about whether we are biting off more than we can chew. Frankly, I'm not at all sure we aren't, but am approaching the endeavour somewhat as "I'd like to find out if I can, and expect to learn new things about game development from the experience", not "this will be an abject failure if we can't." I'm also not at all sure we can achieve our aims via EXE patching, but that's a topic for another thread.
I'll leave the first post for the "executive summary", and start by stating that at the time of this forum's creation, a debate was going on about the four (five?) various options, namely:
- New game engine (this topic)
- BIQ/SAV utilities
- EXE Patching
- Basing off an open source project, e.g. FreeCiv
- Basing off ToTPP
One of the primary topics of discussion at the time of the forum's creation was scenario compatibility, and how a new game engine would affect that. Vuldacon had made the good point that even small differences could have significant impacts on some (potentially many) scenarios.
With that baseline established, my further thoughts after catching up on the latest (final?) e-mail updates):
--------------------------
There are some valid points that some scenarios may be more tied to the peculiarities of the engine than others, and it may be that some scenarios, particularly as they are now (without modifications), may simply play better in C3C. I suspect it will be somewhat like Play the World vs. Conquests - at the time Conquests came out, the Double Your Pleasure mod was also well-tuned to PTW, and while you probably could just take it and play it in Conquests, with some quirks due to the changes, it was eventually overhauled as Rise and Rule, with changes that adjusted it to play better with Conquests. (Note this is my understanding of events that occurred prior to my joining CFC, but I went with it as the best example off the top of my head)
Thus it may be better to approach it like that; the goal is to be compatible as much as possible, but if you already have an existing mod that's working perfectly well with Conquests, it might make sense to play it in Conquests. The new engine could open up new possibilities for both future iterations of existing mods, and for altogether new mods.
Vuldacon also raised a valid point about whether we are biting off more than we can chew. Frankly, I'm not at all sure we aren't, but am approaching the endeavour somewhat as "I'd like to find out if I can, and expect to learn new things about game development from the experience", not "this will be an abject failure if we can't." I'm also not at all sure we can achieve our aims via EXE patching, but that's a topic for another thread.
WildWeazel
Carthago Creanda Est
So to be clear, as far as I'm concerned this is the default choice. I convened us here to make the case that either another option is more worthwhile, or that there's not enough support (technical or otherwise) to justify trying it. So far I'm not convinced of either. 
Also since this is an amateur passion project, ultimately the only new code that's going to get written is what the programmers care to work on. Not to dismiss anyone else's wishes, just that personal motivation is going to be a very large factor. So far it looks like each of us programmers are favoring this option.
I concur with everything Quintillus said here, especially the last paragraph. This is definitely one of those "go big or go home", "you'll never know if you don't try" kind of things.
I'm determined to do it in a way that we get the most tangible benefit out of it. Which reminds me, I need to get those notes finished! I have a roadmap for how we might tackle this.
Also since this is an amateur passion project, ultimately the only new code that's going to get written is what the programmers care to work on. Not to dismiss anyone else's wishes, just that personal motivation is going to be a very large factor. So far it looks like each of us programmers are favoring this option.
I concur with everything Quintillus said here, especially the last paragraph. This is definitely one of those "go big or go home", "you'll never know if you don't try" kind of things.
I'm determined to do it in a way that we get the most tangible benefit out of it. Which reminds me, I need to get those notes finished! I have a roadmap for how we might tackle this.
WildWeazel
Carthago Creanda Est
As promised, here is my research-notes-turned-proposal-turned-design-document now that I've decided I've had enough of it. You can add comments, or just discuss here. https://docs.google.com/document/d/1Y_Vwnewfi78nOot8znd9Jt9X75DlS_KtuTUCINi6fd8/edit?usp=sharing
The major sections of interest to us for this discussion:
Programmers will probably want to read all of sections 5 and 6 at least. Others can focus on 5.1, 5.3, 6.2, and 6.3
The major sections of interest to us for this discussion:
- 5.1 is a general description of what I'm shooting for
- 5.3 is a proposed breakdown into iterative releases by feature
- 5.4 describes team roles
- 6.1-6.4 gets technical, with some basic requirements
- 6.5 covers preliminary thoughts on mod design
Programmers will probably want to read all of sections 5 and 6 at least. Others can focus on 5.1, 5.3, 6.2, and 6.3
Last edited:
@WildWeazel - Of course, I had no idea that your wife has been ill (as you mentioned elsewhere) and I nevertheless exhaled when you mentioned that she's well, as any mention of anything but a short illness. These days, as such an immediate cause for alarm.
Now back to playtime - Unsurprisingly, you've drawn up an excellent technical document. Nonetheless, I am concerned that it you’ve put the proverbial cart before the horse. You’ve described a fine way in which to build (by your own definition), a "Superset of Civ3,” and I understand , perfectly well, the many reasons why this is an extremely attractive goal. That being said, building a superstructure without first describing and fully definig what that superstructure is to contain is, at best, problematic.
I apologize if I am sounding a bit harsh here, but what is lacking is what we actually want the game to do. Given the ad hoc nature of our attention to this project, with 20:20 hindsight it would, perhaps, have been been best to agree to what we want in more than a a, "Wish List" form.
Simply beginning with Units, you state that moving them, without further description or definition, would be part of “0.1". Yet "movement" itself actually needs to be defined: the simple interactions with Tiles, as well as the simple fact that we have three very different types of Units and play: Land, See, Air. "Orders" and "combat" aren't addressed until two stages later (0.3) and that is about it. And - frankly, given the precise goal of plug compatibility, your further statement in 0.3 of, "Unit media,” is - well, frankly, troubling.
Nowhere is there “room” for to discuss how we would actually want the AI to (for example) build and utilize units, with the simple and perfect example being the many problems with artillery. Personally, I believe that providing the AI with the rudiments of strategy beyond the simple massing of units into mega stacks, thundering across the board without rhyme or reason, would be nice along with which units within any stack say, defend or attack, and in some sensible order.
Even Tiles themselves seem problematically laid out, at least for me - Once again going back to 0.1, let’s consider, "Movable units," as this plainly must reflect interactions of units with the Tiles themselves. We should, if nothing else, because we have two large sets of Tiles (Land; C) and three different types of Units which move across, and fight within them.
You mentioned that there have been a number of attempts at building clones of Civ3, and questioned why they have not succeeded. I think that one very simple reason is that every aspect of how the game operates, now must be (to some greater or lesser degree) reverse engineered and, more importantly, every "rule" of how we (or anyone else for that matter) want our new game to work. This means that every aspect of it must be fully defined. How do we want artillery to work? How do we want to improve upon any aspect of the AI's behavior, specifically including matters like improving upon AI unit builds, and cooperation. How do we decide what workers should do? How do we want aircraft carriers to work?
For the record, this was one reason why have been advocating using the stable platform like Freeciv, as all of these (and recall that I did mention that that is already 400,000 lines of code) - let's call them, "modules," are already in place, and, furthermore, they are laid out any format which is readily translatable into the sort of (if you will, backspace) "building blocks," which I believe we wish to pursue in the first place…
… Actually, now that I think of it, "building blocks" is not only an excellent metaphor, but also a realistic description of how we can approach building “C7" on that platform. It's code structure, and descriptions thereof, are (as I have mentioned before) pretty much absolutely the best laid out, and documented, that I have ever seen.
Let’s also linger over that “400,000" lines of code bit. How long would an effort like ours take, with or without (sorry, WildWeazel my friend, but I simply can’t resist
) waiting for Godot?
Irrespective, I think that all of the technical explorations which have been going on are both excellent and necessary for our understanding of what we have, and what can be done. I simply think that we should truly lay out what it is we want to build to do before we go about deciding how to build it.
I also plan to attempt to contact Soren this next week, although I've belatedly realized that, whether or not he possesses the Holy Grail of source code, the IP might not be his to disseminate ...
Now back to playtime - Unsurprisingly, you've drawn up an excellent technical document. Nonetheless, I am concerned that it you’ve put the proverbial cart before the horse. You’ve described a fine way in which to build (by your own definition), a "Superset of Civ3,” and I understand , perfectly well, the many reasons why this is an extremely attractive goal. That being said, building a superstructure without first describing and fully definig what that superstructure is to contain is, at best, problematic.
I apologize if I am sounding a bit harsh here, but what is lacking is what we actually want the game to do. Given the ad hoc nature of our attention to this project, with 20:20 hindsight it would, perhaps, have been been best to agree to what we want in more than a a, "Wish List" form.
Simply beginning with Units, you state that moving them, without further description or definition, would be part of “0.1". Yet "movement" itself actually needs to be defined: the simple interactions with Tiles, as well as the simple fact that we have three very different types of Units and play: Land, See, Air. "Orders" and "combat" aren't addressed until two stages later (0.3) and that is about it. And - frankly, given the precise goal of plug compatibility, your further statement in 0.3 of, "Unit media,” is - well, frankly, troubling.
Nowhere is there “room” for to discuss how we would actually want the AI to (for example) build and utilize units, with the simple and perfect example being the many problems with artillery. Personally, I believe that providing the AI with the rudiments of strategy beyond the simple massing of units into mega stacks, thundering across the board without rhyme or reason, would be nice along with which units within any stack say, defend or attack, and in some sensible order.
Even Tiles themselves seem problematically laid out, at least for me - Once again going back to 0.1, let’s consider, "Movable units," as this plainly must reflect interactions of units with the Tiles themselves. We should, if nothing else, because we have two large sets of Tiles (Land; C) and three different types of Units which move across, and fight within them.
You mentioned that there have been a number of attempts at building clones of Civ3, and questioned why they have not succeeded. I think that one very simple reason is that every aspect of how the game operates, now must be (to some greater or lesser degree) reverse engineered and, more importantly, every "rule" of how we (or anyone else for that matter) want our new game to work. This means that every aspect of it must be fully defined. How do we want artillery to work? How do we want to improve upon any aspect of the AI's behavior, specifically including matters like improving upon AI unit builds, and cooperation. How do we decide what workers should do? How do we want aircraft carriers to work?
For the record, this was one reason why have been advocating using the stable platform like Freeciv, as all of these (and recall that I did mention that that is already 400,000 lines of code) - let's call them, "modules," are already in place, and, furthermore, they are laid out any format which is readily translatable into the sort of (if you will, backspace) "building blocks," which I believe we wish to pursue in the first place…
… Actually, now that I think of it, "building blocks" is not only an excellent metaphor, but also a realistic description of how we can approach building “C7" on that platform. It's code structure, and descriptions thereof, are (as I have mentioned before) pretty much absolutely the best laid out, and documented, that I have ever seen.
Let’s also linger over that “400,000" lines of code bit. How long would an effort like ours take, with or without (sorry, WildWeazel my friend, but I simply can’t resist
) waiting for Godot?Irrespective, I think that all of the technical explorations which have been going on are both excellent and necessary for our understanding of what we have, and what can be done. I simply think that we should truly lay out what it is we want to build to do before we go about deciding how to build it.
I also plan to attempt to contact Soren this next week, although I've belatedly realized that, whether or not he possesses the Holy Grail of source code, the IP might not be his to disseminate ...
Puppeteer
Emperor
Interesting but not surprising there are different approaches to the project.
My approach is that the two anchors are the backwards-compatibility (likely via one-way conversion) with C3C media, mods, and data, and then the wish/feature/bugfix list.
Take for example overlay terrain. At the moment I'm doubting it makes sense to have a hard-coded concept of overlay terrain. Some of the wish list is multiple maps. Using those two anchor points and our choice of C#, I start to mentally design a prototype tile object format that would allow arbitrary layers of features, and the mod rules system would allow modeling the overlay terrain if desired, to add a new spatial layer (e.g. space) and perhaps even further subdivide legacy tiles into sea layer, ground layer, low altitude, and high altitude terrain. This might allow tall mountains to block some air traffic or allow a zone of control that only applies to flying units, for example. It might also allow regional deployment of SDI if there is a space layer.
Or maybe once we mock up some prototype code or have a discussion we realize that's a terrible idea and see a different path emerge.
So from my approach it's hard to imagine AI inputs because I'm not yet sure what a tile or unit is in the new game. I'm still asking what's the difference between a colony, airfield, goody hut, and city from a coding point of view. Can they all be the exact same object schema with just different rules per type ID? Each has map coordinates, a beginning of life, and end of life to account for. If we allow terraforming, perhaps we can ask if tiles and these other things are a common object schema or all inherit from a base object.
Until those questions are worked out some I'm not sure what movement looks like.
But I'm not necessarily capital-R Right for all of us, and maybe not even for myself. I have some more experimental coding to do before reaching firmer conclusions on that.
Reading my post back, I mention multiple maps then launch into new layers of existing maps. But in my head one tile object should work for all maps, and perhaps each map may have different layer types; I'm not sure every map needs to have sea, land, and air. Maybe other maps are just subterranean or space and don't need sea, land, or air layers. Or overlay terrain. But again that will take some exploratory prototype coding to see if a rules/mod API can morph a tile object into that many uses.
Incidentally, I'm not sure if I specifically mentioned it, but I imagine that everything we do to make the game will be repeatable/changeable via the mod system. In the ideal case, the new game is just a skeleton, some import code, and a mod system with a mod to implement the C3C game rules & mechanics.
But it is admittedly hand-wavy at this point with much need for some coding experiments.
My approach is that the two anchors are the backwards-compatibility (likely via one-way conversion) with C3C media, mods, and data, and then the wish/feature/bugfix list.
Take for example overlay terrain. At the moment I'm doubting it makes sense to have a hard-coded concept of overlay terrain. Some of the wish list is multiple maps. Using those two anchor points and our choice of C#, I start to mentally design a prototype tile object format that would allow arbitrary layers of features, and the mod rules system would allow modeling the overlay terrain if desired, to add a new spatial layer (e.g. space) and perhaps even further subdivide legacy tiles into sea layer, ground layer, low altitude, and high altitude terrain. This might allow tall mountains to block some air traffic or allow a zone of control that only applies to flying units, for example. It might also allow regional deployment of SDI if there is a space layer.
Or maybe once we mock up some prototype code or have a discussion we realize that's a terrible idea and see a different path emerge.
So from my approach it's hard to imagine AI inputs because I'm not yet sure what a tile or unit is in the new game. I'm still asking what's the difference between a colony, airfield, goody hut, and city from a coding point of view. Can they all be the exact same object schema with just different rules per type ID? Each has map coordinates, a beginning of life, and end of life to account for. If we allow terraforming, perhaps we can ask if tiles and these other things are a common object schema or all inherit from a base object.
Until those questions are worked out some I'm not sure what movement looks like.
But I'm not necessarily capital-R Right for all of us, and maybe not even for myself. I have some more experimental coding to do before reaching firmer conclusions on that.
Reading my post back, I mention multiple maps then launch into new layers of existing maps. But in my head one tile object should work for all maps, and perhaps each map may have different layer types; I'm not sure every map needs to have sea, land, and air. Maybe other maps are just subterranean or space and don't need sea, land, or air layers. Or overlay terrain. But again that will take some exploratory prototype coding to see if a rules/mod API can morph a tile object into that many uses.
Incidentally, I'm not sure if I specifically mentioned it, but I imagine that everything we do to make the game will be repeatable/changeable via the mod system. In the ideal case, the new game is just a skeleton, some import code, and a mod system with a mod to implement the C3C game rules & mechanics.
But it is admittedly hand-wavy at this point with much need for some coding experiments.
Puppeteer
Emperor
I'll add that us having access to code that is not compatibly licensed is a project killer. We CAN NOT "peek" at proprietary code or incompatibly-licensed code without tainting our code base. We have to reverse engineer stuff, figure things out ourselves, or borrow from compatibly-licensed code.
.... To begin at the end: I'm 100% on the same page. I'm a heavy duty believer in IP, including (although almost certainly not applicable here) the exceptions o the "Fair Use" doctrine.
Taking those two thoughts, together, is a perfect example of my concerns regarding the game itself. For example the approach I had in mind for implementing overlays was much like your own (although I do confess to finding your way far more fascinating) was loosely analogous, beginning with the City, itself, as an overlay constructed of multiple attributes (Land Tile, Sea Tile, Road/RR, etc.)
Nonetheless, I think we need to begin with the equivalent of someone reading a Rules Book when designing the game itself. Rather than (for the moment) delving into overlays, let's stick to another one of your ideas, which I think is not only a great one, but one which has never once crossed my mind: that biplanes should not be able to fly across the Alps.
Irrespective of how the intellectual/coding aspects are conceptualized, from both a mod developer' and a player's POV there are nonetheless two "objects" involved (I use quotation marks to simply note that this level of "abstraction" would never cross the mind of anyone playing the game) - a Unit and a Tile, and each of these should be able to be sufficiently described, for design purposes, exactly as they would be read in a rules book. (I use the word "sufficiently" because we are both thinking/speaking in OO terms, fully understanding that, e.g., a "Class," called, "Tile," can be defined with the notion that different, inheritable attributes - "can link to one map level up/down" - can be added in further implementations, without disrupting any already existing "Category - Class - Type - Object" hierarchy which is already in place.)
- So: back to the "Rules Book," and those Biplanes and Mountains.
All a modder/player should need to know is that there are "Tiles" and "Units," each with its own set of (possibly) wildly varying attributes. No matter how conceptualized and/or coded, there is still a thing called a Mountain which ( for example's sake) cannot be crossed by Wheeled units, or "Low Flying" aircraft; etc. Likewise, there is a thing called a unit which belongs to one of three types (air; land; sea) with each type having its own inheritable attributes etc. And anything and everything involving how a modder/player interacts with the game must be specifiable, and specified, in plain plain language, and with a concision and precision which meets all of the (formally) logical requirements of, "completeness" (i.e., that the Rules contained within the Rule book are both internally coherent, as well as externally coherent with the game itself.)
.. You know, I haven't had this much fun thinking in quite some time!
WildWeazel
Carthago Creanda Est
Puppeteer makes an important point about licensing; unless the original code is actually open-sourced it could be intellectual poison for anyone wanting to work on this. (and if it were open-sourced, this would of course become largely superfluous)
Two different approaches indeed. If we organize the project well, there's plenty of room to discuss the details in parallel with development. The way I see it, the baseline functional requirements are "do what Civ3 does [excepting known bugs]". If we're working in short iterations, that's a good enough long-range target to inform the overall design. As we tackle individual systems and features, we can work out the how (detailed design), and where to push beyond Civ3 (feature/change requests). Taking my arbitrary roadmap and terrain types as an example, we know up front we have to support everything in the current terrain system, and sketch out some basic structures accordingly. We then have up until 0.5 to work out what additional parameters are needed, rules for tile adjacency, how to create new terrains, and so on. Whatever we do prior to that would be largely unaffected by decisions about terrain. There would presumably also be some smaller internal iterations within 0.5 where we'd refine the terrain system.
OTOH it's never too early to start those conversations, as long as we avoid the bike sheds. Oz, you mentioned using the manual as a model. That seems like a fine place to start.
That's pretty much my pie in the sky vision of the mod system. It's basically how Minetest works. I think if we design the core of the game in such a way that it doesn't really know game content from mod, or player from AI, it will be a lot easier to bolt on new features. To put it another way, there would be very little need for any modder to change/rebuild the game itself. Of course mods could be as simple as static data, or as complex as overriding much of the game's functionality. <exaggerated hand waving>
Two different approaches indeed. If we organize the project well, there's plenty of room to discuss the details in parallel with development. The way I see it, the baseline functional requirements are "do what Civ3 does [excepting known bugs]". If we're working in short iterations, that's a good enough long-range target to inform the overall design. As we tackle individual systems and features, we can work out the how (detailed design), and where to push beyond Civ3 (feature/change requests). Taking my arbitrary roadmap and terrain types as an example, we know up front we have to support everything in the current terrain system, and sketch out some basic structures accordingly. We then have up until 0.5 to work out what additional parameters are needed, rules for tile adjacency, how to create new terrains, and so on. Whatever we do prior to that would be largely unaffected by decisions about terrain. There would presumably also be some smaller internal iterations within 0.5 where we'd refine the terrain system.
OTOH it's never too early to start those conversations, as long as we avoid the bike sheds. Oz, you mentioned using the manual as a model. That seems like a fine place to start.
That's pretty much my pie in the sky vision of the mod system. It's basically how Minetest works. I think if we design the core of the game in such a way that it doesn't really know game content from mod, or player from AI, it will be a lot easier to bolt on new features. To put it another way, there would be very little need for any modder to change/rebuild the game itself. Of course mods could be as simple as static data, or as complex as overriding much of the game's functionality. <exaggerated hand waving>
Similar threads
