360 by 180 map possible?

[Attractadore]

Just wondering if I can make a 360 by 180 world map and how much RAM it would take to play it with 34 civs.

 

[isenchine]

That would make 64,800 plots which is theoretically below the maximum limit of 65,536 plots.

Practically, forget it. Expect either graphical glitches or just plain crashes. If it works, wait that the number of cities and units gradually increase and the time to wait between turns will grow exponentially (especially with 34 civs instead of 18). Such a map is unplayable until the end.

What is the practical limit? I don't know - I hate to wait too long, I have a good desktop 64-bit computer and I rarely play maps above 10,000-11,000 plots.

Probably 20,000 is still doable. It would be good to hear from other players on this.

 

[Attractadore]

It can also be 180 x 90 i.e. 16 200 plots, but still. Maybe I should just forget it

 

[Leoreth]

10,000 is already stretching it for me. I'd really recommend not going beyond that.

 

[Yoda Power]

I made an Earth map of about 20,000 plots. It plays fine with all BTS civs (using a mod) untill the industrial age.

 

[isenchine]

So, you mean that after Industrial age it becomes too slow?

I have abandonned the idea of playing Earth maps. I want to finish my games in reasonable time and I can only do that on maps that are far too small for my liking.

Now I only play on "continental maps", with - for example - Europe 5 times bigger than in GEM (18900 plots).

 

[Attractadore]

Actually, 360 by 180 is overkill. For example, the Civ 4 earth map is about 8500 plots with 18 civs (about 400 plots per Civ). 360 by 180 is 7.5 times bigger in terms of area I.e needs 140 different civs to not be a barren wasteland. But 180 by 90 might be kinda ok, since that is 16 200 plots, allowing me to use all the available 34 civs without adding any new ones. Maybe trying that is a better idea.

 

[Yoda Power]

So, you mean that after Industrial age it becomes too slow

Yes. My computer can't handle it.

 

[Nightinggale]

Just wondering if I can make a 360 by 180 world map and how much RAM it would take to play it with 34 civs.

That would make 64,800 plots which is theoretically below the maximum limit of 65,536 plots.

Conclusion: it should work. However When you zoom out you get graphical glitches as only the bottom 128 "rows" of plots are drawn. This makes a valid max height, though playing with the glitch is possible without major issues. At least colonization has this 128 row limitation in the exe and I assume bts has it too (or it would make absolutely no sense to add it to colonization).

Another thing, which is already mentioned is performance. Many AI delays are linear as in 10 cities takes twice as long for the AI to handle than 5 cities. However unit movement doesn't have linear complexity. Pathfinding is done by the exe, which mean it's hidden (unless you add a pathfinder to the dll), which makes it a bit hard to tell precisely how fast it is. However exponentially (or worse!) performance related to number of plots is quite possible and would fit my experience. I don't think I would like to wait for the AI to take turns on such a map even on a supercomputer.

While you would not run out of IDs for plots, you might do so for units or anything else, which can exist in higher numbers than plots.

Being a 32 bit application, the game crashes if it use more than 4 GB memory. C2C has written something about issues when reaching as low as 2 GB. Storing 64k plots isn't the issue, but I don't know how it stores the graphics for it and how efficient it will use the memory to do so.

You can go ahead and try whatever map size you want, but I don't think you will come anywhere near 360x180. Increase gradually and playtest to see the result.

If you really want to play on a huge map, I would recommend that you profile and optimize the DLL as a highly optimized dll will be less affected by the slowdown, or rather a 50% increase isn't as bad if your starting point is better. Sadly it's easier said than done to optimize the code. I have done it with good results, but it is time consuming and I would classify it as one of the advanced modding tasks as it is much more demanding on theoretical programming knowledge than "regular" dll modding.

 

[Leoreth]

Another thing, which is already mentioned is performance. Many AI delays are linear as in 10 cities takes twice as long for the AI to handle than 5 cities. However unit movement doesn't have linear complexity. Pathfinding is done by the exe, which mean it's hidden (unless you add a pathfinder to the dll), which makes it a bit hard to tell precisely how fast it is. However exponentially (or worse!) performance related to number of plots is quite possible and would fit my experience. I don't think I would like to wait for the AI to take turns on such a map even on a supercomputer.

Do you think re-implementing the pathfinding algorithm is worth it? Firaxis probably only implemented a straightforward A* algorithm, but I don't know how much you can actually optimize there. But still I remember reading some people talking about doing something like that.

If you really want to play on a huge map, I would recommend that you profile and optimize the DLL as a highly optimized dll will be less affected by the slowdown, or rather a 50% increase isn't as bad if your starting point is better. Sadly it's easier said than done to optimize the code. I have done it with good results, but it is time consuming and I would classify it as one of the advanced modding tasks as it is much more demanding on theoretical programming knowledge than "regular" dll modding.

Just out of curiosity, are those optimizations available somewhere?

 

[Nightinggale]

Do you think re-implementing the pathfinding algorithm is worth it? Firaxis probably only implemented a straightforward A* algorithm, but I don't know how much you can actually optimize there. But still I remember reading some people talking about doing something like that.

I don't think it's worth it to make your own. Either stick to vanilla or use the pathfinder from K-mod. I don't have experience with the K-mod pathfinder, but from reading the comments in the file, it seems to be designed with performance in mind. It's on the todo list to take a closer look at that one.

Also it seems that K-mod was the optimized mod I was thinking of when I said you should use an optimized mod rather than vanilla.

Just out of curiosity, are those optimizations available somewhere?

Not as standalone modcomps. It tends to be somewhat mod specific how to optimize. I have one general advice though. GC.getDefineINT() is dead slow. If you use one more than a few times each turn, cache it in GC and make a get function to get the cached int. I reduced the AI wait time in RaR by 2.5% by caching one getDefineINT in CVUnit::canUseProfession() because the AI calls it over and over. Naturally you can also just hardcode often used variables, but a cache in GC is more xml friendly. Also if you decide to hardcode, at least use an enum to make sure you only need to change in one location.

Also cache output of often used functions if possible. For instance instead of looping techs to see if you can build a certain improvement, cache if improvements are unlocked by techs and recalculate the cache each time the player gains a tech. I'm not sure if that specific example is a good idea, but that's the fundamental concept in optimization, particularly if you often need to loop all improvements to tell which ones you can build as that would result in nested loops.

Never use std::lists. Use std::vectors instead. Vectors are designed to be hardware cache friendly while lists seems to be designed to maximize cache misses. Lists were a good idea before memory latency became a major performance factor and are now mostly a relic.

Maybe I should write a guide about profiling and optimization. While general optimization isn't easy, it might not be that tricky to write a guide to detect easy to fix specific issues, like how to spot an often called getDefineINT.

 

[Leoreth]

Thanks for the answer. I think it was K-Mod I was thinking of when remembering discussions about a re-implemented pathfinder.

And I agree, K-Mod should probably be considered the reference DLL to base a new BtS mod on, both in terms of performance and AI. Integrating it is definitely on the agenda for my mod at least.

I was mainly asking you because even K-Mod couldn't have covered everything; so it's always good to have all possible avenues for optimization at the back of my mind.

 

[Nightinggale]

was mainly asking you because even K-Mod couldn't have covered everything; so it's always good to have all possible avenues for optimization at the back of my mind.

I don't think it's worth considering optimization in a base on top of K-mod. The main problem is that modders add new code, which in most cases is much slower than K-mod or even vanilla. That's one reason why optimization is very mod specific.

 

[Civciv5]

I made map consisting of more than 25,000 plots in one of my mods with more than 50 civs on it. The first few turns are quite slow, but then it really catches up and runs perfectly fine. But with all the civs tech trading I reach the modern era in the late Middle Ages. Nevertheless, I have a quite powerful computer, so I don't know if most computers can run those kind of maps.

 

[isenchine]

Conclusion: it should work. However When you zoom out you get graphical glitches as only the bottom 128 "rows" of plots are drawn. This makes a valid max height, though playing with the glitch is possible without major issues. At least colonization has this 128 row limitation in the exe and I assume bts has it too (or it would make absolutely no sense to add it to colonization).

Just as an additional note, I experienced these graphical glitches with a map of 230 columns (X) and 98 rows (Y) "only".

I cut it down to 204 columns (X) and left the rows (Y) intact and the glitches were gone. I'm not excluding that this is computer specific.

Now, I still have to play that map until the end!

 

[Nightinggale]

230*98=22540
204*98=19992
None of those values fits any 2^X pattern

Maybe there is an internal limit for allocating 20k plots and it is by chance that I encountered it with 128. Still 128 = 2⁷, which is the number of non-negative values possible with a signed byte. I would be surprised if it has been by chance that this is the number of rows I encountered. It's also possible that there are two limitations, 20k plots and 128 rows. One is a limit in allocated memory while the other is an address limitation.

 

[isenchine]

As I said, it might be computer specific. Civciv5 claims above that he is playing with a map above 25,000 plots.

There is a gigantic world map published of 40,000 plots (250 x 160). I'm not sure if anyone including the author could play a full game on it.

As for my map, I just noticed that the horizontal glitches (several of them) seemed to start at x205 so I cut the map a bit earlier because it was fitting well like that.

I did not try to play the map with glitches as I'm extra-careful and I don't like them!

 

[vincentz]

I have a mapsize of 200x120 (24.000 tiles) which works fine without graphical glitches or any bugs.
It does slow down in late eras, but as long as using a 64bit OS it should be ok.
And I dont have a very good PC (Core2Duo with 4 GB RAM and AMD 7790 1GB GPU).
Anything above the 200x120 caused graphical glitches for me.

or a combined screenie of a Big and Small 34 civ (warning big image)

Spoiler :

 

[isenchine]

Interesting.

To my map, I finally added 4 columns (208 in total) and the glitches came back! It is playable though.

So, is it computer-related or not? Is the limit to not have graphical glitches at 204/205 columns and 128 rows?

(to be continued)

Edit: I finished my game of 6,000 turns on this map of 20,384 tiles (with 24 players). The game went on fine despite the graphical glitches.

The waiting time between turns went up in the second part of the game to 2-3 minutes, never more. But if I take an average of 1 minute per turn, it means that I waited to play my turn in this game at least 100 hours!