The reason why SmartMap never releases the memory is by design in Python 2.4 and that memory issue also affects every other line of Python code in Civ4.
Until version 2.5 Python's internal small-object allocator never released memory from it's pool of memory arenas.
This was changed from Python 2.5 onwards and it would be possible to port that change back to Python 2.4.
- Patch #1123430: Python's small-object allocator now returns an arena to
the system ``free()`` when all memory within an arena becomes unused
again. Prior to Python 2.5, arenas (256KB chunks of memory) were never
freed. Some applications will see a drop in virtual memory size now,
especially long-running applications that, from time to time, temporarily
use a large number of small objects. Note that when Python returns an
arena to the platform C's ``free()``, there's no guarantee that the
platform C will in turn return that memory to the operating system. The
effect of the patch is to stop making that impossible, and in tests it
appears to be effective at least on Microsoft C and gcc-based systems.
Thanks to Evan Jones for hard work and patience.
This is a heavily altered derivative of SF patch 1123430, Evan Jones's heroic effort to make obmalloc return unused arenas to the system free(), with some heuristic strategies to make it more l...
Thanks for this background info. I take it that would mean recompiling python24.dll – which gets loaded from the BtS install directory, so it's not something a mod could fix. Without really having looked into the implementation of SmartMap, I would assume that there is some way to delay the bulk of its memory allocations until the map becomes selected. My understanding (from Toffer's C2C Git commit, which made me aware of the problem in the first place, and iirc also a quick test of my own) is that, currently, all the memory gets allocated upon just opening one of the game setup screens.
My understanding (from Toffer's C2C Git commit, which made me aware of the problem in the first place, and iirc also a quick test of my own) is that, currently, all the memory gets allocated upon just opening one of the game setup screens.
SmartMap does some processing to setup some of it's options. It's probably necessary to do that once the setup screen gets opened the first time. At least that's what i recall but maybe there's room for improvement.
There is a function firstTimeInit. That's actually only called once the script is selected. The 100 MB get allocated already when the module is loaded, which happens upon selecting Single Player on the opening menu, i.e. even before the game setup screen. There's an enormous global array "blueMarbleHeightData". Well, it's perhaps 300k integers. That should result in maybe a few MB? But, apparently, it's responsible for the full 100. Turning it into a local variable is no help. The data can be moved into a separate (.csv) file that only gets parsed in generatePlotTypes. However, I'm not sure if a map script can easily locate a file in its own folder (i.e. in PrivateMaps or PublicMaps). __file__ seems to point to the BtS "Assets" directory. The data also has a great deal of redundancy. And I haven't yet figured out precisely what it gets used for. With the default options, it doesn't seem to be used at all.
I also noticed a bug that causes the script's config file to be created directly under My Games: fileName = civFilePath() + "smartmap90.cfg"
Lack of a path separator. Fixing that moves the config file into My Games\Sid Meier's Civilization 4 rather than into the corresponding BtS directory. Apparently the registry keys that the script consults are outdated. Another hint for me that I probably shouldn't try to mess with this monstrous script given my inexpertise in Python.
I've pulled myself together a little and fixed the early memory allocation by replacing those big int arrays (it's actually two 300x600 arrays) with strings. Probably, just putting quotation marks around the array data (and later parsing that of course) would've been enough, but I've also applied a primitive RLE compression – as Python 2.4 apparently has no string compression onboard. The big arrays using the small-object allocator, if that's indeed what happens, actually seems strange. Well, preventing allocation so long as the script isn't used is an improvement in any case. As far as I understand it now, that data encodes plot types and terrain types for Earth maps, a bit like a WB scenario inside a map script, but with flexible map dimensions.
Also managed to improve the location of the file where SmartMap stores the most recently used settings. At least on my system, My Games\Beyond the Sword is now successfully located.
Arboria: yeah, both selections can affect it. high vs low sealevel changes the landcount from ~1700 to ~2300 and rocky climate noticeably increases peak count.
Tectonics: having to change the player count isn't an effective solution when the dll playercount is still 18, and it also makes random selection worthless. I play huge marathon maps with 22+ players at 3k+ land count and the Mediterranean setting in this map is over 5k. I think it also makes no sense in general for earth maps to be smaller than regional maps and firaxis did increase the grid size for their Terra map. But now I've started to think tectonics is just a bad map script. The plate novelty is about all it has going for it and it has poor climate generation and it usually bugs around the poles.
City bar changes: very opposed to. Noticed them immediately, tried playing with them; looked in the commit log for how to reverse them (to the old even smaller settings) and immediately was more comfortable. I've never understood why Bug pushed the wide city bars as default, but at least they were disableable. Now that the resource bubbles are shrinkable, (I set them to 25) I don't understand why I would want to have tiny resources bubbles yet be forced to have giant growing bars littering the screen.
Should I expect this to happen frequently if I just put myself in slot 0 and an AI teammate in slot 1? Or does it hinge on special settings/ circumstances?
Arboria: yeah, both selections can affect it. high vs low sealevel changes the landcount from ~1700 to ~2300 and rocky climate noticeably increases peak count.
So all maps that use HintedWorld or directly use FractalWorld and disable the sea level setting probably have this problem. There's also MultilayeredFractal for generating continents, which leaves the sea level up to the map script. The map script failing to do so is the problem with Big and Small. As for Arboria, I guess it'll have to subtract the sea level change upfront in order to counter the addition in FractalWorld. And will then also have to repreat the threshold-192 thing (that Firaxis themselves didn't understand anymore, judging by the comment) because the water_percent argument will no longer be be at its default value of -1. A little convoluted, but should do the trick, i.e. fix Arboria just through the script file. Seems that a map script can't just set the sea level to Medium (while keeping the menu hidden). (Edit: Well, Python's reflection capabilities do allow this to an extent.) The DLL could allow map scripts to do that, but then the script would no longer be portable to other mods.
Team_Battleground seems to have largely the same problem; also uses HintedWorld. (Typing this as I go through the scripts, also for my own later reference.) Boreal and Rainforest use FractalWorld, but fully override generatePlotTypes. So that looks good, - but the overridden methods do apply the sea level. So that should either be removed or the sea level menu should be unlocked. At least easy to fix within the map script files.
Donut, Highlands, Mirror, Maze and Great Plains also handle the plot type generator on their own, and they ignore the sea level as they should. Oasis is based on MultilayeredFractal and does not add any sea level impact.
As for climate, all the plot type generators in CvMapGeneratorUtil.py take into account the climate for Hills and Peak generation, and the TerrainGenerator and FeatureGenerator in that file also account for the climate. So any map script disabling the climate menu is dubious.
Arboria apparently handles terrain and features on its own, so that part is fine, but the plot generator is a problem. Which aligns with your observation of Peaks being affected by climate. I think the map script can overwrite the climate-adjusted variables of FractalWorld. So this should be fixable too. Ice Age has a similar problem, and also doesn't override climate-based Jungle placement. Boreal, Rainforest, Highlands, Great Plains, Oasis and Fantasy Realm seem to do plots, terrain and features on their own; fine. Donut too, but it does use the climate for Jungle placement, so that needs to be corrected.
Tectonics doesn't use a stock plot generator for any of its options, but it does use the climate-based feature generator from CvMapGeneratorUtil.py. Confusingly, the addFeatures function is defined twice, meaning that the first definition, with a "no-Ice" generator for the Mediterranean setting, has no effect, and the second definition counts, which uses custom latitudes for Mediterranean. I suppose that will also result in no Ice, so probably working as intended and the first definition should just be removed for clarity. Probably a remnant from an old version (specifically v3.6).
Tectonics: having to change the player count isn't an effective solution when the dll playercount is still 18, and it also makes random selection worthless. [...] But now I've started to think tectonics is just a bad map script. The plate novelty is about all it has going for it and it has poor climate generation and it usually bugs around the poles.
Tectonics has quite varied options, maybe some that no other map script does as well/ at all, e.g. Mediterranean. The geo-simulation idea ought to be much better executed by the PerfectWorld family. Well, the precipitation patterns at least; the plate tectonics aspect is not all that complex in PW iirc. (On a side note, I've recently searched for planet generator apps outside the context of Civ and wasn't able to find something similar – and more sophisticated – than PW. The scientific models all seem to rely heavily on real-Earth data and will essentially only reproduce our planet's actual history rather than allowing a random topology.) Getting all Tectonics options to work well seems like a big project, and, therefore, getting random choices to work well also seems unattainable.
City bar changes: very opposed to. Noticed them immediately, tried playing with them; looked in the commit log for how to reverse them (to the old even smaller settings) and immediately was more comfortable. [...] Now that the resource bubbles are shrinkable, (I set them to 25) I don't understand why I would want to have tiny resources bubbles yet be forced to have giant growing bars littering the screen.
Attaching a screenshot; how the city billboards look over here. (Balloon size is 40 btw, via the "Auto" choice.) Maybe the fader settings for the billboards don't interact well with your combination of screen size, field of view and camera distance. Or do they also seem gigantic in my screenshot? As to why – to be able to read city name and production strings, especially when the orange food bar fills up. But, if the fader approach can't be made to work for (most) anyone, I guess the billboard text will have to be left alone – like the rest of the text elements, which could be enlarged too, but not in a way that is customizable from the BUG options screen. Edit2: I've tried higher field-of-view values and a closer camera distance. This does make the billboards too large, especially compared with the size of a tile. So I've reverted most of the enlargement at close to medium camera distance. And I've also added a link to the XML file (Civ4DetailManager.xml) to the BUG option screens (under "Camera" on the "Map" tab). That'll open in an external editor, which isn't nice when in fullscreen mode, but at least changes take effect immediately and players don't need to dig out the file in their file manager.
BUG enables almost everything by default. I don't like that either, but, if I start disabling stuff, BUG users may well think the feature isn't present at all. (Happens a lot with K-Mod, which disables most of the BUG options by default.) I did change the default for a select few, e.g. "Train military units forever: Selecting a military unit in the Choose Production popup sets the city to train that unit repeatedly." Still can't quite believe that this was enabled by default. Hm, forum search indeed finds nearly a dozen users asking why their cities keep producing units. Edit: Actually, the wide city bars weren't optional at all in BUG/BULL. I had to employ a hack in the DLL to make them optional. Forgot about that myself.
That doesn't seem to be the cause, but I did manage to reproduce the issue while trying it that way. Still getting neither a splash screen nor a popup from my savegame when I turn NoTechSplash off, but I do get the splash screen again on later techs, so the setting is apparently not stored in savegames. Anyway, I'll investigate.
By the way, I found this doc comment for isAdvancedMap (in CvMapScriptInterface.py):
"Advanced maps only show up in the map script pulldown on the advanced menu.
Return 0 if you want your map to show up in the simple singleplayer menu"
I suppose Play Now is the simple menu and Custom Game the advanced one. Although this doesn't seem to align with our experience of certain scripts (don't recall which) not showing up under Play Now regardless of what their isAdvancedMap function says. But I guess it might work as intended, and there are just other (mystery) conditions that can block a script from Play Now.
Got them all fixed fairly surgically (the map scripts): Git commit
About ready then to finally release v1.03. I've even played - and thereby tested - it for a few hours.
Fov: I play at 48 and fairly zoomed out besides that, so the growth on the bars was the problem. But I see you reverted some of it and added an editor to the BUG menu, which was an idea I had thought of making and now don't have to try to figure out.
So something else I've been contemplating is how inconvenient it is that the reminder function is only accessible through alt+M, given how often I use it (and forget accidentally pressing the spacebar).
So I added a button to the interface that launches it too:
Moved the spawn of the box to be closer to the button. Wanted the rooster to have some more distance from the Taurus button than the latter has from the log so it's easier to jerk the mouse to it without worrying about misclicking, but I'm not certain on how far it should be. Leaning towards making it right justified in that left hand box so it's not so close to the science slider either. I have no attachment to that as the icon either, just couldn't find anything better. Attached a zip with only the changed files, all labelled internally as DMM.
Near the End Turn button might be even easier to reach and would make sense as the reminder will appear on a future turn. In AdvCiv, I've already placed a "Next Unit" button there (corresponding to the W key press). However, when a city bar is selected, the filter and command buttons take up all the space there. Ultimately, if a widget doesn't deal with the currently selected unit or city, it doesn't really belong on the bottom of the screen (except for the minimap corner).
Next to the Event Log is also logical because the reminder can write to the Turn Log and that log also deals with the passage of turns. The Options button really does not belong there; should be somewhere on the upper right (where there is no room unless the panel size is increased). (Edit: K-Mod puts the BUG options on the Esc menu; that's arguably not as friendly to new players.) I agree that on the right edge of the upper left panel is probably the least awkward spot. Or maybe it could just be a right-click action for the Event Log button? The relationship between Event Log and reminders is of course not really close enough to justify this.
For the texture, the game already has a few clock buttons. Hurry.dds would fit but is too familiar from the Great Engineer. Button_SpeedNormal.dds and Button_SpeedFast.dds, Button_SpeedFast_Pressed.dds could work well (attaching those); might be unused. Adding a texture isn't a problem, but perhaps the existing ones will blend in better.
Would obviously need to be optional and should have a tooltip. Personally, I rarely use reminders. Even with a button, having to set the timer and type a message is clunky. Would be cool if one could hold down a number key and then select a unit or a city or maybe also an Advisor or leader on the scoreboard, and that would be all that it takes for a generic message ("you meant to do something with this unit"). Probably too odd, and the number keys are already in use for recalling unit selections and city production queues (memorized with Ctrl held down).
Maybe perfect is the enemy of good enough here.
On an unrelated note, I might add some optional HUD upscaling like in this post to Taurus. Larger fonts would go well with that. I've worked out a way to increase font sizes via the BUG Options screen, but it's pretty heavy-handed: Loading one of the BUG ini files already when the mod gets loaded (because, when the ini files are normally loaded, it's too late to change the font size) and modifying the EXE at runtime. Still, if I want to keep the install location as flexible as it is, this might be the only way. Scaling on large displays is perhaps the most important thing to improve currently.
Hello all, I've read through this thread and most of it is well beyond my own comprehension once the deeper coding discussion started, just looking to see if I could figure out a question on my own.
Is there any way to just add/merge the Taurus Balloon Sizing XML to the existing BetterBatAI Mod that I am using on my current games?
As I read the XML it looks as if it only works with the full Taurus MOD active and we can't use more than one mod at a time.
The install notes do offer install to Custom Assets if that would just be the quick fix, but I will make a backup first, otherwise I'll just continue with my current game and start new with just Taurus I guess.
Each of the two mods has their own GameCore DLL and, - CustomAssets or otherwise, - only one of those will get loaded. The balloon size changes are implemented through the Taurus DLL; the XML and Python code only puts the widgets on the BUG Options screen. The DLL part could be merged into Better BAT AI at the source-code (C++) level, but that would require basic programming knowledge and the (free) developer tools for recompiling the DLL. There might be an easier way: The balloons can be resized by editing a few bytes of the BtS executable in a hex editor. That will not allow them to be resized while playing, but perhaps just having them at a particular size will be fine. It'll also mean that you'd have to (backup and) modify one of the game's original files (the most important one, arguably) – as Civ4BeyondSword.exe only gets loaded from the install directory, i.e. mods can't supply their own copy. The change will then of course apply regardless of the mod loaded; perhaps that's all the better.
There's still a modified version of the exe attached to this post. That one also replaces the URL of the defunct GameSpy multiplayer server with Zulan's community server. May not be just the right size though. Instructions for a custom size are in this post (2nd spoiler box); will require the Frhed hex editor. Edit: Oh, this won't work with Steam. Or perhaps it will after installing the "beta/ original" version, but, even then, I think there'll be that Steam obfuscation layer. Will then first have to "unpack" the exe using a small tool named Steamless. And then editing the unpacked version should work. (I'll also update the post I've linked to - to clarify this point.)
The simple permanent editing after a few test sizes has made all my future games visibly perfect. And I can adjust again if a new 34" Monitor mysteriously shows up in my Amazon cart
This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
By continuing to use this site, you are consenting to our use of cookies.
github.com
