[TOT] [TOTPP] Text Formatting

[Prof. Garfield]

EDIT: Changed thread title from Help Wanted

I need some help from people with different operating systems.

I'm trying to figure out the "length" of each character when displayed in a text box. This would allow some formatting of text when displayed, so that information could be displayed in vertical columns. (Unfortunately, Civ II doesn't recognize tab characters.) However, I need to check if this is consistent across systems.

I would appreciate it if others could run the attached script (use the load script button) and check if the vertical lines in the two rows are aligned in each text box displayed. Choose the appropriate option on the screen. If the lines are not aligned, a message will be printed to the console. Please cut and paste those messages. If most or all of the characters don't seem to align, don't bother going through all 255 characters. On that subject, I recommend using the keyboard to select options rather than the mouse. It will go faster.

Please report your operating system along with any information printed to the console.

I think it will be clear what the procedure is when the script is loaded, but if there are any doubts, I can post screenshots.

Thanks.

 

[JPetroski]

If someone is going to try this bear in mind that you do need to get through the first 30 before you might start seeing correct. The first 30 don't appear to be actual symbols so much as square boxes as you can see in the code. I know Garfield said to not bother if "most or all" of these are incorrect but wait until you get to some of the letters before you bail.

Also, Garfield, for the most part (I think the whole part) when I finally got into "true" letters and symbols, if they were "wrong" they were only wrong by about a space or so.

My operating system is Windows 10.

1, , Incorrect Size
2, , Incorrect Size
3, , Incorrect Size
4, , Incorrect Size
5, , Incorrect Size
6, , Incorrect Size
7, , Incorrect Size
8, , Incorrect Size
9,     , Incorrect Size
11, , Incorrect Size
12, , Incorrect Size
14, , Incorrect Size
15, , Incorrect Size
16, , Incorrect Size
17, , Incorrect Size
18, , Incorrect Size
19, , Incorrect Size
20, , Incorrect Size
21, , Incorrect Size
22, , Incorrect Size
23, , Incorrect Size
24, , Incorrect Size
25, , Incorrect Size
26, , Incorrect Size
27, , Incorrect Size
28, , Incorrect Size
29, , Incorrect Size
33, !, Incorrect Size
49, 1, Incorrect Size
65, A, Incorrect Size
73, I, Incorrect Size
84, T, Incorrect Size
86, V, Incorrect Size
88, X, Incorrect Size
94, ^, Incorrect Size
109, m, Incorrect Size
120, x, Incorrect Size
122, z, Incorrect Size
124, |, Incorrect Size
127, , Incorrect Size
129, , Incorrect Size
132, „, Incorrect Size
137, ‰, Incorrect Size
141, , Incorrect Size
143, , Incorrect Size
144, , Incorrect Size
147, “, Incorrect Size
148, ”, Incorrect Size
152, ˜, Incorrect Size
157, , Incorrect Size
158, ž, Incorrect Size
166, ¦, Incorrect Size
170, ª, Incorrect Size
183, ·, Incorrect Size
248, ø, Incorrect Size

 

[Prof. Garfield]

Thanks @JPetroski

This result isn't too surprising, but it is still disappointing nonetheless. It would be nice to be able to format output into rows and columns sometimes (such as in 'tab' help and combat reporting), and being out a space over 5 characters can add up. I wrote a rudimentary calibration tool to get the sizes in the first place. Perhaps I can refine it, and each player can calibrate to their own system fonts if they want better formatting.

 

[Knighttime]

Hi @Prof. Garfield , you and I have so many of the same ideas regarding Lua projects to tackle! I actually went down this same road already, and have built a solution that works "as well as possible" on my system. I'm working on a much larger project (scenario), though, and rather than release this feature as a standalone module, my intention had been to release it as part of that project. (No estimated date available for that yet, unfortunately.)

Obviously the step I didn't take was to solicit input from other people with different setups. When I was considering this, my thought was that the operating system by itself may not be a distinguishing factor, but I was concerned about (a) screen resolution, and (b) installed fonts -- unless Civ2 has its own embedded fonts and the ones installed on each user's machine aren't relevant. I'm not sure how that works.

For what it's worth, my results when running your test are exactly the same as @JPetroski 's. My operating system is Windows 7, though, and I'm using 1920x1080 resolution on a 24" monitor.

My solution includes a table I called "CHAR_PIXEL_WIDTH" that is almost exactly like your "numCharLength" table. I actually skipped ASCII 0-31, though after seeing how many of them do print in your test, perhaps I should go back and add them. Here are the other differences in our tables (my values on the left):

[33] = 6, [33] = 5,
[49] = 9, [49] = 10,
[65] = 11, [65] = 12,
[73] = 4, [73] = 5,
[84] = 12, [84] = 11,
[86] = 11, [86] = 12,
[88] = 11, [88] = 12,
[94] = 7, [94] = 8,
[95] = 0, [95] = 5,
[109] = 14, [109] = 15,
[120] = 8, [120] = 9,
[122] = 8, [122] = 9,
[124] = 6, [124] = 5,
[127] = 0, [127] = 14,
[129] = 0, [129] = 14,
[132] = 7, [132] = 6,
[137] = 17, [137] = 18,
[141] = 0, [141] = 14,
[143] = 0, [143] = 14,
[144] = 0, [144] = 14,
[147] = 7, [147] = 6,
[148] = 7, [148] = 6,
[152] = 5, [152] = 6,
[157] = 0, [157] = 14,
[158] = 8, [158] = 9,
[166] = 6, [166] = 5,
[170] = 6, [170] = 7,
[183] = 6, [183] = 5,
[224] = 6, [224] = 10,
[225] = 6, [225] = 10,
[226] = 6, [226] = 10,
[227] = 6, [227] = 10,
[228] = 6, [228] = 10,
[229] = 6, [229] = 10,
[248] = 10, [248] = 11,

This matches very very closely with the list of differences ("Incorrect Size") that John and I found when running your test.

The bigger problem here, which I was not able to truly solve, is one that your test neatly sidesteps. Namely, a space (32) is 5 pixels wide, so any padding you do will always increase a string length by multiples of 5 pixels. In your test, you always print 5 of every character, thus ensuring that your total test length will be a multiple of 5 and therefore it's possible to achieve perfect alignment with an appropriate number of spaces. In real life, though, with a mix of characters, the best that I could do was column alignments "to the nearest 5 pixels".

As it turns out, this actually produces very, very readable results -- as you can see:

But the perfectionist side of me is still bothered by the fact that the columns are jagged. Unless you have some ideas for how to address this, though, it may be the best that we can do.

I would be willing to share my work on this with the community, but it may take me a little bit of time to extract it from the context of my larger project. Obviously you can pursue this independently if you'd like... but I wanted you to be aware that I do already have a fully-implemented solution running locally.

 

[JPetroski]

I am completely intrigued by the screenshot and wonder what the scenario is!

 

[Knighttime]

@JPetroski Did I mention "the perfectionist side of me"? My scenario building is slooooow because I seem to come up with a never-ending list of things that could be done just a little bit better. I don't want to start promoting it too soon... once I get to the point that it's ready for beta testing by others, then maybe I'll start a forum thread to begin sharing details. Your collaborative work with @Prof. Garfield on OTR is impressive, by the way, and I do appreciate the way you've shared that publicly. This project is entirely my own work, though, and I've made the decision to keep it private until I'm much closer to a final release.

 

[Prof. Garfield]

Obviously the step I didn't take was to solicit input from other people with different setups. When I was considering this, my thought was that the operating system by itself may not be a distinguishing factor, but I was concerned about (a) screen resolution, and (b) installed fonts -- unless Civ2 has its own embedded fonts and the ones installed on each user's machine aren't relevant. I'm not sure how that works.

For what it's worth, my results when running your test are exactly the same as @JPetroski 's. My operating system is Windows 7, though, and I'm using 1920x1080 resolution on a 24" monitor.

OK, if Windows 10 and Windows 7 are producing the same results, I'm probably the odd one out since I'm using Linux with WINE. I should probably test in my XP virtual machine to be sure, however. I thought at some point I might have downloaded the "correct" fonts for Civ II, but I guess not (I'm pretty certain that Civ II uses fonts from elsewhere in the system). If everything is consistent on all versions of Windows, then there is a manual setup for Linux users to do, but that is a minor issue compared to the other things that must be done to get TOT working on Linux.

Hi @Prof. Garfield , you and I have so many of the same ideas regarding Lua projects to tackle! I actually went down this same road already, and have built a solution that works "as well as possible" on my system. I'm working on a much larger project (scenario), though, and rather than release this feature as a standalone module, my intention had been to release it as part of that project. (No estimated date available for that yet, unfortunately.)

If you do work like this, I would encourage you to release it as an independent module, so that others can use it with minimal effort. Of course, I can't be too annoyed, after all, I didn't do it for Over the Reich. I do plan to release the flag and counter system with my next Lua lesson, with, I think, a fairly elegant way to "hook" the state table to independent modules. (In my legacy event engine, I use a global variable, though I've found a slicker way to do it since.)

The bigger problem here, which I was not able to truly solve, is one that your test neatly sidesteps. Namely, a space (32) is 5 pixels wide, so any padding you do will always increase a string length by multiples of 5 pixels. In your test, you always print 5 of every character, thus ensuring that your total test length will be a multiple of 5 and therefore it's possible to achieve perfect alignment with an appropriate number of spaces. In real life, though, with a mix of characters, the best that I could do was column alignments "to the nearest 5 pixels".

For a while, I had the space have a "width" of 60, but divided everything out when it became clear how wide the space "should" be.

My proposed method of achieving alignment is to insert relatively unobtrusive characters (like .,'`") to make the length of the printed characters a multiple of 5, after which padding can be achieved with spaces. The solution you have is pretty decent as it is.

But the perfectionist side of me is still bothered by the fact that the columns are jagged. Unless you have some ideas for how to address this, though, it may be the best that we can do.

I would be willing to share my work on this with the community, but it will take me a little bit of time to extract it from the context of my larger project. Obviously you can pursue this independently if you'd like... but I wanted you to be aware that I do already have a fully-implemented solution running locally.

If you can put this into relatively self contained form, or ideally a module, I would appreciate it very much. No rush. My direct want/need for this is to remake the combat reporting system for OTR, and maybe format 'tab' help a little better. I was planning on making a "text" library anyway, to do things like archive messages and display them to the correct player after production (also an OTR functionality), so this was something on my mind that I decided to work on for a little bit.

 

[Prof. Garfield]

I changed the title of this thread from Help Wanted, since I seem to have gotten enough help.

 

[JPetroski]

Yes, back to the thread!

(but just so you know, when/if you need playtesters, I owe you one!)

 

[Knighttime]

If you do work like this, I would encourage you to release it as an independent module, so that others can use it with minimal effort. Of course, I can't be too annoyed, after all, I didn't do it for Over the Reich.

@Prof. Garfield I like the modular approach, as a general rule, and I think we're mostly on the same page there -- although we might have some differences of opinion about approaches to implementation. Your encouragement is noted and certainly appreciated.

Like you with OTR, I think, I'm identifying and then developing the modules that I want/need as a byproduct of writing the complete set of Lua events for my own scenario project. So in many ways, the actual details of how I want a given module to work, what options or parameters are useful, etc., are still a work in progress as I change my scenario's goals, test for gameplay impact, come up with new ideas, etc. I'm often making changes to a module that don't preserve backwards compatibility, and in many cases I still need that level of developer freedom. Basically I don't want to release a shortsighted module that doesn't even meet my own needs! -- or that is founded upon assumptions that I later determine to be invalid. Related to this, playtesting within my project is the way that I'm handling most of the QA and tracking down bugs to fix.

There's also a point at which I see some Lua modules -- especially bigger, more impactful ones -- as something like the roster of units (icons, names, stats) for a scenario. If a scenario designer wants to release their units.bmp early (prior to the release of the whole scenario), along with some amount of detail from rules.txt, great! Perhaps that's a way to advertise, build up interest, obtain feedback, etc. On the other hand, I would 100% and without reservation support a scenario designer who prefers not to offer any "sneak previews" but simply wants to release their full scenario in all its glory once it's complete.

Certainly there are also some artists here who enjoy creating units independently, for the community at large, without any real intention of building a scenario on their own which will include them. That's fantastic and a huge benefit to people (like me!) who don't have those artistic gifts. I'm also willing to release some Lua modules within such a framework (e.g., supplyLines). But I don't intend to follow this approach in every case.

That being said: a module for displaying columnar text is probably a "nice to have" and not a core gameplay feature, so I'm not necessarily opposed to releasing it independently. I also don't have major concerns about future changes that would disrupt backwards compatibility. I'm sharing my thoughts about the larger issues involved because I've actually wrestled quite a bit with the question of "when and how should I release my work". Hopefully my conclusions/preferences there don't seem arbitrary -- or unreasonably selfish -- to others.

My proposed method of achieving alignment is to insert relatively unobtrusive characters (like .,'`") to make the length of the printed characters a multiple of 5, after which padding can be achieved with spaces. The solution you have is pretty decent as it is.

Hmm. Well, that seems like a trade-off, right? Then the columns would line up (perfectionist: "Yay!") but there'd be multiple different extraneous characters showing up in some lines but not others (perfectionist: "Arrgh...").

Just so you know, my solution actually handles for pixel "overflow" and "underflow": it keeps track of the individual pixel count by which the entire row is "ahead" or "behind" at every given column break, and adds a 5-pixel space whenever 3 or more pixels are needed (i.e., 2 pixels rounds down to "no space" and 3 rounds up to "space") -- but if you need 3 pixels and add a space worth 5 pixels, it remembers that you are 2 ahead so the next time you need 3, you really only need 1 and thus a space is not added. This functionality was added later because I found it necessary to get output I considered "pretty decent" -- without this feature, rows with a lot of columns sometimes got further and further out of alignment as you moved to the right.

If you can put this into relatively self contained form, or ideally a module, I would appreciate it very much. No rush. My direct want/need for this is to remake the combat reporting system for OTR, and maybe format 'tab' help a little better.

I'll see what I can do.

I was planning on making a "text" library anyway, to do things like archive messages and display them to the correct player after production (also an OTR functionality), so this was something on my mind that I decided to work on for a little bit.

Archiving messages is something I haven't tackled at all yet. In case you were wondering...

However, I think it might be good if I responded separately to your other thread about The General Library. That's another area where we're tackling quite a few of the same problems independently, but not in quite the same way, and I'm not sure how (or whether) to deal with that.

 

[JPetroski]

Well, @Knighttime , for what it's worth, I created my OTR development thread in February of 2011, so I can completely understand a decision to keep things private until release is imminent!

 

[Prof. Garfield]

Like you with OTR, I think, I'm identifying and then developing the modules that I want/need as a byproduct of writing the complete set of Lua events for my own scenario project. So in many ways, the actual details of how I want a given module to work, what options or parameters are useful, etc., are still a work in progress as I change my scenario's goals, test for gameplay impact, come up with new ideas, etc. I'm often making changes to a module that don't preserve backwards compatibility, and in many cases I still need that level of developer freedom. Basically I don't want to release a shortsighted module that doesn't even meet my own needs! -- or that is founded upon assumptions that I later determine to be invalid. Related to this, playtesting within my project is the way that I'm handling most of the QA and tracking down bugs to fix.

I suppose there might be an argument to developing an "ad hoc" implementation of functionality for a specific scenario, and then releasing a "polished" version as a separate module, after you have had time to reflect on what is useful. Once you know how to do something, it becomes easier to program it a second time.

I would point out that breaking backwards compatibility is only a problem if someone else is already relying on your module for their own code. And, if that would be the case, then not releasing the code is likely to cause a duplication of effort. If the scenario is likely to be finished in a few months, this is probably not too serious. However, if the scenario takes a couple years of on and off work, the duplication of effort becomes more likely.

There's also a point at which I see some Lua modules -- especially bigger, more impactful ones -- as something like the roster of units (icons, names, stats) for a scenario. If a scenario designer wants to release their units.bmp early (prior to the release of the whole scenario), along with some amount of detail from rules.txt, great! Perhaps that's a way to advertise, build up interest, obtain feedback, etc. On the other hand, I would 100% and without reservation support a scenario designer who prefers not to offer any "sneak previews" but simply wants to release their full scenario in all its glory once it's complete.

Certainly there are also some artists here who enjoy creating units independently, for the community at large, without any real intention of building a scenario on their own which will include them. That's fantastic and a huge benefit to people (like me!) who don't have those artistic gifts. I'm also willing to release some Lua modules within such a framework (e.g., supplyLines). But I don't intend to follow this approach in every case.

I think there is a distinction to be made between code that is fundamentally scenario specific and code that you could reasonably expect to get "off the shelf" in a mature community. Since we're relatively new, and very small, I think a pretty high weight should go to filling those shelves, at least with some basic tools and maybe a promise to offer more tools if requested. There's a lot of code in Over the Reich that I probably could have made general purpose if I had given it some serious thought at the time. Oh well, OTR was/is my "learning" project, I guess. On the other hand, if thinking of the wonderful surprise you're going to spring on us gives you joy, then go for it.

Hmm. Well, that seems like a trade-off, right? Then the columns would line up (perfectionist: "Yay!") but there'd be multiple different extraneous characters showing up in some lines but not others (perfectionist: "Arrgh...").

Well, either we wait for TNO to come back and work some magic to make the game print a one pixel space, or we implement our own TOT engine in LOVE2D or something similar (something that has crossed my mind more than once, and would have various benefits, but also be a lot of work). Or, the perfectionist must suffer.

However, I think it might be good if I responded separately to your other thread about The General Library. That's another area where we're tackling quite a few of the same problems independently, but not in quite the same way, and I'm not sure how (or whether) to deal with that.

Well, the General Library is mostly meant to deal with relatively simple functions, so I'm not too concerned about duplication of effort in that one. It is a relatively straightforward matter to fill much of it in, and I've suggested adding to it as an assignment in one of my Lua Lessons. Of course, if you need that kind of functionality, it would be nice if you used the general library and filled in the functions that you happen to use.

Although, now I'm curious to know what kind of work you're doing in that area. If you've been exploring city and/or unit attributes, then please share, since I don't want to figure that stuff out if someone else has already done so.

 

[Knighttime]

We're all here because we enjoy this, and to have fun -- right? Maybe I shouldn't have opened the door to a more advanced philosophical discussion -- at least in this thread. Perhaps at some point @Prof. Garfield we can continue that discussion via PM, if you're interested in doing so.

I'm attaching the work I did to support column-based output in text boxes. I hope this is useful to you and to others who are creating custom Lua events!

Or, the perfectionist must suffer.

A common fate for perfectionists!
If you prefer to insert various punctuation characters rather than using only spaces, in order to make columns line up perfectly, you're welcome to modify this code to do so. I would probably need to see and compare the output to know which I'd actually prefer, but I'm not planning to pursue that idea on my own at this time.

 

[Prof. Garfield]

I'm attaching the work I did to support column-based output in text boxes. I hope this is useful to you and to others who are creating custom Lua events!

Thanks.

We're all here because we enjoy this, and to have fun -- right? Maybe I shouldn't have opened the door to a more advanced philosophical discussion -- at least in this thread. Perhaps at some point @Prof. Garfield we can continue that discussion via PM, if you're interested in doing so.

Fun? Oh, yes, have fun. Of course...

What we really need to do is increase the size of the community somehow, so there are enough people working that duplication of effort isn't a huge concern. Not so easy to do when the only legitimate way to get the game seems to be to buy a used CD and it takes some effort to get the game to work.

 

[JPetroski]

Just out of curiosity, love2d would theoretically allow us to out free civ free civ and then distribute far and wide?

 

[Prof. Garfield]

Just out of curiosity, love2d would theoretically allow us to out free civ free civ and then distribute far and wide?

In principle, yes. I started a thread in General Discussion about that.