Cross-Platform Editor for Conquests now available! 1.48

As it was, I was a second-year student when I started the project. Armed with the boundless optimism of someone who doesn't know how little they actually know, as well as fairly comprehensive documentation of the scenario format at Apolyton, I jumped in on Saturday, April 4, 2009. In those heady days, Steph's Editor was still receiving regular updates, 0.53 being the latest at that time. Why another one was needed, I'm not sure I knew even then. What I did know was that I felt like maybe I could also make a Civilization editor, and that I didn't know anything about C#, which Steph was using. In retrospect, now knowing how similar Java and C# are, it likely would have made sense to learn some C# and offer to help on that project, or perhaps use some of my newly-acquired C knowledge to help with another utility. But Gramphos's documentation was beckoning, I knew some Java, and starting from scratch was exciting. So the editor began!

After a few weeks, I realized that there was a lot to the BIQ format (more on this later). I could read a few sections, such as the building section, but was still a ways off from reading an entire scenario file. I was starting to wonder what I could really do with the proposed project, and whether it made sense to continue when Steph had already cranked out three updates in April alone and was still adding functionality at a rapid rate. That's what I read Lee_Dailey's post requesting a utility to compare BIQ files. Bingo! Something that Steph's editor didn't do, was probably too niche for it to do anytime soon, and I could do without having to add the ability to modify everything in the BIQ. After a week or so of hasty coding, Civ3 BIQ Compare went live in May, 2009, with support for standard rules only. A couple days later, world maps would be supported, and regular updates would follow into June.

While by no means a full editor, this intermediary goal likely saved the project from languishing forever in an abandoned folder on my hard drive.

It was around June, 2009 when Civ3 BIQ Compare more or less reached its baseline functionality, and my focus switched to building out tabs to be able to edit the scenarios. Buildings came first, naturally. As I was not experienced with coding user interfaces in Java at that time, I was using a GUI-builder. It had some idiosyncracies, and sometimes moving a button or label a tiny amount would throw everything in the tab off. That became pretty old pretty fast, and more than once while walking around campus I found myself thinking that I'd have to learn how to write GUIs so I could be rid of the insane GUI-builder. But in the summer of 2009, the focus was on the finish line. So I stuck with my current modus operandi.

Eventually, though, limitations were becoming apparent. I'd put each of the BIQ sections - resources, units, buildings, and so forth - in their own file from the beginning. But all of the front end - the user interface - was in one big file. And "big" was becoming "gigantic". Over 10,000-lines. It was not only becoming unwieldy to navigate, but unresponsive, particularly as I added more tabs. I realized I'd have to rewrite the GUI or the editor would become too slow to develop before it was even released.

But before that happened, I received an error that my code could no longer be compiled. All that GUI code I'd put in one class? Well, most of it was in a single method. One giant set of tabs within one window, containing a whole editor's functionality, specified in one place. It turns out that giant method had exceeded 64 KB in size, which is the upper limit for compilation in Java. At that point, I had no choice but to split up my GUI code if I wanted to add more functionality. And so, it was to the chopping block. Each tab now received its own file, to which I copied the file that the GUI-builder had automatically generated. I then spliced them all together, and not only was I below the compilation limit, but responsiveness in development was much improved. The GUI-builder no longer worked with the split-off tabs, but in some ways that was a good thing. It made them difficult to modify, as the GUI-builder's code was not particularly human-readable, but it made me learn about GUI coding in Java.

Incidentally, this was around the fall of 2009, about the same time that Steph reorganized his editor from one window with a bunch of tabs to a window that created separate windows for each tab. Based on that and some of his posts at the time, I suspect he may have been running into similar slow-development-environments difficulties, but chose a different solution.

At any rate, it was at this time that many of the foundations of the project were build. These included:

- Lots of auto-generated GUI code that's difficult to modify. This is both directly and indirectly responsible for some of the UI errors in the current editor, as well as the fact that the GUI has not changed a whole lot over the years.
- The file input/output routines being coded in a C style with essentially no data protection. The editor tabs directly accessed the variables stored in the I/O files (such as TECH and PRTO). This was quick and convenient, but also meant that particularly for binary fields, of which the BIQ format has more than a few, the code modifying the binary data was all over the place. This was displayed perhaps best when I realized that the base and real terrain documentation at Apolyton was swapped. Already having code dealing with that everywhere, I simply swapped the labels and left them reversed in the code, it being too big a mess to fix at the time.

But despite some questionable coding practices, progress continued.

Whereas Steph's editor had taken the (probably sensible) route of starting out with just a few tabs editable, I decided I wanted to have all non-map tabs available at release. As a result, it took over a year after the glory days of BIQ Compare to complete version 0.50 of the editor, the first release.

At release, there wasn't really anything the editor could do that Firaxis's or Steph's couldn't do. The one exception was run on non-Windows operating systems. This was largely the reason the editor was billed as the Cross-Platform editor. It was part of the reason for its development, too, as that was the obvious niche to fill. But I didn't actually own any Apple products at the time. So it was really more a case of the project looking for a need to fill and finding one, than starting the project because I needed it to solve a problem.

The initial release of the editor came on August 30, 2010, and a number of updates followed in fall 2010. These focused on improving compatibility with various scenarios, but also included multi-threading of BIQ input. This was perhaps the first feature that was fairly well-designed, at least if more sophisticated ways to do it that had been introduced in 2004 (and I discovered around 2012) are ignored. But with lock-free, thread-safe multithreading, it was an improvement that worked as intended, and has so far escaped serious thought of a redesign.

The winter of 2010-2011 saw some new features added, such as CSV export (a user request), and Autosave. But it was version 0.70 in April, 2011 that was the big one - adding the first support for map editing. Map editing was not very sophisticated early on. It was functional (with a few bugs), but that's about it.

It perhaps make sense now to talk about how the map editing works. Having zero experience with actual graphics coding (as opposed to building user interfaces) before, I did some research ahead of time, perhaps not enough. Already using Java and definitely not wanting to re-write all the BIQ input/output code in another language (knowing how long that took), the options were somewhat limited. I found three main options. Java 2D likely would have made sense, but required Java 1.6, and I'd gone to Java 1.5 since some current users of the editor couldn't upgrade beyond that without upgrading their operating system, and every user counted. Then there was Java 3D, which may have worked, but hadn't been updated since 2008. Learning it in late 2010-early 2011, when it looked pretty much abandoned, didn't seem to make much sense. And indeed, it still hasn't had a stable release since 2008, so that may have been a good choice.

Instead, I went with using Swing, Java's general-purpose GUI toolkit, for the map editing. I figured out that it could be used to draw graphics on a canvas, and though it required a fair amount of manual overhead, it could be used to do the type of image manipulation I'd need for a map tab. What it boils down to is purely-CPU powered graphics, with no graphics card involved. It's relatively slow, and not very elegant, but does work. And on the plus side, it's meant zero complaints about graphics cards not being compatible.

It also means that nearly every aspect of scrolling across the map, interacting with it, and so forth had to be added separately. It's why zoom support took such a long time. If I'd been using a 3D framework, it likely would have simply been a matter of zooming out the camera. Instead, it involved scaling the graphics to whatever scale was requested, and making sure that the scrollbars still worked properly at any zoom level. And just as much of a killer - making sure that clicking on a tile at any zoom still registered as clicking on the correct tile, even if you clicked near a tile boundary or were at some odd zoom like 43.221%. It took multiple tries to get zoom right, as well as many months of time. Ever wonder why most 2D games like Civ3 and Age of Empires II only support a couple zoom levels? It makes a lot of sense to me now.

But there was one other key aspect of showing Civ3 maps that needed to be solved. That was displaying PCX files. Java doesn't support PCX natively, likely because PCX was rather old even by the time Civ3 came out. Looking around for a good Java PCX library, I didn't find any promising candidates. Fortunately, the specifications for PCX is readily available, and not overly complicated, so I wrote a PCX import class so I could import PCXs. Again being purely-CPU-powered (as well as purely in Java), it's not the fastest PCX-importing utility ever, but it does the job. And unlike most, it does have native support for the Civ3-specific extension of the last two colors controlling transparency.

Frequent updates would continue through version 0.75 of the editor, in July 2011, with 24 releases in the first year. Likely not coincidentally, July 2011 was also just before I started my first professional programming job.

Speaking of professional programming jobs, it's getting late, and I have one tomorrow. So this history will have to be added to later...