1. We have added the ability to collapse/expand forum categories and widgets on forum home.
    Dismiss Notice
  2. Photobucket has changed its policy concerning hotlinking images and now requires an account with a $399.00 annual fee to allow hotlink. More information is available at: this link.
    Dismiss Notice
  3. All Civ avatars are brought back and available for selection in the Avatar Gallery! There are 945 avatars total.
    Dismiss Notice
  4. To make the site more secure, we have installed SSL certificates and enabled HTTPS for both the main site and forums.
    Dismiss Notice
  5. Civ6 is released! Order now! (Amazon US | Amazon UK | Amazon CA | Amazon DE | Amazon FR)
    Dismiss Notice
  6. Dismiss Notice
  7. Forum account upgrades are available for ad-free browsing.
    Dismiss Notice

Civ3 BIQ Compare

Discussion in 'Civ3 - Utility Programs' started by Quintillus, May 11, 2009.

  1. Quintillus

    Quintillus Civ III Forum Resident Supporter

    Joined:
    Mar 17, 2007
    Messages:
    5,785
    Civ3 BIQ Compare is a utility designed to make it easier to spot the differences between two BIQ's. It has progressed from a very limited, rushed-out-the-door utility, to one that, despite remaining limitations, works like normal programs and can compare uncompressed, version 12.08 BIQ files both faster than manual comparison, and in relatively comprehensible format.


    NOTE: My cross-platform editor now entails all functionality of Civ3 BIQ Compare and much more. It also removes some of the limitations (such as the requirement that files are compressed). While I may at some point release another standalone version of Civ3 BIQ Compare, it is recommended to use the full editor to have all the latest updates. Currently (February 2011) the major difference for BIQ Compare users will be the removal of the uncompressed BIQ requirement. This program will remain available for download in case it better suits your requirements.

    Features:
    *Export the information in the BIQ file to a file, mostly non-binary format
    *Choose which sections of the BIQ file you want to print out
    *Supports Custom Rules and Custom Player Data.

    Limitations:
    *BIQ files must be version 12.08 (the latest version; most should be this version these days)
    *BIQ must not be compressed

    Future Additions
    *Continued improvement of ease-of-comprehension of output. This will likely be released incrementally.

    Screenshot



    System Requirements

    *Java, version 1.5 or later. You can download the latest version of Java here for Windows, Linux, or Solaris. For Mac OS, it appears it should come through automatic updates. If not, try downloading the lastest version for Leopard here, which requires the second-most-recent version. If you are running an alternate Java version, such as the OpenJDK, you may encounter undesirable GUI behavior (OpenJDK definitely has this).
    *An operating system that supports Java. These include:
    - Windows 98/ME/2000/XP/Vista/7 (including 64-bit XP, Vista, or 7)
    - Linux, and Solaris 8/9/10
    - Mac OSX 10.4 (Tiger), 10.5 (Leopard) and 10.6 (Snow Leopard). I believe this applies to both PowerPC and Intel, but have not been able to verify this.
    I have personally tested the current version (0.45) of the utility on Java 1.5 and 1.6 on Windows XP, and Java 1.5 on Windows 7 (64-bit).

    Download

    Download the current version here (572 KB, 176 KB .zip download).

    How-To

    The How-To formerly applied to practices that have been obsolete since version 0.12, and not present at all since version 0.21. Thus, this section has been removed. However, a new How-To may someday be created.

    PAQ (Potentially Asked Questions)

    Q: My output has boxes and is all on one line. How do I fix this?
    A: If you have boxes, it may be because the .BIQ file contains some non-ASCII characters. This is a natural occurence in .BIQ files, and is no cause for alarm. If the output is all on one line, you are using an outdated version of the program. Download the most recent version.

    Q: My file won't open. Why not?
    A: It probably falls within one of the above limitations.

    Q: I chose a file to save to, but it didn't save anything.
    A: Make sure you click "Export" after choosing the file. The dialog only selects the file. Another way to make sure the file was saved is to look for the text that appears next to the Export button after a successful save (but only in recent versions!).

    Q: I found a bug. Can you squash it?
    A: Probably. Report it here, and I'll see what I can do about it.

    Credits

    *Gramphos, for his excellent documentation of the BIQ format at Apolyton. Also to those who he credits in that thread for their contributions to the BIQ format and those who added more details to the BIQ format in replies to said thread.
    *Steph, for creating an editor for Civ3, linking to Gramphos' thread, and sparking the idea that would eventually lead to this in my head. If you haven't tried his editor yet, you should.
    *Lee_Dailey, for posting about the idea for a BIQ comparison utility at a most opportune moment. Had it not been for that suggestion, there would not presently be a BIQ comparison utility.
    *Everyone who's downloaded the program, been patient with its evolution and dearth of features at initial release, and made suggestions to make it better.
    *And anyone else whose work I may have come across and incorporated in the early stages but since forgotten. That means BIQ pioneers especially - without an understanding of the BIQ format, there can't be any BIQ utility programs.
     
  2. Steph

    Steph Multi Many Tasks man Retired Moderator

    Joined:
    Sep 1, 2002
    Messages:
    18,160
    Gender:
    Male
    Location:
    Pont de l'Arn, FRANCE
    Another programmer!

    BTW, there are a few errors in the online documentation of the file format, and a few missing info.
     
  3. Lee_Dailey

    Lee_Dailey Chief Geek & Bottle Washer

    Joined:
    Nov 27, 2001
    Messages:
    211
    Gender:
    Male
    Location:
    terra, north america, tx, bedford
    howdy Quintillus,

    kool dude! [*grin*]

    i'll be playing with your util later. hafta find one that DOESN'T have either a custom map OR custom player data. most of mine do it seems. i'll pro'ly play with one of the standard BIQ files just to see how things go. from the looks of things, it seems you've most of what i need already there or due soon.

    the compare functions are eagerly awaited! [*grin*]

    if you've a donation page or link - like what steph did for his editor - please post a link to it!

    take care,
    lee
     
  4. Ares de Borg

    Ares de Borg Norman Knight

    Joined:
    Sep 19, 2004
    Messages:
    5,061
    Location:
    Heart Of Europe
    Interesting. Can it export sets of rules to several biqs at once?
     
  5. timerover51

    timerover51 Chieftain

    Joined:
    Jul 10, 2006
    Messages:
    3,450
    Gender:
    Male
    Location:
    Chicago area in Illinois
    Thank you for including the MacOS in your programming.
     
  6. againsttheflow

    againsttheflow unpolitically uncorrect

    Joined:
    Dec 24, 2006
    Messages:
    1,985
    Location:
    northern CA
    Nice, a new utility. Sounds useful especially should the support for custom maps and such be added.
     
  7. Quintillus

    Quintillus Civ III Forum Resident Supporter

    Joined:
    Mar 17, 2007
    Messages:
    5,785
    Yep, I've encountered a few unexpected discrepencies. But with a hex editor and the Conquests editors at hand, I'm pretty sure I've ironed out the discrepencies that would cause errors in inputting the file (except for those involving teleporting). The missing info I don't have all of yet (does anyone?) but what I can find out there I'll incorporate.

    Glad it looks like it'll work. I'm working on adding more already - output to file is now working and will be included in the next update (probably with a couple other improvements as well). As for the donation link, I've managed to set one up here. I think it works - took me to a donation page when I clicked on it from a web browser I never use.

    Not at this point. But do you mean:

    1. Combining the rules of several BIQ's into a new BIQ?

    2. Taking the rules of one BIQ and putting them into a whole bunch of different ones?

    3. Something else?

    The core priorities listed above in future features/current limitations will come first, but with more clarification of what exactly it is that you want, I may be able to do this. But in the mean time, you may want to take a look at the Extract preview in Steph's editor and see if it does something close to what you want.

    No problem! :goodjob: I'd noticed the lack of Civilization programs for MacOS, and even though I'll probably keep using Windows for the considerable future, it seemed like an area where Civ could really use a shot in the arm. And Civ is, you know, important. So Java it was. And I've learned a good amount about Java from this program.

    It should actually work slightly better on MacOS than Windows at this point due to the Java carriage return issue. I don't have any Macs to test it on, but it does perform better in Linux than Windows right now.

    That's the plan. It'll take a little while to add that, but it should be there at some point.
     
  8. Lee_Dailey

    Lee_Dailey Chief Geek & Bottle Washer

    Joined:
    Nov 27, 2001
    Messages:
    211
    Gender:
    Male
    Location:
    terra, north america, tx, bedford
    howdy Quintillus,

    still haven't the time to play with the util, but you otta find $5 in your donation account sometime soon. thanks for the work!

    take care,
    lee
     
  9. Steph

    Steph Multi Many Tasks man Retired Moderator

    Joined:
    Sep 1, 2002
    Messages:
    18,160
    Gender:
    Male
    Location:
    Pont de l'Arn, FRANCE
    If you need help, I already did all this for my editor. If you don't know where an info is, just ask.
     
  10. Quintillus

    Quintillus Civ III Forum Resident Supporter

    Joined:
    Mar 17, 2007
    Messages:
    5,785
    Version 0.11 is up. Changes:

    *Output to a file is now operational.
    *Added notification pop-up opening a file failed.
    *Added descriptions to Culture BIQ output.
    *Fixed formatting irregularities in Buildings BIQ output.

    Yep - thanks!

    Thanks! Might take you up on that as it goes along. Though trying to figure it out on my own first is part of the fun!
     
  11. Ares de Borg

    Ares de Borg Norman Knight

    Joined:
    Sep 19, 2004
    Messages:
    5,061
    Location:
    Heart Of Europe
    I meant 2. :)
     
  12. Lee_Dailey

    Lee_Dailey Chief Geek & Bottle Washer

    Joined:
    Nov 27, 2001
    Messages:
    211
    Gender:
    Male
    Location:
    terra, north america, tx, bedford
    howdy Quintillus,

    spiffy! this stuff is _so_ fun ... [*grin*]

    take care,
    lee
     
  13. Lee_Dailey

    Lee_Dailey Chief Geek & Bottle Washer

    Joined:
    Nov 27, 2001
    Messages:
    211
    Gender:
    Male
    Location:
    terra, north america, tx, bedford
    howdy Quintillus,

    i finally got offa my rump and started looking at things. [*grin*]

    biq-compare version = 0.10

    a few comments ...

    [1] file dialog
    as you noted, there really isn't one. [*grin*] i vaguely recall that there was a library of such standard thingies mentioned on the sun java website. i've no idea if that stuff is still around - or even if i understood what it was - but i have one java app that uses what sure LOOKS like a sun java file dialog.

    the app is the bookshelf server for the iphone bookshelf ebook reader. it's here ...
    http://www.iphonebookshelf.com/

    the author might be willing to point you to the source. it seems rather silly to re-invent the wheel ...

    [2] successful file load
    there's no indicator that a file has loaded. well, not that i have noticed. [*grin*] 'twould be nifty to see something like the file name [and perhaps the path] in the window title.

    ===
    for the following, i ...
    - loaded an uncompressed version of the standard conquests BIQ
    - selected _just_ buildings
    - selected BIQ variables

    [3] terminal output
    when i selected just TERMINAL output & hit the EXPORT button, i expected to see SOMETHING. was expecting a command shell to pop up and was wondering if it would auto-close or not.

    what i got was ... nothing that i could detect.

    i suspect that the result would be obvious if i was a programmer, but i am bumfuzzled. what SHOULD have happened?

    [4] file output
    when i switched output to FILE, input a file name & hit EXPORT i got a file as expected. [*grin*]

    however, i have comment on the contents of that file.

    for each entry there 3 lines [lines 2-4] that are padded out with a LOT of [nul] chars. here's those lines from the 1st such entry ...

    - "description: " followed by a "buncha" [nul] chars.
    - "name: Palace" followed by a "buncha" [nul] chars.
    - "civilopediaEntry: BLDG_Palace" followed by a "buncha" [nul] chars.

    those [nul] chars need to go away, in my opinion. they exist on those lines for EVERY building in the file and that's not gonna make for easy import/export OR comparisons.

    the actual file itself is dang nigh exactly what i was seeking when i make the post you 1st responded to. neato!

    ===
    [5] the output filename
    i would like to have that auto-filled with the loaded BIQ filename plus some very obvious EXT - perhaps ".TXT"? that would hopefully make it less easy to overwrite your output files when getting ready to do a compare. which, btw, i have done twice now ... [*sigh ...*]

    end of nags ... [*grin*]

    take care,
    lee

    ps- i used this & a command line fc util to do file compares and it works oh-so-spiffily! thanks, dude.

    pps- for the windows folks who find the 'nix line endings annoying, take a lookit notepad++ here ...
    http://notepad-plus.sourceforge.net/uk/site.htm

    -lee
     
  14. Quintillus

    Quintillus Civ III Forum Resident Supporter

    Joined:
    Mar 17, 2007
    Messages:
    5,785
    Current status: File browser work started; custom player data mostly implemented, but still needs testing and need to add user-interface access to it if it passes testing (and then test that). Probably will have another release by Thursday, sooner if things go as anticipated.

    There is indeed a standard Java one. I'm working on implementing it - never used it before, but it's looks like it'll work okay. I certainly won't have to re-invent the whole wheel - and that's a good thing, as it's a pretty sophisticated wheel!

    You're right, there isn't any indicator that a file has successfully loaded. However, if the file doesn't open right, nothing will be enabled. And starting in 0.11, there's a message if the file didn't open correctly (forgot to mention that in the change log). I didn't want to have too many popups if things were working right. But a titlebar change would be a good addition. I'll put it on my to-add list. Can't guarantee I'll remember it right away, though - 0.11 still says it's 0.10!

    That's How-To #2 - to see output in the terminal, you have to start the program in the terminal. Probably should add something about that in the tooltip for that choice. There may be a way to auto-start a program such as Command Prompt so that would be visible all the time, but that would probably be different for each OS, and I don't want to get into that if I don't have to. There's a saying that Java's program-anywhere, debug-everywhere, and trying to do that would be asking for that saying to prove itself true. So far all that I've noticed is slightly askew boxes and text when running it in Linux - nothing beyond the aesthetic.

    If you start it in the terminal (see How-To #2), you'll see the output there. It's not that convenient, and that's why file output was in the first update.

    They're there because they're there in the BIQ file itself - as a bunch of zeros. They definitely won't be in the English printout, which you'll probably prefer once it's set up. I'll probably take them out of the BIQ output, too - but I'll have to look into just how I want to do it first.

    In other words, the base problem is that if I ever want to save something to a BIQ, as Ares de Borg has mentioned, I have to make sure I have the right number of null characters for each variable. They certainly aren't sightly on the output, though.

    Good! Glad it's the sort of thing you were looking for!

    I see what you mean. So you aren't as likely to type "output.txt" twice and only end up with one file? Seems a logical file output name suggestion. And the ".txt" certainly is a good idea - helps on Windows and doesn't hurt elsewhere. I'll see if I can add that when I add the file browser.

    Great! Hadn't tested that myself, but figured it ought to based on what I've used of the Unix equivalent. This utility is in all versions of Windows, for those intersted. Typing "fc /?" at the command prompt will tell you all the options as to how to use it.

    Nice! Ought to get that myself; I get enough Unix files for running Windows to utilize it for that alone!

    Okay. So that'd be like, BIQ1 has these rules, but BIQ2 has this really great map, and I want to merge them? Or, BIQ1 has these riveting resources, and BIQ2 over here is great except for its really bland resource set, and I'd like to spice up BIQ2 with those resources? Or both? I know you mean something more than just duplicate the BIQ a whole bunch of times, but you need to elaborate a bit! A programmer occasionally thinks of good ideas, but if you want a specific feature you need to let them know just what it is! We're not mind-readers! And it could be pretty complicated to do this and not cause errors in the BIQ, so I'd like to know what the end objective is before plowing ahead mostly blindly.

    (to see what I mean, think of the interconnectedness in the tabs of Civ3ConquestsEdit. Technologies are a prerequisite for units, buildings, espionage missions, even worker jobs. If you have a world map, what's on it depends on resources, buildings, the world size, the civilizations involved, etc. If you export a different set of rules for one section into the BIQ, you run a considerable risk of causing things to not work, or at least of getting some nonsensical combinations)
     
  15. Lee_Dailey

    Lee_Dailey Chief Geek & Bottle Washer

    Joined:
    Nov 27, 2001
    Messages:
    211
    Gender:
    Male
    Location:
    terra, north america, tx, bedford
    howdy Quintillus,

    [1] util version
    i didn't even notice the version # was wrong! [*blush*]

    [2] file load results
    the file load error [at least when the file is still compressed] is pretty to-the-point. so the "nothing enabled" effect is not really needed. [*grin*] in that case, at least! having the name in title bar is one very handy way to be sure one has done what one _thinks_ one has done. [*grin*] i'm looking forward to it.

    [3] terminal output
    umm, err, just WHY is there a terminal output option anyway? i presume it's there for debugging - at least that's why i do it when i'm diddling with python. you might wanna remove that option from the UI when ya get farther along with the util.

    [4] embedded [nul] chars
    since each such item [buildings with different length names in this case] will vary in the exact number of [nul]s needed, i would simply have the required length of the field saved and use that to pad out the field when you get ready to re-write to a BIQ file.

    [5] notepad++
    try it some time when you are in the mood. it's a nifty program and really rather fast. i sometimes use it to snoop in binaries when looking for undocumented options or version info. mostly i use it for seeing several files in tabs so i can copy-n-paste cleanly. also, the html and python hiliting is handy.

    thanks again, dude!

    take care,
    lee
     
  16. Blue Monkey

    Blue Monkey Archon Without Portfolio

    Joined:
    Jul 31, 2005
    Messages:
    10,901
    Location:
    Limbo (non-Landmark Virtual Aether)
    Post a thread over in the Mac C3 forum. It could just be an announcement with a link to this thread as the main place for discussion. You'll probably get some testing & feedback from Mac fans. For sure AlanH will see it - he's made a few utilities for the Mac players.
     
  17. Quintillus

    Quintillus Civ III Forum Resident Supporter

    Joined:
    Mar 17, 2007
    Messages:
    5,785
    Version 0.12 is now live. Changes:

    *Title bar changes to reflect opened files.
    *Now have file choosers for both opening and closing files (optional on close - you can just type in the output path if you wish).
    *Can now handle Custom Player Data. All that's left to add is Custom Maps and taking away the requirement that there be custom rules.

    However, the final change resulted in one regression:

    *The program no longer usually catches invalid input files. This is a short-term tradeoff of getting Custom Player Data support. Basically, it means the program is not idiot-proof. Only open files that meet the now less-stringent requirements. Idiot-proofing will be added back in later.

    So added to the list of changes with these new features:

    *Re-implement bad input choice catching.
    *Limit inputs to BIQ files only (which helps with the above).
    *Possibly change the default look and feel, because Java's default one (on Windows/Linux) doesn't look so great with Open/Save dialogs.

    And one gotcha:

    *The Browse dialog for saving files selects the file you want to output to. You still have to click the Export button to actually do it. Consider it short-term file overwriting protection. An actual warning message will be added later.

    Neither did I - until too late! But it's right this time!

    Well, the file load error works well in 0.11; it doesn't make itself known much in 0.12. Tradeoff for quick implementation of Custom Player Data. I've got a couple ideas as to how to solve that problem (it's unfortunately not as easy with mixed binary/string data as with only strings), so that should be re-implemented soon.

    But the nothing enabled effect adds to it, and prevents users from trying to do things that won't work. No reason to risk users thinking things will work when they won't.

    At first it was because it was what I already had before I saw your first post, and thus it was easy to implement. Now it is mainly for debugging - there's no good reason not to export to file now, especially with the file browser. I probably will either take it out or make it less prominent in the future.

    That's probably what I'll end up doing. It's not in this update, but it will be soon.

    Perhaps I didn't make it clear in the first post that this is very fresh software - I'd been working through the BIQ a little bit per day earlier this month, but hadn't expected to release anything until several weeks from now at soonest until I saw your post, realized I was pretty close to being able to do that, and sped things up.

    So there's still a lot of development cobwebs on it, as well as missing features and potential traps for the user to fall in (such as the lack of overwrite notification). But it does do what it says it will if it gets what it wants, and cobwebs are being dusted off with each release (although occasionally dusting off a cobweb reveals a spider, as with the lack of file open failure notices in 0.12). So all these features are on the to-do list, but they don't happen magically. Hence why the to-do list is longer than the Features list :D!

    Sounds useful. Considerably more so than Notepad! :lol: And as Notepad's my #1 opened program according to Windows, a better version couldn't hurt!

    Thanks - I certainly will before too long.
     
  18. Steph

    Steph Multi Many Tasks man Retired Moderator

    Joined:
    Sep 1, 2002
    Messages:
    18,160
    Gender:
    Male
    Location:
    Pont de l'Arn, FRANCE
    Hi!

    Did you get my PM?
     
  19. Quintillus

    Quintillus Civ III Forum Resident Supporter

    Joined:
    Mar 17, 2007
    Messages:
    5,785
    Now I did - missed it before. I think the flashing mailbox is gone? Reading it now.
     
  20. Lee_Dailey

    Lee_Dailey Chief Geek & Bottle Washer

    Joined:
    Nov 27, 2001
    Messages:
    211
    Gender:
    Male
    Location:
    terra, north america, tx, bedford
    howdy Quintillus,

    i'm familiar with the "zero dot" version scenario. [*grin*] i'm enjoying watching you work ... while lazing about, myself. the new features look like fun. i'll try and get some more play time soon, diddle with v-0.12 and give you some more back seat driving - err - "feedback" later.

    take care,
    lee
     

Share This Page