1. We have added a Gift Upgrades feature that allows you to gift an account upgrade to another member, just in time for the holiday season. You can see the gift option when going to the Account Upgrades screen, or on any user profile screen.
    Dismiss Notice

[REFERENCE] Working with PCX Palettes in GIMP

Discussion in 'Civ3 - Tutorials, Reference, & Guides' started by Blue Monkey, Jul 28, 2017.

  1. Blue Monkey

    Blue Monkey Archon Without Portfolio

    Joined:
    Jul 31, 2005
    Messages:
    11,039
    Location:
    Timeless Isle
    Working with PCX Palettes in GIMP

    GIMP can be confusing about how to work with indexed colors as needed for C3 pcx files. Terminology is not necessarily consistent. Different parts of the process are in completely different menus. Here are some general explanations. There's a summary of the process at the bottom of the post.

    This post is only about pcx – the static files such as civilopedia icons, terrain, and so on. Other types of files - such as unit animations - use a very different palette arrangement. Someone else (Vuldacon iirc) has written very detailed explanations of how to work with those palettes. Be very clear – what is written here is only about pcx.

    I'm currently using GIMP version 2.8.16 on a Mac. Your UI may look slightly different from the screenshots depending on what version you use, but the basic actions are the same.


    Terminology

    About the use of “Green” and “Magenta”

    As has been mentioned many times in many posts in many threads it is the location in the palette/colormap that determines what is invisible in-game – not the particular color. The standard game files use bright Green (RGB 0 255 0 ) and Magenta (RGB 255 0 255). The reason those two colors were used is because they are highly visible and rarely used for in-game content so it is very easy to tell what will be invisible. So most of us use the same for our files. Makes seeing what other people have done in creating graphic files easier. Same goes for understanding illustrations accompanying posted explanations and tutorials. For all of those reasons it has become common practice on the forum to write “green” and “magenta” as shorthand for “the colors in the last two slots of the palette which are reserved for what needs to be invisible in-game.” The explanation below follows the common practice in both text and pictures. But keep in mind any colors can be used. This comes in handy if your pcx includes pinks and greens that are close enough to those colors to be confusing.

    You will see posts where people use “reserved color” to refer to the last two slots. In my own thinking and in writing this up I use “reserved” also to mean the two colors I want to make sure show up in the colormap, even if I have to change them around later.


    “Colormap” vs. “Palette”

    This can be a second source of confusion. Most of us casually use “palette” for what GIMP calls a colormap – the indexed (ordered) set of colors associated with a particular image. Gimp uses “palette” to refer to files which are saved separately from the image. But it occasionally uses “palette” when referring to manipulations of the colormap – confusing when learning what to do and how to do it.


    Techniques

    To change an RGB image to indexed:

    In the menu: Image > Mode > Indexed … brings up the dialogue window.



    In the Conversion window choose optimum palette. This will generate a colormap unique to your graphic. Using pre-existing palettes is a whole other thing we won't get into here.



    This step can be confusing because the actual colormap index is numbered 0-255 (256 different colors) but by default GIMP wants to make a palette with 255 colors. :dunno: Make sure to increase the Maximum number of colors to 256. If you don't then the game engine may try to fill in those unused slots. You can end up with problems like green or magenta showing up in game because they are not in the last two slots.

    In the same window you can set dithering – the specific mathematical technique that will be used to reduce the number of colors used. Don't worry about how that works – we just care about the results. Not every image gets best results with the same dithering technique. Experiment.




    To get the Green & Magenta in the proper slots:

    I'll explain in the section further down on reserving colors why I've added green and magenta dots to the image before indexing. For now just know that they help make things easier in the long run, and will be removed before saving the final pcx.

    After you have indexed the image you can rearrange colors in the colormap.

    This stage can be confusing because it's in a different part of the menu than changing the mode.

    In the menu: Colors > Map > Rearrange Colormap … brings up the dialogue window.



    Don't worry if the other windows disappear when the Rearrange dialogue opens. They'll come back when you close it.

    The last two slots in the colormap are where the game engine looks for the “invisible colors”. It doesn't matter how all the other colors are arranged.

    In the Rearrange window: just drag and drop the magenta and green to the last two slots. Don't worry about the little numbers under the color box. They will get changed automatically by GIMP when you close this dialog.



    You can safely drag and drop any other colors you want as well. It won't change how your image looks. You're not moving pixels in the image or changing their colors, just changing where in the colormap the game engine will look for what to do with them. There's even a way to sort the whole colormap, but it isn't really useful for our purposes. Just make sure that green and magenta end up in the last 2 slots.


    To change colors in the colormap:

    This is useful if you want to have green and magenta as the invisible colors but they aren't in the colormap. It is also one way to adjust other colors to smooth out problems caused by the dithering.

    Yet more confusion at this stage: you have to go to yet another part of the drop-down menus: Windows > Dockable Dialogs > Colormap brings up the dialog window.



    In the Colormap window select a color by clicking on it. Then click on the pencil icon.



    This brings up a window very similar to the one you use for selecting colors in the Toolbox. Use whatever method works best for you to get the color you want.



    After you've changed the color click Okay. The Colormap window will still be open. So you can repeat the process to change as many colors as you want.


    Ways to reserve or make room for green and magenta in the colormap:

    This section is last in the order of explanation because it is optional. However, adding areas of color must be done before indexing.

    If you try to correct the colormap by changing the last two colors you may end up with green or magenta in the actual image. This happens because you have edited a color that is actually in the image, and - as shown above - it doesn't make any difference where the color is in the colormap. There are different ways to avoid this.

    One way to correct this is to pick a color very close to the one you want (that got changed to green or magenta). Then Select by Color, and Fill. But making sure every single pixel that is green or magenta gets changed can be very tedious. Or ...

    If you've got a large area of the same color in your RGB image you can paint big fat dots of green and magenta before you change mode to indexed. Paint a solid area rather than one or two pixels to make sure they stay in the colormap when it is reduced. After it's indexed just go back and paint/fill the green & magenta to match the surrounding color. Or …

    Maybe you can't or don't want to paint over part of your image. You can make a frame or border in the RGB image that will get cropped after indexing. This frame makes the image larger than the size you want. It gets cropped after you index – restoring the image to the size you actually want. But the colors in the part you've cropped away stay in the colormap even if they aren't present in the part of the image you keep. Or …

    Sometimes if your image has very few colors you will end up with blank spots in your colormap just because there are not enough to make 256. Make part of the border a rainbow gradient. Or make a gradient between two of the dominant colors in your image. You can make these gradient borders much thinner than I've illustrated. Not all of the colors need to end up in the colormap. You don't want them to get chosen instead of an important color in the part of the image you will keep. Make sure there is still a large area of green and magenta in the border so that they will be reserved in the colormap.

    Examples of four ways to get green and magenta into the colormap by including them in the image


    With any of these methods you may still end up with some green or magenta if the colors used in the image are close to those. In the example icon the green on his hat might cause a problem. You can always use other colors in your border that are not even close to the colors in the image. In the past I've used bright Blue and bright Yellow, for example. Bright colors that are very different from colors in the image are good choices because they are easier to spot. You can leave those two colors as the reserved ones and rearrange the colormap as needed. Or after you index just change those two colors to green and magenta.

    If you do use the borders technique always make sure to crop the image down to the correct size before saving and using the pcx. If you forget to crop you will with unpredictable or at least unhappy results when you try to use it in-game.


    Summary
    • It's whatever colors are in the last two slots of the colormap that are invisible in game. We use and talk about green and magenta as the invisible colors only because they are what Firaxis uses in the original game files.

    • In the menus: Image > Mode > Indexed brings up the dialog to convert RGB to indexed. Make sure to set maximum number of colors to 256. This dialog is also where to set how GIMP dithers (chooses which colors to keep).

    • In the menu: Colors > Map > Rearrange Colormap … brings up the dialogue window to drag and drop green and magenta into the last two slots of the colormap.

    • In the menu: Windows > Dockable Dialogs > Colormap brings up the dialog to edit particular colors. Click on a color, click the pencil icon at the bottom of the Colormap window, then change the color in the window that pops up.

    • You can make sure green and magenta are reserved in the colormap by putting big areas of them in the image while in RGB mode and then repaint them after indexing. Or by using them a border around the image that gets cropped after indexing.

    While getting used to working with palettes lots of “opportunities” for questions are going to come up. Be sure to ask. The only difference between an expert and a novice is that the expert has already made most of the common mistakes and knows how to correct or avoid them. :coffee:
     
    Last edited: Jul 28, 2017
    Meteor Man likes this.
  2. Laurana Kanan

    Laurana Kanan Don’t underestimate who I am.

    Joined:
    Apr 10, 2014
    Messages:
    1,712
    Gender:
    Female
    Location:
    Near the Greatest Snow on Earth
    Very nice summary with good detail BM. I used to always refer people to @Kyriakos 's excellent tutorial (or, sometimes Gray Wolf's), but I think I like your layout a bit better.
    Imo, an easier method is to index with 254 colors and then use the "plus" button on the Colormap dialogue to add the two transparency colors.
    This can also be reached directly on the Colormap dialogue by left-clicking on the arrow at the top.
    I saw this mentioned before years ago, and I still don't understand the purpose for doing this. Back when I used GIMP and had unused index slots, I would just add black to all of them in the Colormap dialogue. I also seem to remember that GIMP would automatically fill in any unused index slots with black anyway once the indexed image had been saved.

    In PS I have set-up a custom "forced colors" palette of Magenta and Green in the correct positions that are automatically added to an image when I index it. Out of curiosity, I'm wondering if you know of a way to accomplish the same thing in GIMP, which could simplify the indexing process for others?
     
  3. Blue Monkey

    Blue Monkey Archon Without Portfolio

    Joined:
    Jul 31, 2005
    Messages:
    11,039
    Location:
    Timeless Isle
    There always multiple effective ways of doing things. I'm glad you pointed out some alternatives.

    I'm not at all familiar with that technique. My only practical experience with a preset palette is the one I use when making bmp maps to convert in Quintillus' editor, but that's a whole palette for a specific purpose.

    There are certainly much more sophisticated ways of working with palettes than what I've posted here. Adding color slots to a smaller palette, creating and using palettes separate from particular images, none of those are first steps in the learning curve (imho). This got written in response to some questions by Theov in another thread. My aim here was to give a simple dependable guideline to get people started on the way to finding their own best practices. Knowing where to find things is the first big hurdle, right?

    Sometimes I don't like the result of the dithering with the green and magenta included. I will do the dithering with the image taking up all 256 slots, then spend time deciding which colors are close enough to another shade, and are used very little in the indexed image, that those pixels can be selected and filled with another color, thus freeing up those slots in the color map for green and magenta. That used to be the only way I worked. But trying different colors until I found one that wasn't used too prominently in the image often took a lot of time i would rather spend on the more creative parts of making things. Making a green and magenta border then trimming it really streamlined the process of producing enough icons, splashes, etc. for a whole project. It makes sure those two colors are in the final colormap, it's simple, and gets reliable results.

    Each creative person finds the techniques that work best for them. I prefer to do as little direct colormap manipulation as possible. And in my experience teaching people that don't know much about working with palettes a way to do things that keeps them focused on the image rather than the dialogue boxes is the most effective way to get them started. I always try to lay things out in a way that builds fundamental understanding* rather than giving someone a bunch of shortcuts (advanced techniques) right off the bat. They may be lost when something happens that they don't want, but that needs more understanding to fix than just knowing some very specific techniques. But that's just me.

    Exactly. The number you set in the dialogue is the maximum number of colors. Most images have much more than 256 colors. But with some images GIMP could very well create a palette with much fewer. And it puts those "black" empty slots after all the other colors. Which means that (if you've added a border or other wise included them in the image) the green and magenta are not in the last two positions. You're in a situation where you have to know what to do with those empty slots. I've seen unpredictable things happen with those extra unused slots - like green and magenta getting shifted away from the end again after I've edited colors or rearranged the colormap and closed the dialogue. Sometimes the "extra" colors would actually be black and would stay in place - I wouldn't need to edit the color. In some cases I had to go back and edit the colormap slot by slot so that what looks "black" but is really "empty" has another color. It was too unpredictable. Which is why after reading someone's suggestion here in C&C many years ago I started experimenting with using gradient borders when the image doesn't have a lot of colors. I can't imagine trying to make the Red Cross wonder splash without using a gradient border. Would have taken far too long any other way. For someone just learning about making icons for Civ using the borders is easier to understand (imho), fairly quick, and less likely to give mysterious results they don't know how to fix.

    Long story short. I wrote up a beginner's reference guide to help clear up some confusion. I don't consider it a full-blown tutorial. And it certainly won't have much to offer to someone who already gets it and is looking for faster, more efficient ways to do particular things. Even so, I hope you will post more details about how you do things, and others will as well. Who knows what will make the light bulb go on for someone else?

    -------
    * One example of fundamental understanding (this actually happened). One particular color I really wanted to be in a wonder splash kept getting lost in the dithering process. There just wasn't enough of it in the image, even though it was visually important. Already knew that I could make sure magenta and green were in the image by using the border technique. :hmm: Lightbulb! By adding the color I wanted to the border I could make sure it was there in the final image without having to do any pixel by pixel work, or colormap editing.
     
    Last edited: Jul 29, 2017
  4. Bluemofia

    Bluemofia F=ma

    Joined:
    Dec 8, 2003
    Messages:
    7,976
    Location:
    Dimension called Elsewhere
    So I'm a weirdo who uses 3 art programs to do my graphics these days.

    The actual image manipulation, is MS Paint and Paint.net, but for palette work, I use Irfanview.

    Irfanview doesn't have the ability to re-arrange the palette via drag and drop, and any palette changes will visually appear immediately. To change things:
    1. Decrease color depth to 256 colors.
    2. Manually adjust the RGB for the current pink and green to match the last and 2nd to last colors.
    3. Export the palette, and exit out (or switch forward and back images via the arrow keys)
    4. Import the palette back in. Edit the palette again to confirm Pink and Green are the last 2 slots.
    5. Save.

    It doesn't run into any weirdness with less than 256 colors, as it just replaces everything as black. If you happen to have black, it will be the first black, and not the subsequent ones.
     
  5. Laurana Kanan

    Laurana Kanan Don’t underestimate who I am.

    Joined:
    Apr 10, 2014
    Messages:
    1,712
    Gender:
    Female
    Location:
    Near the Greatest Snow on Earth
    @Blue Monkey I guess I take a different view, in that some of the methods you outlined above seem to be unnecessarily complicated for a beginner. However, no worries, I understand what you were trying to accomplish with this tutorial and I'm sure it will be of benefit to others.

    Here's links to the other tutorials I mentioned: How to make a working palette for pcx using the Gimp & GIMP PCX tutorial in case anyone finds something useful there. Unfortunately, though, it looks like all the pictures are gone now from them.

    Below is a pic of what I was curious about if there was a method in GIMP to achieve the same thing. I'm not aware of one, but it would certainly stream-line the process where you wouldn't have to worry about the transparent colors.
    Spoiler :
     
  6. Blue Monkey

    Blue Monkey Archon Without Portfolio

    Joined:
    Jul 31, 2005
    Messages:
    11,039
    Location:
    Timeless Isle
    Spent some time searching the web. I can't find any mention of forcing colors in GIMP. Now that I see it, it does seem like it would be a real time saver. In the screenshot you've got green & magenta in the first 2 slots rather than the last 2. Does Photoshop display the palette from 255 to 0 ?
     
    Last edited: Jul 29, 2017
  7. Laurana Kanan

    Laurana Kanan Don’t underestimate who I am.

    Joined:
    Apr 10, 2014
    Messages:
    1,712
    Gender:
    Female
    Location:
    Near the Greatest Snow on Earth
    Yes, for whatever crazy reason the index numbers in PS color tables are reversed. :crazyeye:
     
  8. register

    register Chieftain

    Joined:
    Jun 10, 2014
    Messages:
    274
    Very nice tutorial Blue Monkey, thank you for that. I was trying to follow it but still don't understand how to pick on coloramp the right color I need. I have many slots for many shades or white or yellow but I can't find which one caused the problem as per attached images. Do you think you could add more explanation how to pick exact color I need? I did't add any transparency slots because I thought I don't need for this particular image.
     

    Attached Files:

    • 1.jpg
      1.jpg
      File size:
      1.4 MB
      Views:
      112
    • 3.jpg
      3.jpg
      File size:
      1.4 MB
      Views:
      116
  9. Blue Monkey

    Blue Monkey Archon Without Portfolio

    Joined:
    Jul 31, 2005
    Messages:
    11,039
    Location:
    Timeless Isle
    All the pcx files for civ need 2 transparent colors.
    Post an attachment with the pcx & I will look at it for you. Once I see what is going on with the actual pcx file I can post an explanation so you will know what to do differently next time.
     
  10. register

    register Chieftain

    Joined:
    Jun 10, 2014
    Messages:
    274
    Sure, why not. Thank you

    I have three examples with the same problem, I don't know how to pick the right colors. I think that was the issue I always had why I never finished any palette properly.
     

    Attached Files:

  11. Blue Monkey

    Blue Monkey Archon Without Portfolio

    Joined:
    Jul 31, 2005
    Messages:
    11,039
    Location:
    Timeless Isle
    It was a simple fix. Once you've done it a couple times it should be easy and quick for you. Took me a total of less than 10 minutes (closer to 5) to fix all three images.

    In the spoiler the three steps are illustrated.
    Spoiler Fixing a palette :

    In the colormap you can see the problem right away.


    (A) Following the directions in the "to change colors in the colormap" section of the reference guide (first post) ...
    Select a color that has several others very close to matching. Edit that color to magenta. Then do the same with another color for green.

    (B) Pick and Fill to replace the magenta and green in the image.




    (C) Following the directions in the "to get the green and magenta in the proper slots" part of the reference ...

    Drag the magenta and green to the last two slots



    Done.



    NB:
    You can pick colors on the image with the Edit Colormap dialogue open. This lets you easily find colors that closely match. Besides using this to get the magenta and green you can also edit colors to make areas which are very pixelated blend better. You can simply edit the color, or pick & replace. You don't have to rearrange the colormap for this part.

    Sometimes this is quicker than just picking & replacing from the main toolbox. You can click on the color you want to be the new one in the colormap, where you can see how closely they match. And you can change as many colors as you want by repeating the process with the whole colormap visible.


     

    Attached Files:

  12. Blue Monkey

    Blue Monkey Archon Without Portfolio

    Joined:
    Jul 31, 2005
    Messages:
    11,039
    Location:
    Timeless Isle
    @register -

    Please let me know if the files are working now. Also if the further explanation in the spoiler was any help.
     
  13. register

    register Chieftain

    Joined:
    Jun 10, 2014
    Messages:
    274
    I was trying to follow it and everything going to good until I have to rearrange colour ramp. When I dragged magenta to the last slots everything started looking weird?
     

    Attached Files:

    • 1.pcx
      File size:
      1.8 MB
      Views:
      86
    • 2.pcx
      File size:
      1.9 MB
      Views:
      53
    • 3.pcx
      File size:
      1.9 MB
      Views:
      74
  14. register

    register Chieftain

    Joined:
    Jun 10, 2014
    Messages:
    274
    Ok, think I fixed that. I was replacing the last colour with magenta instead moving to the last slot. I will check my results later this evening.
     
  15. Blue Monkey

    Blue Monkey Archon Without Portfolio

    Joined:
    Jul 31, 2005
    Messages:
    11,039
    Location:
    Timeless Isle
    That is actually a very easy fix. And you can do it without dealing with the colormap menus.

    From the main toolbox use the eyedropper tool. Pick a color that is almost the same as what the magenta is really supposed to be. That becomes the foreground color. Now use the "select by color tool" - it looks like a pointing hand. Pick to select all the magenta. Then from the edit menu choose "fill with foreground color".

    You can select and change colors in the picture before you move magenta in the colormap or after. It doesn't matter at all. After you have selected and changed the color the magenta is not in the picture itself. But it is still in the colormap/palette, so that part is fine.
     
  16. register

    register Chieftain

    Joined:
    Jun 10, 2014
    Messages:
    274
    Yes, it's working for me now, nice and easy. Thank you, sent you pm also.
     
  17. Balthasar

    Balthasar Wise Man

    Joined:
    Mar 11, 2005
    Messages:
    3,268
    It seems that there are many ways to reach the same destination. I begin by putting my image on a transparent background, indexing (with the background still transparent) and then adding-in the key colors last. I've found that this completely eliminates any issue of bleed-through. Also, it makes working with layers far easier (I use a transparent background as default anyway). I've posted a crude animation to illustrate my technique:

     
    Last edited: Jun 4, 2018

Share This Page