Getting Civ 4 Units into Civ 5

27th January 2014 - This tutorial is completely out of date! An updated tutorial for fully converting Civ 4 units with their animations to Civ 5 is available here.


Are you sitting comfortably? Then we'll begin...

(Don't be freaked out by the length of this - it's just because I am trying to cater for more or less complete Blender noobs.)

What you'll need...

This zip file with the files used in the tutorial as well as a Civ 5 mod containing the completed conversion.

1. Blender 2.49, PyFFI and Blender Nif Scripts - follow the order of installation in the instructions.

2. Nifskope

3. A paint package that supports DDS texture file import/export such as Paint.NET.

4. IndieStone NexusBuddy 1.0a

5. Optionally GrannyViewer to view GR2 files.

You may also need PakBuild to unpack any Civ 4 art you might want to convert (a lot of big mods pack their art too):


1) Stage 1 - Cleaning up NIF in Nifskope

For now I won't go into this stage in detail. If you can import the NIF file you want to convert into Blender then you can (probably) skip straight to Stage 2.

It is only relevant for community-made units that have such a messy structure that they need to be cleaned up in Nifskope before they can be imported into Blender. The example unit I am using for this tutorial was a typical example of this. I'll give some pointers on how to clean up meshes using Nifskope later.

This is a nice clean tree structure for a NIF file and what we are aiming for in this first stage.
-Scene Root
----MD
-------MD NonAccum
----------BIP (or Bip01 or HorseBip01 or whatever) - Weapon etc meshes will be children of bone nodes
----------Mesh 1
----------Mesh 2
----------Mesh 3

For any unit that imports without any issue into Blender you probably won't need to do Stage 1 at all. We'll see how people get on.

If you're interested, you can probably learn a fair amount by comparing ibutho.nif with ibutho_nifskope_clean.nif in the attached zip.

Stage 2 - Blender - Making a single mesh with a single material and texture

First up...

Blender User Interface Basics and Hotkeys:

Spoiler :

3D View:

Zoom - Mouse Wheel
Rotate Camera - Middle Mouse Button + Drag

Front View - 1 (Numeric Keypad - if you don't have one on your keyboard you can change the options to use normal number keys)
Side View - 3
Top View - 7
Shift + any of the previous three reverses the view so Shift + 1 is Back View and Shift + 7 is Bottom View

TAB switches between Object Mode and Edit Mode

General Blender Hotkeys:

G - Move selection (remember as Goto) (move mouse then LMB when done)
R - Rotate selection (move mouse then LMB when done)
S - Scale selection (move mouse then LMB when done)
A - toggles Select All/Deselect All
X - Delete selection

G followed by X, Y, Z - Move selection along specified axis
S followed by X, Y, Z - Scale selection along specified axis
S followed by a number - Scale by multipler .5 for half, 2 for double.
S followed by X, Y, Z followed by a number - Scale by multiplier along specified axis.
R followed by a number - Rotate the specified number of degrees.

Ctrl + LMB Drag in a loop - Lasso Select
B then LMB Drag - Box Select
M followed by X, Y, Z - Mirror selection in specified axis
Shift + D - Duplicate (Clone) selection

Object Mode:
Right Click to select an object
Right Click to select an object, then Shift-Right Click another to select another object to join it to, then Ctrl-J will join the objects into a single object.

Edit Mode:
Ctrl-P: Se(P)arate selection as new object.

To get a unit from Civ 4 to Civ 5 we need to convert it from the NIF file format via the FBX file format to the GR2 file format. The main problem people have run into is that the Blender FBX export script doesn't like multiple objects or objects such as swords and shields which are not rigged to the skeleton. You can see this mentioned in the documentation - "Do not used skinned meshes as children of other objects (except their own armature) This doesnt work reliably".

The only reliable way I've found to get units across is to combine everything into a single mesh with a single material and texture. Here I explain in gory detail how to do it.

The high level approach in Stage 2 is this:

A) Load the NIF without animations.
B) Join weapons, etc to body. In the case of the Zulu I used the club and shield are already in the right positions relative to the body and skeleton. This means when I Join them (Ctrl-J) to the body they are in the correct position for animation already.
C) Make your single texture using the Bake Texture method.
D) Assign weapons, etc to bone Vertex Groups. This is what is meant by "rigging".
E) Export to NIF and check everything looks OK.
F) Re-import exported NIF with an animation and export as FBX. Repeat for each animation you want to convert.

Note for step B, If you're weapons aren't in the correct position you can use G to Move and R to Rotate them in the 3D View - I find it easier to do this in Edit Mode (TAB).

A) Load the NIF without animations

In the attached ZIP file there is a subfolder nifskope_clean that contains the Zulu Ibutho unit from Total Realism all ready for Stage 2, so you can play along at home!

First, split your Blender window into two with the left hand side showing the 3D View and the right showing the Scripts view. There should be guides around on how to do this if you're stuck.

In the Scripts window go to Scripts > Import > Netimmerse / Gamebryo to import the file ibutho_nifskope_clean.nif from the attached ZIP.



Import using these specified options with the exception of Import Extra Nodes which it is better to leave on.



Note: no animation yet and I don't use Use the Force!

The unit loaded up into Blender:



With focus over the 3D View, press Numeric Keypad 1 to get Front View. Now switch to Texture View as shown.



Slight problem, our texture appears to be black in parts. This is easily fixed. Right click to select the body of the unit. Remove the vertex color by using the X in the location shown here:



You should see the texture appear. Repeat this for the feathers on top of the head - the texture is black anyway so it's hard to see the difference.

Now, at present the unit is using four texture files scout_128.dds, zulu_impi.dds, morningstar.dds and modernmerchant_128.dds. We need to combine these into one so that we can have a single mesh with a single texture.

B) Join weapons, etc to body

Now, we need to join all our separate objects into one.

Right-click to select the Club, hold down Shift and then right click the body object, Ctrl-J to join the two. It is important that you select the Club first to join the Club to body rather than the other way around. Press A to Deselect All when your done.



Now repeat this for Shield, Spear, the Features on the head and the two earring things, joining them one by one to the body.

Once you've got everything in one object, press TAB to enter Edit Mode. Everything should have yellow vertices highlighted including the feathers and earings.



C) Make your single texture using the Bake Texture method

Now, switch the right-hand panel in Blender to show the UV/Image Editor as shown:



Next, click this to create a new UV texture layer. The unit will go white but don't worry.



The new UV texture layer should be set as the Active as shown (the grid icon) the original one should stay as the rendering one (the picture icon).



Still with everything selected in the 3D View press Ctrl-C and then choose the following options:



This copies the UV layout from UVTex to UVTex.001 your new texture layer.

You should see the mess in next screenshot in the UV window. This is the current UV mapping for the unit. It basically tells the graphics engine how to apply a 2D texture to a 3D mesh.



Because we are combining several textures into one, we have the problem that a lot of elements are overlapping. We need to fix this first this before we can bake our texture.

We still have our six separate materials which we can use to select the separate elements of the unit. Switch to the second material (the morningstar one used for the Club) using the small arrows shown below, press A to make sure everything is deselected in the 3D View and then press the select button for that material. Now the UV window shows only the mappings for the Club:

Now, press A in the UV Window to select all then G and move everything to the left of the square like this:



Now, back in the 3D View, deselect all (A) and use the small right arrow to move to the next material which should be the shield.

In the UV Window to select All then G and move everything to the top of the square this time. We are just moving the UV for each element out of the way so there is no overlap and we can see what we're doing.

For the spear (4th material) move to the right of the square in the UV, skip the feather because that's OK where it is. For the earing (6th material) put it beneath the square somewhere.

The UV view should now look something like this with no overlap.



Now we have a nice jigsaw puzzle to fit these elements in back into the UV mapping. Use Lasso select, together with S to scale, R to rotate and G to move selections in the UV window until you've fit everything in. Note: the club mapping that we stuck to the right of the square can be made quite a lot smaller - like to about an eighth of one side of the square. You can use mouse wheel to zoom into the UV view and the middle mouse button drag to pan - just like in the 3D View.

Jigsaw fun...



Here's my completed jigsaw - you'll probably do a better job - it doesn't matter if you've arranged it differently provided there's no overlap and a least a small amount of space between each piece of the mapping.



Now for the cool bit. Select Image > New in the UV window. Set the width and height to both be 256. This will create a plain black image. In the top Blender menu select Render > Bake Render Meshes > TextureOnly. Blender has created our single new texture file for us, based on the UV layout work we just did. How cool is that!



Now go to Image > Save As... in the UV Window and save it as a TGA - we can convert to DDS later.

Next, delete UVTex at the bottom using the X and rename UVTex.001 to UVTex.



D) Assign weapons, etc to bone Vertex Groups (This is what is meant by "rigging")

Now we need to go through the five materials other than the main body, and make sure they are rigged to the unit skeleton.

In the 3D View, deselect all using A and select only the second material again - the Club (the morningstar material).

In Blender you rig parts of a mesh to Bones by assigning vertices to Vertex Groups. In this example, we have the Club selected to rig it to the right hand bone (BIP R Hand) we just select the Vertex Group BIP R Hand and then click Assign (see screenshot below).

Deselect all, go to the material 3 and select. This will select the shield. Switch to Vertex Group BIP L Hand and click Assign. Assign the Spear (material 4) to left hand also, and materials 5 and 6 (the features and earrings) to BIP Head.



Press TAB to enter object mode and delete all but one of the materials.



TAB again to enter Edit Mode. Press A twice so that all vertices are selected and press the Material - Assign (rather than the Vertex Group - Assign we have been usign). This makes sure the whole model is using the same material which we can assign to our new texture.

E) Export to NIF and check everything looks OK

Now, TAB to Object mode and press A until all is selected. You should do this prior to any export.

In the right-hand pane, switch from the UV view to the Scripts view again and export the model as a NIF. I use these options:



If you get a complaint about unweighted vertices you know you haven't rigged a bit of the mesh yet. Blender will helpfully select the part for you so you can just assign it to an appropriate Vertex Group.

Now, use your favourite 2D graphics program to open the new TGA texture we made and save it as a DDS. I used Paint.NET which has built in support for DDS.

Now open up the exported NIF in Nifskope, switch the NiSourceTexture on the unit mesh to the DDS texture you just saved (click where ibutho.dds is in the screenshot below). You should see that we now have the entire unit weapons, feathers, earrings and all as a single mesh. Now save the NIF.



AFTER you've saved the NIF in Nifskope, you can use Spells > Animation > Attach KF to check that the unit is animating OK.



Phew, that's the hard work over, now our NIF is ready to used to make a Civ 5 unit.

F) Re-import exported NIF with an animation and export as FBX

In Blender, import the final exported NIF we've just been looking at in Nifskope, but in addition to the previously used options now select "Import Animation" and select the aztec_jaguar_md_idle.kf animation as the Keyframe file.

You'll probably find that the mesh isn't matching the pose of the skeleton. Easily solved. Select the mesh using right click and click "Add Modifier". Pick the option Armature and then type "Scene Root" next to the "OB:". The model should snap to the skeleton pose.



If you want to check the animation plays OK you can use Ctrl + Left Arrow to go to an Animation layout. Press the play button and you should see the model breathing nice and smoothly.

Now we can export our FBX for import to Civ 5.

In the Scripts window select Scripts > Export > Autodesk FBX. I always pick Scene Objects instead of Selected Objects just to make sure I haven't missed anything - I leave the other options as default.

(At this point if you'd like to rig your unit to use Civilization 5 animations then see this thread.)

Stage 3 - Use NexusBuddy to convert the FBX file into a Civ 5 unit

Now we have an FBX that we can use as a source for NexusBuddy 1.0a.

Notes:
1. Before exporting your files from NexusBuddy set the scale to 1.2 or so. (You can edit the fScale later manually if you want) The scale of Civ4 units is much smaller that Civ5 so your units will be microscopic otherwise.
2. It is a good idea to check the outputted .gr2 file in Granny Viewer just to make sure it has come through OK.

If you follow lemmy101's NexusBuddy guide you should be able to get the converted unit showing up in game, like this:



Stage 4 - Convert other animations

I didn't import multiple animations into NexusBuddy simultaneously. Messing around with frame numbers seemed too much like hard work.

My process was:
1. Create all your animation FBX files, by import the same NIF into Blender with a different animation KF file each time then exporting to FBX. I named the output files, for example, housecarl_idle.fbx, housecarl_run.fbx, housecarl_strikeA.fbx.

2. Use NexusBuddy to make the initial mod files and set up your test mod in ModBuddy with using the idle version, housecarl_idle.fxb in my example. This is Stage 3 above.

3. Delete the export folder from your working directory.

4. Load the next animation, housecarl_run.fbx into NexusBuddy. Set the material and texture stuff and then export.

5. Copy only the outputted animation to another location e.g. make a holding pen folder called animations.

6. Repeat steps 3-5 for each animation you want to convert.

7. Add the animation files to your mod and manually edit the fxsxml file to point to them. See my housecarl unit if you want to see how I set the fxsxml.

It would be nicer if I could just load my FBX files one by one into NexusBuddy without having to worry about the frame numbers. I prefer my method because although it is repetitive, I don't have to think as much which is bonus when converting units.



If you're wondering if this process also works for mounted units - the answer is YES! See here.

.

 

Examples

 

Reserving for more stuff. 2

 

Reserving for more stuff. 3

 

Reserving for more stuff. 4

 

Reserving for more stuff. 5

 

I'm starting to feel overwhelmed by how much there is to learn, and I'm already at the stage of getting some units loaded into the system.

Time to shake off that feeling, and try to follow your guide as opposed to the other guides, and see where it gets me.

Thanks for the hard work.

 

It looks a bit overwhelming, but that's because I'm going into a lot of the detail for newbies. Have a go and hopefully you'll find it's not too bad. The main reason I've made this is because I don't want to be the only one capable of converting stuff!

 

Thank you so much for the detail and thought you put into this. Much appreciated.

 

Forgot the Zip file for the tutorial - it's now attached to the opening post.

 

First, i want to thank you for what you re doing,i really appreciate it

Secondly,i'm stuck at some point:

When you get your UV in the right side panel and your pink mesh in the left one,how do you select the second material and assign it to get to the next step?

You tell to "go to the second material",but in edit mode you can only select the "material indice" and assign it to the selected faces

When i do that and try to do the next step,it doesn't do what you show (actually,nothing changes )


Edit: Got it.
You wrote CTRL+A but i only pressed A key to select all in UV window and it worked

 

Perhaps this helps...



Yeah, the Ctrl-A was an error - i've amended it now.

 

When I try to load the animation in NifSkope I get this error: "The following controlled nodes were not found in the nif:"
"Editable Mesh@#0"

That's supposed to be the shield, which I do see is still there. Is this because I joined them all? I try loading the .kf in blender and I can't apply the modifier(armature) to scene root. It seems to be gone, just a bunch of x's, y's and z's all over the body. When I use the arrow keys to see the animation I just see the letters move around.

I'm going to start from scratch and see if I can pin point my errors.

 

No biggie. Those errors can be ignored in Nifskope. They shouldn't affect the animation if you hit play. Just close the error popup and move along.

It is just because there is stuff referenced in the KF animation that is not in the NIF anymore probably because it is a custom unit.

I have similar errors when viewing animations on my final cleaned up NIF as you can see below, but it ports to Civ 5 just fine.



Edit: Also bear in mind that you might be starting with a very dodgy NIF that needs Stage 1 work that I haven't gone into yet. You mind be better trying with a vanilla Civ 4 unit first because the modded ones can be a real mess structurally.

 

Yeah I think that's my prob. I'll try a vanilla unit. Cheers.

 

Or you could download the zip file for the tutorial and follow it through. Once you've done that you should have the general idea.

 

Well the prob has followed me lol. The letters are there again. This only started happening when I change the import settings. I noticed in your tutorial that you don't have these letters in your pics.

Spoiler :

With that said I think it might be blender and python related. I'm going to reinstall them and try again.

 

I'm not sure the letters are really a problem. Your screenshot probably just has everything selected at once, while mine had a particular object selected.

Does it stop you from proceeding through the tutorial?

Also, you don't need to view animations until you have done most of the work and exported and reimported the unit with the animation included. Most of the Blender work is done without animations loaded.

Ctrl + Right Arrow key just changes the Blender layout to different settings and one of themes is laid out with the various windows useful for animation.

 

There was so much information that I just committed Seppuku...
It was a sweet release