Custom Leaderhead Development Thread

[CaptainBinky]

Note: If possible, could we keep discussion here constrained to the *development* of leaders as opposed to which ones you'd like to be added..?​

CONTENTS

Latest Developments
Leader Specification
FBX Exporting
Nexus Art Tool
Useful Links
Contributors

Getting new leader models into the game follows the same principles as new units since they work in very similar ways. So initially, we've been focusing on simple units before moving on to leaders.

Please see this thread for information on how to get a new unit into the game (as per the screenshot below).


LATEST DEVELOPMENTS

Custom leaders modded in-game successfully!!!
New custom units now modded into the game successfully!!!
Material Editor non-functional -- FIXED (use Lemmy's material tool, see here)
5Kb null animations on export -- FIXED
Exported animations not playing in Nexus viewer -- FIXED
No skin deforming with bones -- FIXED (ensure you pick a skinned shader in the Material Editor, max of 32 bones if using the unit shader)

Alfred the Great, in-game, work-in-progress.
Some mesh errors need repairing and normal maps need adding.


Test model imported into the Asset Viewer, animating correctly


And here it is in-game!​

As an interesting side-effect of making the code changes required to fix the 5Kb error, Lemmy has managed to get the game assets viewable in the Granny Viewer (previously, these assets would crash the viewer). More info here.

(if anyone is interested in progress of my own leader, screens are here. Wirapuru's WIP is here - click the spoiler to show.)

---------------------------​

Leader Specifications:

• Poly Count
  
  Approx 10,000 triangles (unknown what LOD specs for lower graphics settings are at this stage)​
• Textures
  
  Multiple textures per leader but primarily, diffuse map (512x512) and normal map (2048x2048)*
  Texture name conventions are as follows:
  

  texturename_diff = Diffuse (colour) map
  texturename_norm = Normal map
  texturename_sref = Specular reflection map​

  
  The texture name should match the object it is applied to. For example, I have a leader comprised of the meshes "arthur_skin", "arthur_costume" and "arthur_hat". Texture names would be "arthur_skin_diff.dds", "arthur_skin_norm.dds", "arthur_hat_diff", etc.​
• Materials
  
  One shader per mesh, multiple meshes per character. In other words, a shiny reflective hat would need to be a separate mesh to a furry collar.​
• Facial Animation
  
  20 to 30 facial bones. Very similar in set-up to Civ IV leaderheads​
• Skeleton
  
  It doesn't look like the skeleton setup is too important however, as with units, ensure that your skeleton is parented to a root bone at the origin
  In 3DS Max, create a 'dummy' object, set its position to x:0 y:0 z:0 and then link the root Biped bone (normally called 'Bip01') to this object. Make sure you add this dummy object to your skin​

*note: This is based off image file size so may not be completely accurate


FBX Format Export:

3DSMax

Biped is supported through the FBX export / import process, however, the Biped Mixer is not. So you must "Compute Mixdown" and then "Copy Mixdown to Biped" before exporting animations.

Ensure you export using the *2010* version of the FBX exporter. All other versions yield errors in tests.​

Blender

Blender 2.49b was used to generate Firaxis' test FBX file. Other versions untested.​

Nexus Art Tool:

Unpacking the PAK (*.fpk) files

Load the Nexus Art Tool, select the "Nexus" tab, click "Script Browser", select "Extract PAK (*.fpk) files", click "Run Script", and... wait.​

Materials:

The material editor in the Nexus Art tool has bugs preventing it from being terribly useful at the moment. Instead, use Lemmy's Material Editor (see here) to transfer a material from a Firaxis model and modify it.

With this tool, it's also possible to make re-skins of existing units.​

Useful Links:

Granny Viewer - Free viewer for checking *.gr2 files outputted by the Nexus Art Tool.
Nexus Tools Wiki - Currently a bit pointless since it's basically just a ripped off old version of this post
First Look at Nexus - Deliverator's experiments with unit reskinning
Jaroth's Civ 5 Leader Info spreadsheet


Contributors:

CaptainBinky, Lemmy101, Wirapuru, Ekmek

 

[Wirapuru]

The first issue is alternate leaders for civs. Obviously, having two English or two French civs is possible but it would be ideal if there was a small mod which could parse the civ names and check that only one of each alternate was in the game, so you don't get English Victoria vs English Elizabeth, for example.[/I]

Just to keep it going about this one, my main doubt is if this is feasible with the current Lua-based SDK. If I can speculate, it would involve messing with how the game session is created, after the setup. It can be done?

Please note that, while having another UI and structure to allow the player choose between more than one leader per civ (like previous games) would be ideal (and, I suppose, harder to mod) this proposal is about to creating a clone of an existing one and assigning it another leader. This new civ would be named like ENGLAND_LH_{custom_name} (in example) and some script would make the math so two 'ENGLAND{rest of identifier}' wouldn't be in the same game.

 

[CaptainBinky]

To sidetrack briefly... Hooray, I have a test skinned character imported into Nexus and this time, it's not broken horribly. Not entirely sure what I've done differently to the source file this time... will try and figure out. Animations not playing yet - the Animation Manager claims to autoplay them in the 3D view, so possibly they're not importing correctly...

 

[Louis XXIV]

BTW, I'm just going to say that I'll be following this thread closely. I have no real artistic talent, but having people who can make quality leaders excites me because it means mods can be one step closer to a professional feel.

So is your priority (after you figure stuff out) to focus on making some alternate leaders for current Civs or to expand into knew Civs?

 

[cool3a2]

You might want to write Ekmek a pm to point him to this thread. AFAIK he's already trying to set up a civ5 leaderhead for a while and seems to have some insider info. At least, he might be interested in this thread.

 

[CaptainBinky]

Hi Louis, for me... well sort of a bit of both, we'll see. Once I've got a custom mesh importing correctly and I know what the process is, I'll have a handle on how long these things will take to make and therefore maybe do a couple of them... we'll see

@cool3a2 - thanks! Have PM'ed.

 

[Ekmek]

Well the animation probably hasn't been attached and you have to set that up with te animation editor. Then there is the state graph that you have to work with.

I was told by Firaxis that I should first start with a tile improvement then a unit then a leader. So far I've imported a building and tried both as a TI and as a Wonder but it won't display in game. Kael and I are both talking to Firaxis about what went wrong. THe good news is that there will likely be patches for the mods tools.

BTW, what program did you use to create your mesh?

 

[CaptainBinky]

Hi Ekmek. The test model I used was made in 3DSMax, using Biped bones. I tried biped just on the off-chance figuring that it probably wouldn't work so next I'll do a test using standard Max bones.

Surprisingly, however, the bones imported with the mesh just fine, all displaying correctly in the 3D viewer. The animation though, no luck. The test anim was 1000 frames though which might have been a tad excessive, but it was just one that I had sitting on the model from some previous work.

When I add a new anim in the animation manager, it defaults to an end frame of 515. Which is slightly odd - not really the sort of value that you'd have for a maximum frame number limit. Another sign that all is not well with the anim data, I guess

But yeah, if I had to hazard a guess I'd say biped is the problem because, well, it nearly always is

(The model, by the way, imports posed as it is on the first frame of the anim. So the skin and bone data is either importing correctly and the anims not, or the pose is collapsed into the model at some point during the export/import process)

edit: Okay, I've just done a test using standard Max bones, and it produces exactly the same results. Does anyone know if the animations actually play in the Animation Manager during import (like it implies)? Or do you have to set it all up in the Asset Viewer and Anim Graph / State Graphs in order to see?

 

[Wirapuru]

Oh well, I'm seeing an everest to climb here but optimism all the way

I'm using xsi and I just got my dummy model to Nexus, at least not all messed up like maybe was yours before CB (you know, scrambled eggs, here too).

I had (obviously) to freeze all transformations and plot all the animation to at least it be imported and show itself static on Nexus. After finally at this stage (which I have been trying since yesterday) I went to see where all the animation is handled - the Animation Manager - and I got a bit confused.. what can we do there? I did a simple 50f animation to test and nothing referencing the plotted animated frames shows up. Seems like we have to import the plotted animated frames as xml? I tried to guess everything there but no clue so far..

Hi Ekmek =] nice to know all that - gives hopes.. besides the updates, do you think they can release any kind of documentation or specifications about formats, etc in some time?

I'm feeling (I guess everyone) a bit lost with Nexus, how things must be prepared to be imported, and then manipulated through Animation Manager and Material Editor? Can you provide any info on that?

I guess I reached my 50th try here (CaptainBink must have around that too =P), I'm trying to accomplish things I've seen before on another games engines (I'm not that experienced but have done some mods here and there) but really, I'm feeling with no luck at all.

I'm willing to take that learning path you mentioned, tile >> unit >> leader, but even there I don't think I'll too far without a bit of info on how to do it.

edit:

edit: Okay, I've just done a test using standard Max bones, and it produces exactly the same results. Does anyone know if the animations actually play in the Animation Manager during import (like it implies)? Or do you have to set it all up in the Asset Viewer and Anim Graph / State Graphs in order to see?

I didn't see that 'xD totally second this question.

 

[CaptainBinky]

I would have thought the anims ought to come in along with the FBX - it would seem sensible that all the animations would be in one long timeline and that the purpose of the Animation Manager is specifying the start and end frames etc of the individual animations within the timeline so that they can be spat out as individual files during export.

I may be wrong about that, but I'd be very suprised if we're supposed to bring animations in as XML.

 

[Wirapuru]

I would have thought the anims ought to come in along with the FBX - it would seem sensible that all the animations would be in one long timeline and that the purpose of the Animation Manager is specifying the start and end frames etc of the individual animations within the timeline so that they can be spat out as individual files during export.

I may be wrong about that, but I'd be very suprised if we're supposed to bring animations in as XML.

I think it's weird too, since FBX files are completely able to handle skinning and animation (if I'm not mistaken this was one of the main functions of the format - it was mainly used for motion capture in the past). What made me think that was:

Spoiler :

That's the only feature I found that seemed a bit coherent in all this.

edit: I would be very pleased if we had some example files to learn from... unfortunately all civ 5 are in a very proprietary format, no way to see how they were made =/

 

[Abremms]

something of note that I've been thinking in regards to leaderheads is that the current stock leaderheads work before they are animated well enough that you can tell how a leader feels about you through facial expression and body language. if we use static leader heads, or custom heads, it could be alot harder to get a read on them, for that we would need a way to make the relationship modifiers visible like in IV.

 

[Ekmek]

Animations seem to be a big challenge. If you open up a civ5 model you'll see in the state graph that there are trigger numbers for activating it. I assume you might have issues with it.

On another note the animations might not play on the initial import but after you export it to the gr2 and reopen it -it might. I'm only guessing here but it based on my model not showing the materials that I assigned but once I export it to a gr2 and then viewed the fxsxml the materials were there (though a bit messed up )

But really lets just try to get a nonanimated model in first. mine just do not show in game which is the hold up. But optimistically we are still way ahead on modeling where Civ4 was at this point.

I am trying to get Firaxis to let us view the animation and material editor when look at a civ5 model so we can atleast have some values to compare it to.

 

[CaptainBinky]

I agree, baby steps. At the moment, though, I'm focussed on familiarising myself with tool - I'm much less concerned about actually getting it in-game. If I can get a skinned untextured model displaying in the tool like I can load the Riflemen and other units, that'll do me for now.

The actual state stuff looks fairly... well, "simple" is the wrong word... but it seems do-able by studying the game assets. Issues like animation just not seemingly loading at all, or exported files not then importing seems to be the stumbling block for the time being.

I'm not entirely sure I have my tool set up correctly, since if I try and load a model that I've exported, I get "missing file xxxx" errors (when I validate) for the animation and model files even though the file exists in the same place as the assets which load fine (and the model displays fine albeit unlit and unanimated in the viewer). There's something else going on here, for me.

 

[Wirapuru]

But really lets just try to get a nonanimated model in first. mine just do not show in game which is the hold up. But optimistically we are still way ahead on modeling where Civ4 was at this point.

That's ok for me, I guess it's a step bigger than my leg to try to create an animated leader right now in this stage. I'll try to create something like you did, maybe to make a cube to show up as a wonder ingame - it would be some progress.

Do you mind to share at least a bit of how was the process you tried to create this building/wonder? Right now I'm thinking it would be import FBX then exposrt as civ5 format. Then edit some XML file to reference that new resource, would be it?

 

[CaptainBinky]

Incidentally, Wirapuru, if you open the XML files exported from the Animation Manager, you get things like this:

<AnimationDefinitions>
<AnimationDef name="NewAnimation0001" ec="1001" start="0" end="515" settings="Default" />
</AnimationDefinitions>

So that "import" and "export" is just for animation references as opposed to the keyframe data itself. I think we can safely say that the physical animation data is definitely contained within the FBX format (which, by the way, supports morph as well as bone data - it's possible morphs are used for the facial animation as opposed to facial bones)

 

[Wirapuru]

Ow.. nice catch on that, one less thing to care about =) Did you see any clue of any animation information from your FBX on Nexus anywhere yet?

 

[CaptainBinky]

Well, here's what's interesting... During the import process, the progress bar has a little pause when it says "Animation", and beneath you can see it say "Sampling Bones" and "Compressing Bones".

This is exactly what you'd expect - probably taking the spline keyframe, sampling it at a keyframe per frame, and then compressing the data. It looks like it's definitely reading in the data.

HOWEVER... the animation.gr2 file that gets spat out is 5Kb for that 1000 frame animation, which contains some pretty complex (i.e. difficult to compress small) motion. So that's alarm bells right there.

 

[Wirapuru]

My xxxxx_animation.gr2 is 5kb too, with the simplest 50f animation ever - so your bell is calibrated =/

The main .gr2 doesn't seem to store anything either, since exporting the same model with 50f and 1000f didn't make a explicit difference (1kb more, from 314kb. A diff on notepad++ give two almost totally different files, but who knows what Nexus do, maybe it's all hashed)

So far I'm speculating that:

1. Nexus isn't importing directly any animation from FBX (hoping not); or
2. It is importing but we have to assign something through those screens; or
3. We have to export the keys from 3d soft and import them someway in Nexus.

or maybe it's nothing like that at all, eh.

 

[Wirapuru]

More information revealed.. seems like Nexus imports animation but we have to, somehow, reassign it on the tool. I say this because I have downloaded the Granny Viewer ( get it here ) and if you open the exporter.temp.gr2 (probably the filet Nexus generates after importing the FBX) you'll have the animation there, you can even play it without problems.

But the final .gr2 generated by Nexus doesn't have any animation, and, on my case with 1000f, only 25k less. I can't do any more investigation right now but when I can I'll try to dig this further.