Custom Leaderhead Development Thread

[CaptainBinky]

Yeah, this is a really good start - top stuff Lemmy

Now, if only the bloomin' Granny format wasn't a nightmarishly expensive super-protected format, we would be able to get the files into a 3D package so we could get a decent look at them Oh well, we can at least learn a few bits n bobs from the Viewer but... well, it's all just a bit tantalising now

edit: Weird stuff now. Seems that Lemmy's version of Catherine - the version that plays in Granny viewer - still loads up and plays fine in the Nexus Tool. Completely weird since the way this normally works is 1) Files that work in Granny viewer do not work in Nexus and 2) Files that work in Nexus (i.e. game assets) crash Granny Viewer.

 

[Wirapuru]

There's an easy way to say what in the gr2 header makes the difference? (I know, unfair question

 

[The_J]

I've been working with the Cap'n Binky tonight and I've managed to repair the Nexus 5k animation file bug by decompiling / recompiling one of the DLLs with a few changes.

Depending on how much of this is illegal (is the source of the .dll available? [Note: If i don't know anything AND it's not anywhere here, then i don't care...]) and/or intended you might want to start a thread in the bug section about this.
Would directly move it to the confirmed ones.

 

[Deliverator]

Yeah, this is a really good start - top stuff Lemmy

Now, if only the bloomin' Granny format wasn't a nightmarishly expensive super-protected format, we would be able to get the files into a 3D package so we could get a decent look at them Oh well, we can at least learn a few bits n bobs from the Viewer but... well, it's all just a bit tantalising now

If you can open the processed vanilla files so that they work in GrannyViewer then presumably there's a chance you can use this method to get them into Milkshape3D or 3DSMax. I have 3dsMax7 if you want me to test something.

Anyway, good progress folks.

 

[Wirapuru]

Deliverator, did you get gr2decode's plugins to work with firaxis gr2s? I've tried here (with more than one max version) with no success.

 

[Deliverator]

Deliverator, did you get gr2decode's plugins to work with firaxis gr2s? I've tried here (with more than one max version) with no success.

Not myself, but that was with the untouched vanilla GR2s. It seems that Lemmy101 has managed to convert the vanilla GR2s using his recompiled DLL so that they actually open in GrannyViewer. If it opens in GrannyViewer then I see no reason why GLX Extractor wouldn't be able to grab the data, which is the first step in the gr2decode process. If that was done then perhaps the plugins and the rest of the import process described would work OK.

 

[lemmy101]

Depending on how much of this is illegal (is the source of the .dll available? [Note: If i don't know anything AND it's not anywhere here, then i don't care...]) and/or intended you might want to start a thread in the bug section about this.
Would directly move it to the confirmed ones.

The bug is in the Civ5NexusModes.dll plugin for Nexus. The problem is that it internally names all the anims 'temp', so when it cycles through to determine which anim goes in each of the created anim files, it doesn't find any that match their name and thus outputs an empty shell granny file for each one.

It's debatable whether Firaxis would care really about me poking with it.They've clearly used unobfusciated .NET to ease in people writing plugins for it anyway, and the DLL is useless on its own, but I'm not taking any chances. I hear on the grapevine a fix for this issue may be on the way at some point anyway and since we've not managed to get working models in yet I wouldn't be helping anyone out massively by providing it.

If it opens in GrannyViewer then I see no reason why GLX Extractor wouldn't be able to grab the data, which is the first step in the gr2decode process. If that was done then perhaps the plugins and the rest of the import process described would work OK.

I got an obj file but it was bust and wouldn't go into max (invalid vertex data).

Besides without a skeleton or animations it'd be pretty useless for the immediate thing we're trying to do, as the main reason we wanted to get it into max is so we can see how Catherine's anims are set up, and thus why she gets through the system and Binky's test anim does not.

EDIT: Just spotted your earlier post about the process, didn't realise there was more to it than that. Will let you know.

 

[CaptainBinky]

Okay, the bone animation is playing perfectly in the Nexus Asset Viewer now.

This means:

1) Lemmy's modified *.gr2 exporter is being read by Nexus perfectly
2) The mesh and animation *.gr2 files from Lemmy's exporter display perfectly in Granny Viewer
3) The skin data is somehow nerfed in the Asset Viewer - the mesh is just being transformed by the root bone - like no vertex weights are being applied at all.

edit: slightly more progress now - if you use one of the skinned shaders in the material editor it works slightly better - in that bits of the mesh animate correctly. But since texture loading seems to not work in the material editor or.. in fact... any of the other options, it's a bit difficult trying to figure out how to proceed from here. Ideally there'd be some way to apply a shader that does work (eg Rifleman's) onto these test models, but it appears that the shader data is stored in the *.gr2 files themselves. As it is, we're just dealing with a solid black wonkily animated mesh

 

[CaptainBinky]

Okie, where we're at:

Alright, so the test model has no head and his legs are sucked into a void but... what we're looking at here is a character exported via FBX, sent through Lemmy's exporter, with a shader copied from the Rifleman, animated using the Nexus Animation Trigger app. And although you can't tell from a static screen, although the mesh is a bit wonky it's animating perfectly correctly.

Errors with the skin such as these are typically caused by vertices skinned to too many bones or no bones.

 

[lemmy101]

Okay so I examined the material definition in the Granny Viewer, and it seems that the material stores the number of bones used by the shader, which was taken from the rifleman and therefore wrong. I've edited the file so this is set to 52, the number of bones in Binky's test model, and saved the modified file out.

Binky gonna check it out soon.

EDIT:

 

[lemmy101]

Rightio: Binky's been playing with different skinned models he has on his harddrive, trying to put them into the game, and it seems that when using the UnitShader_Skinned you are completely limited to 32 bones which explains why the test model he was using was nerfing up. And now I think about it I was foolish to think hacking the value would help.

Reason being: In DirectX shaders, you can have an array of matrices for the bones to be used with skinning so that skinning transformations can be accelerated on the GPU instead of done in code and therefore by the CPU. The amount of space you have depends on whether you're using Vertex Shader 1.0, 1.1, 2.0, 3.0 etc.... The unit shader is probably using a lower shader model, with less effects, since there are many of them being drawn on the screen, hence the inherent limit on the number of bones that hacking the value couldn't possibly solve. since the leaders are on the screen alone they have probably been privileged with a higher shader model, and therefore a larger bone matrix array on the GPU.

But there is added complication in using the leader shader, so we'll concentrate on the unit one for now with a lower bone count model.

This all means we should be able to get a skinned model from a game we worked on through the pipeline and into Nexus animating properly. At which point the new challenge comes from trying to apply all the correct textures to the different meshes in the model without using the broken material editor.

Then we'll move onto trying to use the leader shader, which should afford us a higher bone count.

Will keep you all posted.

 

[Ploeperpengel]

I'd already be happy with a working unitproduction pipeline. Allthough 32 Bones is quite restrictive if we want to have some evil flying Dragons who are truly bad ass in the game.
Keep it up guys you're awesome

 

[lemmy101]

The plot thickens. Apparently only the gods can intervene and determine whether a file will go into Nexus okay. Binky tried about 10-15 characters from previous game projects and every one of them was broke in a weird and unique way.

Apart from one. One worked fine.

Confused as to why she worked and others didn't, he tried deleting her one unique thing, a wobbly eye bone and reexported. Broken.

So is it something to do with the eye bone? He exports the original, that worked last time perfectly. Broken.

Gah. ;D

 

[CaptainBinky]

Oh My God

Right, important news here. Make sure you're using the 2010 version of the FBX exporter from whatever 3D software you're using. Not the latest 2011 one. Not 2009. Not 2008 or any other of the earlier ones. Very specifically the 2010 one. Turns out, that was the final piece of the puzzle. Lucked out there with an "I wonder if there's been an updated FBX exporter" moment

Screenshot incoming...

edit: TADA!!!

If this was an animated screenshot, you'd see that he was merrily running along nicely

(I'm using a different test model here because this one has less than 32 bones, since I'm still using the skinned unit shader)

 

[The_J]

lolwut?
er...i mean great to hear that actually something works .
After the next weekend i want to see the first custom 3D leader .

 

[lemmy101]

This may be as far as we get until the material editor is fixed as setting up multiple materials for leaderheads via hacking the gr2 directly will be an extremely complicated task compared to simply hacking in a single material. Sadly the Viewer.exe is in a native exe so there's little pokery I can do to repair it, so it's likely a case of waiting. It does however mean we're armed with quite a bit of info on how the process works so will make it easier for when these issues are fixed.

 

[Ekmek]

This may be as far as we get until the material editor is fixed as setting up multiple materials for leaderheads via hacking the gr2 directly will be an extremely complicated task compared to simply hacking in a single material. Sadly the Viewer.exe is in a native exe so there's little pokery I can do to repair it, so it's likely a case of waiting. It does however mean we're armed with quite a bit of info on how the process works so will make it easier for when these issues are fixed.

multiple materials? you mean like if the skin and clothing comes from elsewhere?

EDIT: Also, I'm 99.9% the FBX was based off the one inside 2.49b blender. Why? thats a long story.

 

[CaptainBinky]

multiple materials? you mean like if the skin and clothing comes from elsewhere?

As in different shaders applied to different parts of the mesh - skin shader, fur shader, clothing shader, etc. It's possible they physically split the mesh up so that there's 1 shader per mesh, but even so there'd still be multiple shaders in one file.

Also, I'm 99.9% the FBX was based off the one inside 2.49b blender. Why? thats a long story.

? What do you mean "based off"?

The source tool for, for example, russian_catherine.gr2 is "3D Studio MAX 12.0 (64bit)", which is Max 2010 and the FBX export plugin supports multiple version formats.

 

[Ekmek]

As in different shaders applied to different parts of the mesh - skin shader, fur shader, clothing shader, etc. It's possible they physically split the mesh up so that there's 1 shader per mesh, but even so there'd still be multiple shaders in one file.

I notice that if you select different shaders it gives you more options on materials.

? What do you mean "based off"?

The source tool for, for example, russian_catherine.gr2 is "3D Studio MAX 12.0 (64bit)", which is Max 2010 and the FBX export plugin supports multiple version formats.

When they were developing the FBX import they used an FBX I generated off of blender. AFAIK blender never updates the FBX so it might be different then the ones you are using for max.

 

[CaptainBinky]

When they were developing the FBX import they used an FBX I generated off of blender. AFAIK blender never updates the FBX so it might be different then the ones you are using for max.

/facepalm

Would have been nice to know that at the start, to be honest.

I feel like I've just wasted a sizeable chunk of my life.

You might want to write Ekmek a pm to point him to this thread. AFAIK he's already trying to set up a civ5 leaderhead for a while and seems to have some insider info.

Anything else that we might need to know?