Getting Civ 4 Units into Civ 5 - Full Conversion

[Deliverator]

There was also overlap, where some vertices belonged to 2 groups - and the distortion seems to begin at exactly those vertices that were assigned to the particular problematic bone(s) and no further.

I got to pretty much the same point as you by playing with your Centaur files - it is a good simple example of the problem. It does seem to be do with overlapping vertex groups - but it must be a particular scenario where they cause a issue as some models with overlapping groups work fine.

In the Granny format every vertex can be weighted to up to four bones where the total weight should always 255. The BR2 export script calculates these weightings. I can alter the script so that all the weighting goes to a single bone and see what that does.

Edit: I don't think it can be do with "problem bones" - a bone doesn't really have any properties other than a position, a vector direction and hierarchical connections to other bones. I think it's more like to be the pattern of overlap in the vertex group mappings. Could be wrong of course.

 

[Civitar]

I had a go at the Dragon and got, as Nomad eloquently put it, a soup sandwich. I think this is because I didn't use a "cleaned" NIF, instead I just used the original NIF. I can't seem to export NIFs properly, and it's really bugging me. Also, when I got the first soup snack I duplicated it, exported both to NB2, joined one's horns to its body, then bone weight copied from the one with horns detached to the one with a single mesh, and this seemed to help a bit.

 

[Deliverator]

Further on the Centaur. If you delete everything EXCEPT the arms from the mesh then the arms render fine in Nexus Viewer, but then you have drastically reduced the complexity of the unit. So there's something else going on here... will keep digging.

 

[Nomad or What]

Further on the Centaur. If you delete everything EXCEPT the arms from the mesh then the arms render fine in Nexus Viewer, but then you have drastically reduced the complexity of the unit. So there's something else going on here... will keep digging.

Thanks Deliverator! I am just posting what I encounter so that maybe it can lead you to the solution.

I do agree with you about complexity. For instance, to cite Civitar's issue with the accursed Red Dragon (he must still be pissed at me for subduing him with my ranger that one time in Fall from Heaven...but I digress ), I also encountered problems when troubleshooting it. The initial problem was in one of the wings, so I deleted the wing. That only shifted it to another "bone" so I deleted that part of the mesh - and so on. Mind you that the dragon is extremely complex (he prefers humans seared to delicate perfection for example - oh, and he also has a LOT of vertices!) so I can understand how the complexity of the export increases the probability of corruption. What is curious to me is how the corruption can be consistent based on the input - and my only guess is that the .br2 file "stabilizes" the .fbx export to some degree. You would know the answer to that question much better than I.

You have done a tremendous amount to get us all to where we are today when it comes to importing graphics and animations - I am very confident that the solution will be discovered in a matter of time. Keep up the good work!

 

[Deliverator]

BREAKTHROUGH!

I'm pretty sure I know what the issue is now.

Key Point: The Granny format has a limit of 32 bone bindings (what Blender calls vertex groups) per mesh.This means that if you have a model with more than 32 bones in it you need to split your mesh into separate chunks each of which have no more than 32 bone assignments.

This makes sense of something I've always found puzzling with the vanilla models. Why, for example, are the War Chariots split into weird chunks where the back section of one of the horses is a separate mesh? Now, we know - its because Granny requires each mesh to have no more than 32 bone assignments. Presumably this helps the efficiency of the graphics engine.

Anyway, this discovery is VERY GOOD NEWS because it is very simple to take your mesh and lasso select chunks of it in Blender in Edit mode (Ctrl-LMB drag) and then press P to separate the selected chunk into a new mesh before exporting to BR2. The aim is make sure no mesh has more than 32 bone assignments (vertex groups).

I tested this by dividing the Centaur into 5 chunks (a bit excessive but I was testing the theory!) and now because none of the meshes now has more than 32 bone assignments the model works fine in game with no glitches! Happy days.

Thanks Nomad for the Centaur example. That really helped me notice that something was happening around the 32 bone mark.

I'm fairly confident this will resolve the issues with the Sandworm, Kraken, etc and other units that I gave up on due this issue. Let's hope so.

 

[Nomad or What]

Deliverator, please see my post here...

Well, In essence it says, "AWESOME, YOU ROCK!!!"

Prepare to be bombarded with fantasy units!!!

 

[Deliverator]

1) The objects imported from the original .nif file were so small in scale that they did not appear in Nexus or in-game after the conversion, even though seemingly appearing fine in Granny. I didn't realize this until I saw a teeny pink blip in Nexus while moving the cursor around in frustration. I should have realized it earlier because there was no grid while displaying the unit in the Granny Viewer. I had to scale the unit 200x larger than its original size. Unfortunately, after doing so the animations no longer worked and creating a new clean.nif file after re-scaling caused the imported animations to be deformed. So, I had to use the original clean.nif file, import each animation, scale it 200x and then export to .fbx. This was a bit tedious, but successful.

You know you can just scale things up in the ArtDefines XML/SQL? e.g.

	<ArtDefine_UnitMemberInfos>
		<Row>
			<Type>ART_DEF_UNIT_MEMBER_TEST_UNIT</Type>
			<Scale>100</Scale>
			<Model>test.fxsxml</Model>
			<MaterialTypeTag>CLOTH</MaterialTypeTag>
			<MaterialTypeSoundOverrideTag>FLESH</MaterialTypeSoundOverrideTag>
		</Row>
	</ArtDefine_UnitMemberInfos>

Well, In essence it says, "AWESOME, YOU ROCK!!!"

Prepare to be bombarded with fantasy units!!!

Thanks. I look forward to seeing lots more units! I'm pleased that the tools and the process seem to have finally got a stage where a few more people can grasp it.

 

[Nomad or What]

You know you can just scale things up in the ArtDefines XML/SQL? e.g.

	<ArtDefine_UnitMemberInfos>
		<Row>
			<Type>ART_DEF_UNIT_MEMBER_TEST_UNIT</Type>
			<Scale>100</Scale>
			<Model>test.fxsxml</Model>
			<MaterialTypeTag>CLOTH</MaterialTypeTag>
			<MaterialTypeSoundOverrideTag>FLESH</MaterialTypeSoundOverrideTag>
		</Row>
	</ArtDefine_UnitMemberInfos>

I didn't know that and it is very useful information. In the case of the scorpions, however, they appeared so minute in Nexus that I didn't even see them at first - so I had no idea how much to scale it until compared with one of your .blend templates. Now that I am aware of the potential pitfall I can avoid it in the future and simply compensate for it by setting the parameter in the ArtDefines per the above example.

Thanks again!

EDIT: Actually I was aware of that setting in ArtDefines; I thought for a moment that you were describing a way to define it in another location. One thing I have discovered, though, is that when you adjust the scale or the unit to compensate for its smaller size upon importing what you also do is scale the fx_triggers event effects as well. So, in the case of the centaur it is about 1/10 the size that it should be and a scale of about 1.1 is required to make it the appropriate size in game. However, when attacking a city the flaming arrow effect is also about 10x larger (which make for some pretty hefty arrows!). Do you know of any way to set the fx_triggers effects scale to offset this?
:END EDIT

 

[Deliverator]

I am trying to import Deliverator's Sandworm by doing a full conversion.... The image produces numerous stray mesh vectors in-game although it appears beautifully in Nexus and Granny:

Hopefully the cause of this can be identified and this unit will be ready to go!
:END EDIT

Responding here rather than continuing to hijack the Dragon thread.

It seems that having vertices where the total bone weights do not add up to 255 was the cause of this issue. I updated the BR2 export to enforce this and the model works OK in game with no stray meshes. The side effect is that the rigging is no longer as smooth due to the fact that a lot of the bone mappings have to be discarded to stick to the limit of four per vertex. I'm going to see if I can mitigate this a bit before I release the fixed script.

 

[Ekmek]

Responding here rather than continuing to hijack the Dragon thread.

It seems that having vertices where the total bone weights do not add up to 255 was the cause of this issue. I updated the BR2 export to enforce this and the model works OK in game with no stray meshes. The side effect is that the rigging is no longer as smooth due to the fact that a lot of the bone mappings have to be discarded to stick to the limit of four per vertex. I'm going to see if I can mitigate this a bit before I release the fixed script.

I would have never thought of that. I hope its possible to only enforce this IF bone weights dont add to 255 and leave it as is for everything else.

 

[Wodhann]

A script for that would be really nice. I actually got (and solved myself) that problem before, but didn't see nomad's appeal.

If you're going to fix your script around that, maybe you can have an algorithm that balances out each weight so that it is proportionately filled to add up to 255? That would be really handy.

 

[Nomad or What]

Responding here rather than continuing to hijack the Dragon thread.

It seems that having vertices where the total bone weights do not add up to 255 was the cause of this issue. I updated the BR2 export to enforce this and the model works OK in game with no stray meshes. The side effect is that the rigging is no longer as smooth due to the fact that a lot of the bone mappings have to be discarded to stick to the limit of four per vertex. I'm going to see if I can mitigate this a bit before I release the fixed script.

Spoiler :

Small sacrifice, I think; Other than very specialized units like the sandworm I can't see this ever needing to happen. Even at large scale it should still look relatively decent in-game using only four per vertex. At least for me it is better than not having a sandworm at all .

Good job solving yet another puzzle - you are on a roll this week!

 

[acehart]

someone can helpme in convert all helicopters from civ4 to civ5? for my mods (helicopter mod)?...

 

[Nutty]

acehart:
Just to be sure, you have you seen all of the helicopters that have already been converted/reskinned in the units thread?
copter crazy, Mil Mi-8 (10 skins), AH-64 Apache (6 skins), and the reskin in my signature...

 

[Deliverator]

I would have never thought of that. I hope its possible to only enforce this IF bone weights dont add to 255 and leave it as is for everything else.

If there are no vertices that are weighted to more than 4 bones then the export behaviour will be the same as before. So no worries there.

A script for that would be really nice. I actually got (and solved myself) that problem before, but didn't see nomad's appeal.

If you're going to fix your script around that, maybe you can have an algorithm that balances out each weight so that it is proportionately filled to add up to 255? That would be really handy.

That is exactly what I've done, but first I sort the bone mappings for each vertex by weight, highest to lowest and select only the first four highest weighted bones. Although some bone weighting still has to be discarded this seems give a much smoother result for the Sandworm example at least.

I'm pleased with this fix because it has also resolved the stray vector issue for this Kohan II Fire Lizard assembly I previously abandoned. So I'm now not aware of any instances of this problem that aren't accounted for by this fix to the BR2 export or the previous discovery about more than 32 bone mappings per mesh.

Get the latest version of my Blender scripts here.

 

[Deliverator]

@Nomad: The Sandworm in the CFC database isn't the finalised version of the Sandworm that we had in Dune Wars. The texture was improved and there we a few tweaks to the UV mappings. I've quickly converted the later version from Dune Wars and combined into your mod which anyone is welcome to use if they want.

 

[Civitar]

Now that looks like an Arrakis sandworm! I always wondered what happened to the plates in his skin.

 

[Nomad or What]

@Nomad: The Sandworm in the CFC database isn't the finalised version of the Sandworm that we had in Dune Wars. The texture was improved and there we a few tweaks to the UV mappings. I've quickly converted the later version from Dune Wars and combined into your mod which anyone is welcome to use if they want.

Spoiler :

AWESOME! No worries on whatever modifications you wanted to make; I was trying to convert your work anyway.

Glad you took care of the Fire Lizard as well; that was another one I was having issues with

A win-win for everyone, methinks

 

[bane_]

I'm trying to get Kohan II units (from your file with all units) to Civ5, but the animations do not work: I always end up with a freezed mammut!
I tried the zeroing the rotation and translation as per the Kohan -> cIV tutorial, but did not work as well, the damn mammut doesn't want to exercise.

Nomad or What gave me some good tips in another problem I was having regarding multiple meshes with no bone assignment, but now I encountered another problem.


EDIT:
I understand this may be too little information, but please someone tell me what could cause this issue:

Spoiler :

FBX export starting... E:\Civ Modding\Bowman\bowman.fbx
Found WORLD node: "Bow01" exporting...
Traceback (most recent call last):
File "H:\Blender Foundation\Blender 249b\.blender\scripts\export_fbx.py", line
2969, in fbx_ui_write
GLOBALS['BATCH_OWN_DIR'].val,\
File "H:\Blender Foundation\Blender 249b\.blender\scripts\export_fbx.py", line
2083, in write
ob_arms = my_arm = my_object_generic(ob)
File "H:\Blender Foundation\Blender 249b\.blender\scripts\export_fbx.py", line
533, in __init__
for item in armOb.getAllProperties():
AttributeError: 'NoneType' object has no attribute 'getAllProperties'

This is from another source, but I did not want to double-post.

 

[bane_]

I can't get the Nexus to work. I followed the instructions in the first page (from Nutty and Deliverator) but I'm still stuck. I either get 'The Viewer application could not attac to Nexus' even with Nexus open; 'The Viewer has closed unexpectedly. Would you like to restart?'; or I get this report when I try to go ahead and open the .fxsxml anyway:

Spoiler :

Application: Nexus
User: Unkown (Not an in-house user.)
Time: Sunday 20/07/2014 02:12:32

Runtime: 64 bit
OS: Windows 2008 (Service Pack 1)
Machine: MAINROOM3

Exception: System.Xml.XmlException
Source: System.Xml
Thread: Main Thread
Description: There are multiple root elements. Line 2, position 2.

Stack Trace:
at System.Xml.XmlTextReaderImpl.Throw(Exception e)
at System.Xml.XmlTextReaderImpl.ParseDocumentContent()
at System.Xml.XmlLoader.LoadNode(Boolean skipOverWhitespace)
at System.Xml.XmlLoader.LoadDocSequence(XmlDocument parentDoc)
at System.Xml.XmlDocument.Load(XmlReader reader)
at System.Xml.XmlDocument.Load(String filename)
at Firaxis.Framework.XmlDoc..ctor(String fileName)
at Firaxis.Framework.XmlDoc.LoadFromVirtualSpace(String file)
at Civ5NexusModes.AssetViewer.AssetViewForm.LoadAsset(String pending_asset)
at Civ5NexusModes.AssetViewer.AssetViewForm.set_Asset(String value)
at Firaxis.Framework.ToolStripOpenButton.ToolStripOpenButton_ButtonClick(Object sender, EventArgs e)
at System.Windows.Forms.ToolStripSplitButton.OnButtonClick(EventArgs e)
at System.Windows.Forms.ToolStripSplitButton.OnMouseUp(MouseEventArgs e)
at Firaxis.Framework.ToolStripOpenButton.OnMouseUp(MouseEventArgs e)
at System.Windows.Forms.ToolStripItem.HandleMouseUp(MouseEventArgs e)
at System.Windows.Forms.ToolStrip.OnMouseUp(MouseEventArgs mea)
at System.Windows.Forms.Control.WmMouseUp(Message& m, MouseButtons button, Int32 clicks)
at System.Windows.Forms.Control.WndProc(Message& m)
at System.Windows.Forms.ToolStrip.WndProc(Message& m)
at System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m)
at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)

When I click the Nexus I get Steam running and it offers me to start Civ5 as well. If I do click in one of the two DX options I get Steam trying to run Civ5 AGAIN and it gives me an error telling me how stupid I am to try to get a game running that is already running.

After a while I actually got the Nexus to run the 3D Viewer alone (it was usually greyed-out), but when I go to the Asset Viewer it gives me the aforementioned report 'Thanks for Playing' and closes it all.