I'm working on a big mod rightnow. If you look at UNITS.xml . Route to the file is "(Drive installed)\Games\Civilization6\steamapps\common\Sid Meier's Civilization IV\Base\Assets\Gameplay\Data"
Did you see highlighted area? That is. right click and hit 'open with' and pick text editing programs to view the file (Notepad++ is recommended, but use normal Notepad if not available)
This is what's inside the file. Find the syntax that has <UnitAiTypes> and <UnitAiInfos>. The entire section should look like this
This is where to tell AI to use your custom unit. but if you made independent unit mod similar to what I did before (and a big mod is to bring them all together) This is not neccessary. My experience is that 'be patient'. but first look at your unit profile and how you define gameplay characteristics and assign classes (PromotionClass tag is where unit class is generally assigned. This is where normal melee and anti-cavalry units are defined), then prerequisites ((PrereqTech or PrereqCivic, and StrategicResource), then mandatory obsolete (conditions that this unit will no longer be able to made from that point on; MandatoryObsoleteTech or MandatoryObsoleteCivic ), and so fourth.) Once these codings are revised, edited, and properly checked (Hit the 'clean solution' and 'build solution' respectively first to finalize the mod project or these will not take effect). Playtest it and.. be patient, it takes time for an AI to use custom units. well it needs about 10-30 turns to see them recruited the first of such.
oh! EDIT ONLY associated files of your project. NOT the core game ones!!!!!!!
EDIT If you feel like you must add these codes to have AI use such units in the way you intended them to do. First you must be aware what class the unit is. then copy codes mentioned here and adapt with your unit gameplay XML file.