The only problem with the lancer is that it lasts without a str upgrade long after almost every other unit is almost double its strength. It's good against any knight unit (other than Stampy) and is even decent against cavalry (because of cavalry's penalty v. mounted) and has str equal to musketmen so it fills its combined role as Renaissance mounted/anticavalry pretty well. After the last cavalry disappears from the field the only thing a lancer can attack without it being a suicide mission is artillery.
This means it sits around anywhere from 15-30 turns(depending on your tech path and your enemy's tech path) a completely impotent unit sucking up maintenance. I don't care anything whatsoever about historical accuracy (if I did, playing CiV as long as I have would have given me an aneurism quite a while ago) I care about gameplay. This problem could be made a lot less irritating by simply adding an upgrade in the modern era. An airship/zeppelin at flight, some kind of truck mounted heavy caliber gun at combustion, really anything that could logically counter the landship. Roles or whatever else aside, I'd just like a useful unit in that era that can survive attacking something, anything at all.
That's all it would take to make me happy with 5. Maybe in 6 they could hash out more defined unit paths like Kasper was talking about. I'd just be happy with the simple fix of adding a unit between metallurgy and combined arms. That's just waaaaaay to far apart and I can't believe nothing was done about it in BNW. Instead they wasted time and resources on that shameless plug, the XCOM, which will rarely ever even be used because of how late it comes.