Mounted troops also couldn't hold a position unless they fought dismounted. You couldn't tell a unit of horse to go to that hill and hold it against attack and expect them to do so. It robs them of any combat advantage they might have until you get to WWII era tanks which is due to the fact that they have guns and thus can be entrenched to a degree. Light and Heavy horse shouldn't get defensive bonuses.
With two notable exceptions, which Civ has never accurately modeled.
First, starting in the 17th century (specifically, Count von Mansfeld organized the first unit in 1624) a new 'class' of Light Cavalry appeared in Europe: Dragoons. They were basically mounted infantry: use horses to get into battle quickly, dismount and fight on foot with muskets. They were frequently used to 'hold' special points ahead of the army, like bridges or defiles, or to cover the flanks of a defense or advance, so they relied on precisely those 'terrain advantages' that other Cavalry could not use. They also tended to be mounted on whatever cheap, small horses were available, because they weren't expected to actually fight on them, which made them cheaper to raise and maintain than 'regular' cavalry.
What makes Dragoons particularly and potentially useful for Civ is that they very quickly became used all over Europe for Reconnaissance/scouting duties. Within 100 years they also became good enough on horseback that some armies (Prussia for starters) were using them as regular 'battle cavalry' - charging enemy infantry, even (with less success) fighting regular cavalry on horseback. In other words, they could potentially be a late Renaissance-early Industrial Era Scout unit and/or the first Light Cavalry unit in the game that can actually get Terrain advantages.
Second, as rifles became the standard infantry firearm in the 1840s - 1850s, cavalry began getting off their horses more: a man on a horse was simply too big a target. The first to do this on a regular basis was the United States Cavalry, whose first 6 regiments included 2 regiments of dragoons, 2 regiments of Mounted Rifles (infantrymen on horseback) and 2 regiments of 'cavalry' - so they started with a strong bias for dismounted action.
That means that the Cavalry of the Industrial Era in Civ should get Terrain Bonuses, because they were increasingly armed with good rifled carbines (and, for instance, the 'karbin' used by the German cavalry in 1900 was almost identical to the standard infantry rifle of 1939, so in useful firepower they were almost the equal of infantry units)
Finally, interesting you should mention WWII, because the second largest cavalry force in history was the Soviet Army in WWII: in April 1943 their cavalry units included 9 Corps and just over 230,000 mounted men. Although propaganda photos almost always show 'Red' cavalry charging and waving sabers, and they certainly trained to do that, most of their combat action was dismounted. In fact, the cavalry were the first troops to have half their men armed with sub-machineguns for higher firepower in the assault, and by late 1943 every cavalry division included a tank regiment, so a 'cavalry attack' most often resembled an elite infantry attack: assault troops with submachine guns blazing, supported by tanks, self-propelled guns, and heavy mortars, all of which were part of the 'cavalry' Corps.