A spearman has defence 2. Say it's in a metropolis (size > 6) on a hill, and it's fortified.
Defence bonuses:
fortified: 25%
Hill: 50%
Metropolis: 100%
That's 175% bonus to its defence. I wouldn't be surprised if the comp rounds that 5.5 (2*2.75) to 6. That means the spearman effectively has a defence of 6 when you attack it.
Now Cavalry has an attack of 12 iirc (I'm playing the DyP mod, so I'm not sure if they changed it from the default), and in theory it should get through that spearman... but luck is fickle. If you had had 2 cavalries you would most definitely have gotten through.
Musketmen, however, afaicr, have pretty low attack (lower than 6). Not much of a surprise they didn't cut it.
If you want to solve that problem partially, install the DyP mod. They changed the hit points there so that, while the above scenario can still happen, it is exceedingly unlikely given that the hps of a veteran spearman is 4 and the hps of an veteran cavalry is 9. Can still happen, but very very unlikely.
Daniel