What difference exactly? When I think "mounted infantry" I think about poor riding skills...
On the other hand, when I think "cavalry" I think about skilled, experienced riders.
Well, most cavalry in 20th century fought mainly dismounted. So you would call it "mounted infantry" (guys who ride to battle on horses but then fight on foot). However, most of these units were also trained horse riders & were also trained in mounted combat - apart from being trained to fight like infantry. Not mentioning that they usually had "cavalry" in their name. And they usually had sabers apart from rifles & other weapons. And ocasionally charged.
I wouldn't call them "mounted infantry" thus.