I think that they should take damage each turn in a mountain. Mountains are not easy to traverse, and they take a huge physical toll on the person travelling them.
I don't think it should be a lot...maybe 5% per turn, but it seems to make sense to me. Moving on mountains is a huge advantage when there are minimal creatures that can do so, and it shouldn't be so easy.