When do you trump realism? There are a couple things that come to mind:
Too random: If a gameplay effect is too random, like a football game where a player can suddenly get injured for no reason, the player gets frustrated and the game is no longer about skill. Of course, if there's a random dice roll when the player attempts a death defying move and the player gets injured, that's a bit more acceptable... so long as the death defying move is known as such.
Too small: Imagine an RPG about killing dragons that also forces you to eat, go to the washroom, fold your clothes... or else. Not to say that remembering to go to the washroom is a bad game mechanism, but that it doesn't fit with the scope of killing dragons. Details are important, but they ought to reinforce the scope and focus of the game, instead of distracting from it.
Too repetitive: A repetitive action, like holding elections every three minutes, is annoying. But this doesn't mean it has to be eliminated. It can be automated and simulated. It can be elongated, so you have an election every half hour of real time. Or, in the case of Civ 1-3, it can be abstracted: Democracy has generally more happy people, and your empire is still generally continuous.
Too unbalancing: Imagine a game where you can choose between a sword or a gun. In the name of realism, the gun is way more effectively and beats the sword 95% of the time. The game quickly becomes gun versus gun, and nobody ever picks the sword. You should either remove the sword, or give people a new reason or incentive to pick the sword. It might not be realistic, but maybe the game would be better if the sword could block bullets. Or maybe the gun should be heavy and slow you down.
But, to me, this isn't a question of realism. Smaller armies have beaten larger armies, and technologically inferior armies have defeated technologically superior armies. It's a question of tactics. Tactics are very realistic.
Luck is not the issue. Making the game more random so smaller armies beat larger armies more often, and spearmen beat tanks more often does inherently make the gameplay better. If you made it so someone who was really brilliant with their spearman could beat someone who was really sloppy with their tank, that might actually be a good game. The key is being able to point to superior skill, not superior luck.