There are some terrain/climate combinations that do not support large groups without advanced technology. That technology includes the ability to provide supplies from Far Away.
So I would base 'attrition' on three things:
1. Type of terrain. The obvious ones are Tundra, Ice, Desert. Less obvious is Rainforest (Jungle) - if your troops are not acclimatized, the tropics are deathtraps from disease. Look up what happened to the French Army on Haiti, or Chinese armies from northern China who tried to march into the semi-tropical south of China in the Classical Era.
2. Size of force. We can assume that any military unit represents 1000 or more men, so all of them would be subject to attrition. Units in the Reconnaissance/Scout line would not, because they represent much smaller numbers of men in those units, and so should be able to 'scrape by' even in extreme terrain.
3. Supply Technology available. This is going to have to be Implied, not Explicit, or we get into a very messy game of Supply Lines and Military Economics that will drive customers away in droves. Instead, starting as early as the Classical Era you can 'supply' units in attritional terrain as long as they are next to the coast and you have a ship adjacent to them on a coastal tile. That extra naval unit represents the investment required to keep the unit 'supplied' - and note that Every Unit in the bad terrain has to be adjacent to the naval unit at the end of the turn, or it suffers Attrition. Other 'supply' can be made available with the Combustion Technology, but for every unit in attritional terrain at the end of the turn, the Maintenance cost in Gold and Resources is doubled. Even today, keeping military or civilian units intact in the desert or the arctic is Expensive
The result of a system like this would be to make it very, very difficult to exploit ice, tundra, or desert tiles early in the game with armies, and would also keep 'scouts' relevant for much longer in the game since they would be your only units that could trot through that terrain without attrition until late in the game.
One caveat to Attrition: any tile with an Improvement, Wonder or District on it, regardless of other terrain, would not produce attrition in Units - the structures imply an infrastructure and population there that can support a unit there.