I've always been a good swimmer, but I don't do it often because I hate chlorinated, swimming-pool water. I love swimming in fresh water and in the ocean, but living in a city in the Northeast, neither is very convenient. The river that runs through the city was swimmable as recently as the 1940s-1950s, and there were public beaches up and down it. But by the time I was a kid in the 1970s, the river was so polluted you needed a tetanus booster if you went in. The harbor was even worse. The river's been cleaned up a lot in the last 20 years, but the beaches are long gone, and even though people sail and row on the river all the time, I'm not sure swimming in it would be a great idea. I think swallowing some river water wouldn't be life-threatening anymore, but it would be unpleasant, at best.
A remember reading a while back about refugees drowning in the Mediterranean trying to reach Europe because they couldn't swim. The reason they couldn't swim was because the wildlife in their part of Africa made swimming too dangerous. Your village might be near a river, but get anywhere near a hippo or an alligator, and you're a goner. So I suppose there are places where swimming for recreation just makes you look insane.