Uranium wouldn't kill whoever touches it either. U238, the most common isotope (something like 99.3% of the earths uranium) has a half-life of something around 4.4 billion years. That's almost the age of the Earth. Which is why depleted uranium (which has a higher concentration of U238) is used for combat armor and armor piercing rounds. The second most common isotope, U235 (most of the remaining 0.7%) is not a whole lot more radioactive, with a half-life of about 700 million years. It's only the rarer isotopes of uranium that would actually do anything. Granted, eating U238 probably isn't the greatest idea, but unless you do that or something similarly dumb, it's not going to shorten your life.
The ancients handled uranium compunds all the time without knowing what it was or having any problems with it. It wasn't actually isolated and identified for what it was until 1789, and radioactivity wasn't known until 1895 (a uranium compound was found to fog up photographic plates)