It really is hard to fathom, and it makes it feel like some of the civs in the game aren't full-fledged rivals, but are just flavor: mostly waiting around for you to discover them once the Age of Exploration arrives. And that, additionally, has this icky lingering subtext from our-world's colonialist period and its authorizing presuppositions.
But viewing it just from a game standpoint: Really fundamental to my sense of a Civ game is that all civs are starting out together in the year 4000 and developing as their leadership allows, and that my game is fundamentally a matter of how well I do relative to them: that we're all playing the same game according to exactly the same principles from the very start.
I don't want to start playing Sorry with red, green and yellow, and have blue just show up (out of the blue, ha ha) having been playing by its own rules for the first third of the game-time, and now awkwardly integrating with the other three colors.