I agree with this, and it's felt off since IV. Partly this can be fixed with better border expansion (VI is better than V at this but still not perfect). But IV also made the map more interesting by having barbarian cities, which I really miss. They were always in good locations so it introduced competitions with the AI to conquer them, and then after astronomy they were dotted on the separate islands and continents and so added a conquest aspect to colonisation. They should return this by implementing a new barbarian class of city state with unique mechanics.
Perhaps also a benign form of barbarian, e.g. wandering nomad civilian units who can be induced to join a civ (or captured) with some sort of benefit, like a unique tile improvement.