What matters is activity. You might have only a handful of civs (e.g. EFZI where you start with 5 and then they're whittled down as the story progresses) but if there's a ton of resources and trade routes, plus unit movement and production, then the game still takes a lot between each turn and the next.
That's why scenarios like TAM have options like ‘no barbarians’ or ‘no wolves’.
That's why scenarios like TAM have options like ‘no barbarians’ or ‘no wolves’.