From experimenting a bit, it seems that you don't have to be online together, but whoever creates the game can open however many slots for human players. Pick your civ or "random" and hit ready. When the creator of the game hits ready, it creates the game in your list. Then you can send your friends a "Join Code" and they can go in the game (even when you're not in it any longer) and pick their civ, and hit ready. When every player does that, I think you're able to actually start the game.
I'd love if anyone else has any experience with this if they could chime in, because I'm not 100% that's how it all works, but it seemed to work that way.
Also, I'm noticing if you choose to stay in the game after you make your move, the game has to reload to "see" any moves other players have made. Bit strange.