While a larger city has to produce more food to regrow, don't forget it also has higher production. If you do everything one population higher, then if it takes 10 turns to build a settler, then the extra 10 food cost of regrowth is balanced by getting an extra production from a 3 yield tile for 10 turns. If you have a useful 4 yield tile, the balance is at 5 turns!
I suspect that the advice to start settlers earlier is correct, but I'm not yet convinced. The main counterargument is that your first few population points are fairly cheap as compared to settlers. I haven't figured out how to analyze the problem, and haven't tried making a simulation, so I'm not yet really sure what's best.