I'm no expert, but I'm sure you're giving yourself a whole load of work later on. Growing cities, especially your capital is fairly fundamental to building a successful civ.
I generally don't even switch my cities into "production" until they have between 5-8 population, and then will consider building a settler.
It's amazing how different the strategies are in this game....