As all things civ, it depends, especially the first ones.
Generally speaking, you want to minimize the builders you build early, because every one you build will increase the cost of the next one.
A high level strategy (used in competitive multiplayer iirc) is to forego the production of builders almost completely, until you get Feudalism/Serfdom for the +2 builder charges, and only then start to get a mass amount of builders to improve your infrastructure.
Generally I play along that line myself (though not as strict since I prefer single player or playing multiplayer with friends against Deity AI), and what I usually prioritize is instead to get a large amount of settlers out as fast as possible.
While getting settlers out doesnt increase your production as a builder would do (improving lands, especially building mines or chopping), it does add to the overall production that your empire has, and it does increase the number of trade routes you can potentially get (which does create production assuming they are internal).
The only times I do go out of my way to get an early builder is if I need a particular eureka, I really need to chop (usually I save this for Etemenanki if the game allows for abusing it), or if my lands are extremely unproductive (hardly any production yields), or if I want a particular luxury to sell off early for extra gold (or for some reason really need more loyalty).
But usually, you can secure enough food and production early on (which is the two yields you want to focus on above all others) by settling in good spots and making sure that you manually work the high food/production tiles to help your city grow.
Either way, the key to knowing how many builders you "need", is a question that usually arises in the middle game (especially when you have Feudalism).
At that point your cities should be relatively undeveloped tile wise, and you ought as usual to prioritize food and production, and to maximize the return yields.
Farms usually dont pay off that much (they only give +1 food unless you make a huge cluster), but mines do (+2 production), hence you want to maximize the amount of yields that an improvement gives you (+2 from improving a tile is simply better than +1 after all).
I thus usually make it my priority to get plenty of those up, assuming that I still have positive food when working those tiles.
And the key metric to look for when deciding for how many builders you "need", is to try to improve the number of tiles equal to the number of population each city has.
Thus if you have 10 cities with populations ranging from 1-9 for instance, you want to make sure that (roughly speaking) you have enough builders that each of your 10 cities roughly can work as many improved tiles as their population demands.
Which means that some cities just get 1-2 tile improvements for the time being, while others get closer to 9 (assuming the tiles are worth improving).
Obviously some exceptions apply, for instance if you have some unimproved lands (wonders especially) that naturally have high yields, in that case you "save" a builder charge since you cannot improve it anyway while working it as before.