Like Winston Hughes said, it's hard to give hard-n-fast rules for CivIV, though we typically try to get as close as possible to good "rules of thumb".
I used to struggle with building decisions until I came to terms with city specialization.
City Specialization is another concept in Civ4 that goes totally against strategies and methods in previous Civ titles but is a vital concept for moving beyond Noble and Prince.
Anyhow, early in the game, identify the best city site suited for high production and settle it and/or identify your best production city. In that city, build only a Granary, Barracks and +

% buildings (+

and +

when necessary) and focus that city on military production. I typically never stop generating military units from this city once I start.
If you get to the point where you think you have too many military units, you should consider settling more economy-boosting cities or conquering your neighbour(s).
On the contrary, if you get to the point where you think you don't have enough military units, find your next-best production site and devote it to military production also (or build Heroic Epic, Military Academy, etc.).
Check out the two links mentioned at the top of
this article on city specialization.