That's fine for a rule of thumb, but there are always different opportunity costs to consider. The granary is a less mandatory early build if you're doing a lot of early expansion, i.e. 4 city liberty with 2 post NC expansions, or if you're crunched for happiness, or if there is no bananas/wheat/deer. If you had the mental capacity you could calculate a precise conversion rate of hammers to food to gold to beaker and always make the highest expected value decision. Food is more important than hammers in early game but you should still generally work a 1F/3H horse plains or a 1F/1H/3G luxury plains over a 2F/0H/0G unimproved grassland.
The more unclear conversions like the +25% gpp from garden or the FPT from the temple/shrine with no strong religion (investing in FPT for industrial era GP) are much harder for me to prioritize, but clearly belong in the build order somewhere.