It depends on how you look at it. May be a bug, may be a feature but certainly somewhat unusual. Anyhow, this is how it works. May be I'm wrong here though.
Let say you have a city with +2 food size 1. It needs 22 food to accumulate to grow so growing in 11 turns. On size 2 it needs 24 food to grow to size 3 (12 turns). On size 3 it is 26 food to grow to size 4 (13 turns). So, growth is incremental. The bigger the city is, the longer it takes to grow. In civ3 there were jumps from town to city to metropolis, so now the growth is gradual but the net result is similar.
Now lets take a city at size 3. On turn 0 it has 0/26 food in the bin and growing in 13 turns at +2 fpt. Then you build a granary and it has no immediate effect. However, when the city becomes size 4 (0/28 food, 14 turns without granary), your food bin is already half-filled with 13 food (13=26/2). Now it can grow in 8 turns with 1 food transferred to the following cycle (28-13=15, 16/2=8 and 1 left over since you need only 15 to get to 28 from 13).
Lets say we build the granary not on turn 0/26 at size 3 but on turn when there is 7 turns to grow to size 4 and there is 12/26 food in the bin. This means that all the rest of the food which you produce (26-12=14) goes to granary. but since the granary can handle no more than 13 food at that city size (half of 26), you get 13 food in the bin when the city reaches size 4 which is exactly the same situation as it is when you build granary on turn 0/26 of size 3.
Now, if the granary is built at size 3 when there is 6 turns to grow to size 4 and there is 14/26 food in the bin, then when the city reaches size 4, you get 12 food in the bin (26-14=12) and will have 8 turns to grow but without surplus 1 food. During the next growth cycle to size 5 the granary does get full of course. If the granary is built at size 3 when there is 2 turns to grow to size 4 and there is 22/26 food in the bin, you get only 4 food (26-22=4) at size 4 and will have 12 turns to grow (28-4=24, 24/2=12).
It is somewhat counterintuitive and not documented but more or less evident imho.
Now, starvation is a different story, I don't know how it works. Apparently similar to civ3, you'd have to eat up the granary first before actually starting on current food supply but I have not tested it yet.
This means, for maximal benefit and minimal head ache, might be a good idea to finish building granary early in the growth cycle or before the middle of the growth cycle of the city (or shortly after middle). That should give the most benefit for growth.