Since we're doing the way-too-early-to-bother kinds of speculation, I'll mention the following;
So far we've observed
-A Granary costs 50 production,
-Builder costs 44.
-A Granary adds +1 food and 2 housing.
-A Farm adds +1 food and .5 housing.
-A newly constructed non-capital city has a housing capacity of 3. (Capitals appear to have a capacity of 7)
Judging by those premature numbers, a builder first seems clearly better than a Granary. Especially in the capital where housing seems to be a non-issue at the start of the game. The builder is cheaper than the granary, so the initial +1 food bonus comes online quicker, and on first growth, the +1 food bonus becomes +2 food (+1 from each farm).
The above doesn't even factor in resources or if your first 3 builds were farms to get an adjacency bonus. If we assume that that bonus doesn't come from a tech (which I personally think it will).
Until we see the real numbers for certain, there's virtually no way that Granary first beats out the builder, really. If there's maintanence, then the obvious path would be to build a worker, use all 3 builds immediately, and then a warrior to defend the investment. Barbarians usually don't start roaming in the first 25 turns and despite the changes to them in civ6 I doubt that to change. A "grace period" is pretty standard.
Edit:
More numbers for the sake of discussion, all subject to change;
-Scouts cost 20 production.
-Warriors cost 30.
-slinger cost 25
-trader cost 34
Looking at it again, the fact that a scout costs half as much as a builder, I'd probably just build a scout first and then the builder. The scout is significantly cheaper than everything else and the early gold/eureka/hut bonuses of exploring make going anything other than scout not worth it in my mind. By the time your builder is online your city would likely be pop 3 anyway so it's growth would receive a burst going forward. I highly doubt the difference in growth speed of builder first over scout first will recoup any potentially loses of exploring.