No granary is like the kiss of death, it's kind of surprising ANYONE picked that option. It's like choosing to do worse (of selecting the worst possible option) on purpose. The granary is the #1 economic building in the game. Most of your power will still come from worked tiles of some kind, and you want to get on them quickly. It is also the easiest health boost. Losing forge/CH/rax hurts a lot but it easily the least devastating of the options.