Very early in game happy cap is low enough to postpose a granary, but other than that its usually my 1st building, and its built on each and every city.
Exception are captured cities, where Theater comes first, but granary usually is 2nd. The only reason for it not being the 2nd is: if captured city is very big (size 15, 18), and borders very tiny (5, 6 tiles to work with when revolt stops), then city will probably be starving, and granary will be useless. Then i delay granary and build more culturural buildings first (whipping or buying), and Granary when borders expand enough for city to have surplus food.
This is a quite common scenario in my games, as usually i aim at taking capitals first, completely sorrounded by enemy's culture.