Each city can only work the tiles in the BFC.
not castles, forts. or you could build the improvement necessary (plantation, winery, etc) to still give you access to the resource (health, happiness etc..)
also, another reason for improving outside of BFC is to chain irrigate farms, if a city...