If I understand this correctly, basic improvements (Farm, Clay Pit...) do not provide any extra yields, they are indicators that those tiles are being worked. Buildings provide yields on their own, you don't need a citizen there. If later you put a building on an improvement, that tile will be converted from rural to urban, and the citizen used to be on the tile will be evicted, allowing you to reassign it immediately elsewhere. In cities (not towns), citizens can be assigned into buildings as specialists.