True. But neither does a focus in agricolture results in more food.
What usually is left of those cultures, are the results of what they built, to support their farming. Aqueducts, cisterns artificial canals.
Now, in my mind, those workers are the peoples living in the farms, your labour is...