You could switch all your folks to workers, move the houses to get the best possible production out of what you have and use a few harvests. That might get you enough to rebuild at least some of the pastures. If you use enough to get under the 24 harvest cap, you'll get some trickle while waiting.