@HorseshoeHermit, I think we are both circling the same ideas on City Development.
I believed back then, and now, that the Population has to be divided into them that works the land, which for most of history was the huge majority of the population, and them that works in the city, with a certain degree of 'specialized' skills and training.
Keeping it simple, the Population Points we all know and 'love' in Civ would be the bulk of the population that works the land around the city - the farmers, herders, miners, etc that feed the city with food and materials.
In addition to those Points, though, there is a separate group, until the Industrial Age and later much, much smaller, which uses many of those materials to fabricate other goods, and ideas, and form the basis for what the sociological historians call Urbanization and Civilization.
I do think that virtually every Building in the city could have a Specialist Slot, or more than one. What benefits they give would depend on the Building basically, but also could depend on the formal Training/Education available for those Specialists, and that could depend on other Buildlings in the city (Technical College, University, School, etc) OR the Social/Civic choices made: Guilds immediately spring to mind as a Social/Economic institution that augmented and regulated training and skills among a large number of Crafts and manufacturing in the Medieval (European) City.
Even the city walls could have a Specialist slot - filling it would be what gives the city defenses a Ranged Factor, because somebody has to man the catapults, cannon, wall guns or pour the boiling oil.
What massively speeds up early development of a city and its infrastructure are the changes and improvements in Tools: there is an absolutely enormous difference in the amount of time required to clear a field with stone tools and axes and with iron or steel tools, and construction with virtually all kinds of materials: woods, stone, brick, etc, gets fundamentally faster with metal tools, and then with Power Tools in the 20th century.
Humankind shows the faster City Development by having Techs in the late game that allow a new city to be built with all the existing Infrastructure already constructed in it. Since that game has about twice as many Buildings and Infrastructures available as Civ VI does, that's a simply massive change - new cities may, in fact, be more productive than a city that's been around for several thousand years, simply because you rarely have time to build all the structures available in a normal game.
That's a mechanic that should be examined further, though, because it points to an in-game solution of sorts to tfhe slow later city development that has been embedded in Civ for as far back as I can remember.