Sure but over time a lot of slavery systems also had the population of slaves disappear, and since Civ takes place over the course of centuries the slaves disappearing could just be reflective of historical accuracies. Besides you make it sound like workers aren't worth building in Civ VI, but of course they are.