Something else that hasn't been mentioned, not that you ever need to justify something that's good for game balance...
Slaves can be oppressed with very little backlash. Slaves are not just poor people -- they're property. They are not citizens. They rival about the same amount of rights as a prisoner, and even some prisoners were treated better than slaves. Furthermore, even many SLAVES accept that they are slaves, and their children will be slaves, and that this is just the way it will always be.
The draft actually affects all people equally (with the exception of the extremely wealthy and well connected, but don't ask me about that, ask George W. Bush and Dick Cheney). These are not just full citizens, but often people who are well liked in their communities, with commitments and responsibilities.
It's understandable that people would care more about their fellow citizens -- from working class to middle class -- going off to war. Slaves are isolated from society and aren't even considered citizens. Sometimes they weren't even considered human beings.