ah but define what is human. You would define what is human by 'beings like me' because in a pre-historic culture it is near impossible to prove what is sentiant. Especially when you have wise men trying to find spirits in the rivers and the clouds.
Picture it: A culture has lived for generations in seculsion, forged a small village in the wilderness and slowly started to develop a language and a sense of self. The people begin not to just feel and react but instead they think and are pro-active in taming the enviorment to their needs.
Then: Other things arrive. They look similer but perhaps they are taller, their heads are shaped differently, they move in a different way, they dress differently (if at all). There is no communication. They have their own language. They do things completly differently, while maybe you plough the fields, they mine the hills. So strange, so different. They are not 'Human'. There is no word for Human in your language yet, no concept for it.
In the terms of the pre-historic people, We are 'English'. that is our word for 'Human', When these things that call themselves 'Romans' or 'Vikings' come along they are aliens. We have never met them, cannot communicate with them... We cannot comprehend that they are 'Beings like us'.
Thus if they are not 'Beings like us' and are 'Aliens' why must we say we are equal? perhaps they are merely animals that have learned a new trick, perhaps they are gods or perhaps they are the so called spirits the wise men speak of. But once they are something else then we have allready decided that they are worth something different then ourselves. Then it's just a matter of whether they are worth more or less...
(Note: i'm just presenting a counter arguement. In civilzation everyone seems to speak the same world-language which kind of makes things simpler

although you could argue that the "we have known centuries of peace" bonus is actually the continued progression of learning each other civilzations ways and coming to recognise that you are beings of equal worth)