I think the book takes a minimum of the top 4 civs and determines their power and since you only have 2 active players that is how the barbarians got in there. They just put barbarians into each slot that is empty. So it doesn't mean there are 2 different barbarian groups.
.....wait a minute...... nevermind, How could they be the LARGEST? I could see how they might be the most powerful, because of # of units, but not largest in territory size.
I think some of the people who posted before me are partially correct, just needed to state it more clearly. Each barb camp would count as 1 tile, so if there was more camps than you have tiles they would be bigger than you in size.
I bet if the book would show up if you were the only civ, then 3 barbs would show up. But when you are the only civ playing the game treats it like you've won (no music, etc), so I don't think the book would show up.