So I suppose the problem with your Black people is, that they came to the USA as slaves.
When a group of people escape from slave hunters, the smarter ones will do it, while the dumber ones will get caught.
You caught the statistically dumber ones and transported them to the US.
Afro-Poles came to Poland of their own free will - so we got mostly the smartest and the most ambitious ones.
I think that the demographic which is able to emigrate in modern times is much more important than the demographic which were taken as slaves centuries ago.
Recent immigrants from Africa to the US actually outperform recent immigrants from everywhere else (including Asia) when it comes to intelligence tests and academic achievement. This has a lot to do with the fact that those emigrating today tend to come from the most elite segments of society.
It is worth noting that Europeans rarely if ever acted as slave hunters in Africa. They almost always bought slaves from other Africans. Such slaves were usually prisoners of war. In some cases, once the slave trade had become more profitable and tribes needed to buy European firearms in order to defend themselves from local enemies that were also trading slaves for guns, native African groups would attack neighbors specifically with the aim of catching slaves. Often, however, the slaves that the Africans sold to the Europeans were the warriors and political leaders of enemy tribes. It was not uncommon for the leader of one group to sell slaves for many years and then himself be sold as a slave once he lost a battle.
While genetics plays a role in determine intelligence, I think that environmental factors are more important. (You cannot clearly separate nature verses nurture, of course, because genes are more like switches than blue prints; they work by triggering the production of specific proteins in the response to specific combinations of stimuli, not regardless of conditions.) Those who are malnourished during their early life are not as likely to become as intelligent. Having higher social status has been shown to increase levels of hormones which are linked to higher intelligence in some areas. Being poor and worried about subsistence reduces mental functions.
A really important area to consider is the use of corporal punishment to discipline children. Corporal punishment of children is strongly correlated with lower IQs as well as with higher rates of aggressive or criminal activity later in life. The difference in IQ is about as great between children who were or were not spanked as it is between blacks and whites. The effect can be seen within races, and likely is a major cause of the difference between races too. Statistically speaking, black parents are far more likely to use corporal punishment and also tend to hit much harder when they do so.
This may itself be a relic of slavery. Slaves which were beaten by their masters or drivers tended to use similarly severe methods when they were trying to control their own children. They might not have been quite as severe, but thought they had to be tough in order to prevent someone else from whipping the children worse. Such domineering behavior is not good for developing intelligence. Parents tend to copy the parenting methods used on them when they were children, so harsher corporal punishment continues as a tradition in the Black community. Methods of discipline are getting better with time, but rich white families have been much quicker to adopt more peaceful parenting styles than have poor black ones.
Another important factor in developing intelligence is paternal involvement. (Perhaps it would be better to say the involvement of more than one committed parental figure; there are not nearly as many studies on children raised by committed homosexual parents, but children raised by lesbian couples do not seem to turn out any worse than those raised by more conventional heterosexual partners.) There was a time not all that long ago when black couples were even more likely to stick together to raise children than were white couples. That has changed, perhaps in part due to the way welfare payments were structured. (Giving more governmental assistance to single mothers than to married couples creates a major perverse incentive. This could be avoided through social safety net reforms like replacing means tested programs with a basic income guarantee.) Another major factor is how the criminal justice system has broken up families by imprisoning so many young black men. The War on Drugs has disproportionately hurt the black community, as blacks get arrested more and sent to prison longer even though they don't actually use illicit drugs any more than whites do.
(Genetics may be more important when it comes to things like heart disease and hypertension, which are a huge problem for African-Americans but are not particularly common in Africa. There is a theory that hypertension is linked with traits that made it easier to survive the Middle Passage, the stage of the triangular trade where death was so common that it would seem odd if natural selection did not weed out those genes less suited for the horrible conditions. On the other hand, hypertension could also be related to their high rates of corporal punishment. I recently read that studies have linked hypertension later in life to stress experienced at the ages when children are most likely to live in fear of spankings. Obviously diet probably plays a role too. Traditional African-American foods often use the less healthy cuts of meat, which were given to their slave ancestors because the masters did not want them.)
Those trying to claim genetic inferiority of Africans is the cause of lower IQ on average among African Americans also have to contend with the fact that most African-Americans are partially of European ancestry. Many Blacks have more European genes than African genes, and only a few (like Oprah Winfrey) could take a genetic test and find not a single marker of any European ancestry. Their European genes are likely those shared by the more elite Whites too, as it was the wealthier planters who impregnated more slave girls.