It's not the issue itself whether one identifies races or not, but if some races are projected to be superior in some way. Otherwise, you don't see problems arising with mere categories existing. Human thought is based on categories, which the more intellectual/scientific they get, the more they approach complete neutrality. Imagine the absurd case of thinking that primes shouldn't be their own category, because that leads to more interest on them in regards to most integers.
The problem with categories in less intellectual/scientific/closed system groups, such as race as a term irl, is that the vagueness of its intended use easily can lead to suspicion of biased and polemic misuse - and at times indeed the suspicion is correct. But you won't be rid of that by cancelling an entire category, nor (should be very obvious) would a human society without the notion of race be unable to see (say) skin-tone difference. It's like expecting that if we somehow lose the notion of "fat", people will be unable to identify weight differences.