It absolutely has a huge effect on the lives of blacks and Hispanics. It overwhelms nearly all other considerations. Imagine spending a portion of your life in jail for no other reason than that you are black, and a white person who had does exactly the same thing did not spend a single night in jail for it. Imagine that your father spent your childhood in prison and your mother on welfare for no reason other than the fact that you are black, and a white family would not have had that happen.
How does this not have a huge effect on how your life turns out?
As a "hispanic" who has lived in the US, I can say that I was never affected by racism in any way whatsoever. Neither has anyone in my family or any of of the people I know who live in the US.
So yeah, racism is certainly very meaningless compared to other issues.