Your speculation is wrong. There is a popular saying around the online gunner community.
"A pistol is only useful for fighting your way back to the rifle you left at home"
90% of people shot with a handgun survive. When it comes to a high powered rifle, that statistic decreases significantly.
The size of the bullet does not determine the size of the wound channel. Infact at high velocity the size of the bullet is really insignificant. The velocity a bullet travels determines the size of the wound channel.
High powered rifles simply have more stopping power than even the most powerful handguns. There is no question.
I'm not a physics man so I'll just show you some pictures instead.
"Wound channels" with common handgun rounds.
"Wound channels" with common rifle rounds.