Could one not apply the same logic to a Sikh turban, or Jewish skullcap?
Certainly. However, first, it would tell you very different things. Unless the person doing the hiring is a bigot. And second, it would be illegal. It is a very different proposition.
The simple truth is, that once the tangible evidence is past, decisions are going to be made on the intangibles.
And there's nothing in the world you can do to stop that.
Several years ago, before the recession started, I applied for a job at Yale University. I didn't get it. But the process was informative. They advertised for people to show up at a certain office at a certain time. I got there, there were at least 30 people in the waiting room. Men and women.
Only a couple were wearing suits. A few more had on sport coats. Others were in shirt and tie, and it went downhill from there.
I had on a suit, and carried my material in a briefcase. Even though 30 odd people were there before me, I was the called in to a back office to sit down privately with a recruiter only a few minutes after I got there.
Now someone else was more qualified, and got the job. But if I had been equally qualified as one of those people who didn't take the time to suit up, then I really think I would have had an advantage.