As JR pointed out, an officer only needs "reasonable suspicion" that you may have or are about to commit a crime. "Terry stops" were ruled
legal by the court years ago... not illegal. In practice in NY (and almost every other big metro police force) this means being black or latino and in the wrong neighborhood, basically. The vast majority (9 out of 10) of people stopped by this practice in NY were completely innocent
according to the NYPD. If you were a cop, and you stopped and frisked 10 people and 9 of them were completely innocent and doing nothing wrong, I think questioning your perception would be warranted.