You're missing the potential of a Goldilocks type condition, where too much one way or another might prevent life from occurring. Just because the values aren't extreme, doesn't mean that these aren't special. For example, being at the center of the galaxy would mean the solar system is more likely to brush close to another solar system, with disastrous results to worlds that might have otherwise harbored intelligent life, whereas the outskirts might not contain sufficient heavy elements to make planets; it's not implausible that only systems not too close nor too far from the center can spawn intelligent life (or life at all). There could be many many of these parameters that need to be just right for life (or intelligence) to occur. They might each be fairly typical bulk parameters, but the worlds that can support life (or intelligence), might require a very special kind of typical!
However, that's not the full story! That something really really special need not be anything astronomical! It could be that life itself is really really special and rare, even if brute astronomical conditions are common. It may be that worlds of many, sizes and shapes, in a variety of orbits around a variety of stars in a multitude of areas in plenty of galaxies might all be about as likely to have had life as Earth. It's just that that probability is very small that life would develop in the first place.
So in the end I don't believe you can assume that life (or intelligence) is common.