Because there is such a grey area with it.
Currently I am listening to the choral section of Beethoven's 9th on youtube. I don't own the recording, and I doubt the person who uploaded it owns the actual recording (which, given that it features Karajan, is likely owned by Telarc). Does that person's ownership of a copy of the actual recording allow him to put it up on youtube? Am I guilty of copyright infringement because I listened to it if it is decided that his uploading of it was copyright infringement?
As to the argument that it cuts down on sales, I find that a bit suspect. Of the CDs I own, many of them* I purchased precisely because I listened to them on youtube and decided I liked the music. In that case, the possible copyright infringement resulted in a net positive for the owner of the actual recording.
*List:
Berlioz La Damnation de Faust
Dvorak Requiem and The New World
Verdi Requiem (and soon Rigoletto)
Orff Carmina Burana
Prokofiev Alexander Nevsky Cantata
Tchaikovsky Piano Concerto A
And many more including Bach, Mozart, Beethoven, Bruckner, Handel, and Liszt.