You can get the Great Library on every level, but the higher you go up, the more luck you need. On Immortal and Deity you're dependent on the AI not beelining to it straight, on Immortal I know from experience the AI can get it in the 30-ish turns, you just have to hope they're not that quick.
Philosophy will often turn out to be the best tech you can get for free on the highest levels. But if you don't build the Great Library you will also often see that the AI gets it as late as turn 80 - 100. You never know this beforehand. The only thing you can keep an eye on is when the AI gets Writing, but you'll probably not know all civs on the map yet.
Emperor would always give a more than decent chance, though.
I personally play Immortal and it's bit 50-50 whether I'll go for the Great Library or not. If my production is average I won't try.
There are sacrifices to be made, it hurts your expansion, empire strength, and depending on your resources also your production and happiness. For example, you have stone, and perhaps marble, but you don't want to research Masonry, because then you'd certainly miss out on Civil Service or whatever as your free tech.
The Great Library is nice, but the sacrifices or often overlooked.