I think any non-profit social service organization could at least be exempt from paying taxes.
I don't think churches should be tax-exempt just because they're churches. Of course some churches also provide social services. I think exempting a religious organization from paying taxes violates the principles of separation of church and state, and of religious liberty, so a church that wanted to operate a non-profit social service would have to make that a separate organization and keep separate books (which they might want to do anyway, I suppose).
Public libraries around here are municipal services, not non-profit businesses. But before public libraries were the norm, there were social libraries, which were lending libraries open to the public, but requiring a membership or subscription fee. Benjamin Franklin founded one of those in Philadelphia well before our War of Independence. I think they were common up through the 19th Century. I'm not sure how many of those are still around today, but there are some, and I suppose they could be tax-exempt non-profits. I'm not sure private libraries should be tax exempt. I think private schools are tax exempt organizations, so I suppose their libraries would be too, whether they share their collections with the public or not.
I assume museums are like libraries, but I have no idea how many museums are municipal services, the way public libraries are. I imagine a lot of them are non-profits, though. I think the National Gallery in Washington DC is free. I don't know about the British Museum in London, or the Louvre in Paris, or other comparable places. Are those free? Or, at least, inexpensive?
EDIT: Also the Smithsonian museums are free to enter, funded partly with federal money. If I can ever get my [butt] to Washington DC, there's a lot you can do without spending a dime.