Actually most games Oxford isn't worth building if you're going for domination, and Wall Street is almost never worth building. But simply telling you what not to build isn't going to make sense until you understand the reasoning behind it and the alternatives.
In a nut shell, usually the best way to run an economy is maximizing how often your slider is on 100% To avoid going broke you build wealth in cities, and if you have stone/marble/gold/etc you build those wonders for fail gold, and when the AIs get a decent stack of gold sell your cheapest techs to mop up the gold. Lastly sell excess resources, and sometimes even your only copy of a resource to the AI for GPT.
While you're running 100% research, any +

% buildings do virtually nothing, and that's why banks/markets/grocers are usually not good buys.
Observatories/universities do affect beakers and not gold, so the decision to build them is usually based on how much time is left in the game. Most military breakouts happen around liberalism, and the same time when you start building unis/obs, so they are usually not worth it.
And while Oxford is a decent building in itself, it requires you to build several unis to get it. If you compare the cost of unis/obs, observatories are significantly cheaper, so if you're somewhat close to the end, building a few observatories in key cities and ignoring unis/oxford is the best route. Usually unis/oxford are only worth it in space victories.
Harbors affect trade routes, so assuming you have foreign trade routes / need some health they are decent buys, better than say aqueducts or grocers.
Monasteries are sometimes worth it in very big

producing cities, but that's only if you're a long time away from scientific method.