For 180 hammers I could build two Swordsmen with 35 combat strength. You may be correct that +5 is significant in a one-on-one scenario, but in a two-on-one scenario, the advantage clearly goes to the Swordsmen.
Upgrading immediately will likely be the best option, assuming finances can handle the increased unit maintenance costs. However, zipping through the tech tree to get more advanced units will no longer be as much of an advantage as it was in previous Civ games.
Opportunity costs: ~2 Settlers; 1 Settler, 1 Builder, & 1 Scout; 2 Scouts, 2 Warriors, & 2 Slingers; 1 Holy Site & 1 Shrine; etc. There are so many viable options, and none of them are inherently superior to any other.
In the end, geography will decide. If your starting location has plenty of mountains and rainforests, an early campus may indeed be one of the best starting options. Mountains and forests may lead you to go down a religious route. And if there are no mountains at all, it may be better to build some scouts and troops and start hunting for cities that would be better suited for such specialization (while you figure out what to do with your capital).
As I mention above, a lead in science is not as large an advantage as it previously was, especially if you have had to sacrifice production, economic output, and other aspects of the game in the single-minded pursuit of campuses.