So I am playing on Prince which is normally quite easy for me ... but lack of gold is brutal. I cannot afford to keep any kind of army at all and the AI seems to have plenty of gold for its defensive armies.

Even with my two caravan trade routes and three cities, circa 1000BC I just can't afford to keep any more than a handful of units
What qualifies as "a handful of units", and what social policies are you using? If you only have three cities, it sounds like Tradition is a no-brainer, and that comes with free upkeep for three units and a bunch of free gold from your capital.
Also, switch out the caravans for cargo ships, they're worth more money. If you can't, at least make sure you have a caravansery in the cities you're running trade routes from. Bonus points if one of those cities can be on a river.
And the amount of STUFF you can build is just mind-blowing ... and technological progress just shoots by so quickly, you basically can only build a small portion of what's available, and you have to rush units out if you want any use of them before they are obsolete. It feels like there's so much choice now - units, buildings, policies - that it's totally impossible to fully appreciate it. The game feels like a huge confused rush. I don't like it
Well, yeah. You need to pick and choose your direction - you can't do everything all at once. Religion is probably the most noticeable in this regard - in G&K you could put a bit of effort in at the start of the game, get yourself a good faith-producing pantheon, and your religion would basically run itself from that point onwards, giving you bonuses. If you try that in BNW, there will be someone next door who went MAXIMUM JESUS and took the entire Piety tree, built Borobodur (3 free missionaries) and your holy city will be converted before you can blaspheme.
Decide at the start of the game, "Am I going to try to compete in the religion game?" and work out what you want from it if you are. Otherwise, don't worry about it, leave shrines, temples and the Piety tree to others.
This isn't just the case with religion, other game concepts have it too. Take cultural victory, for instance. You can spend a ton of resources generating culture, running specialist writers, artists and painters, taking the Aesthetics policy tree, building archaeologists and all that stuff. OR, you could go, "I'm not going to try to compete in this area", use the resources you would have spent on culture and great works to build an army, or bribe city states, or get more science, or something else.
I might be wrong, but it sounds to me like you're trying to compete in every area of the game simultaneously, and not doing very well at any of them. Try this: Play Brazil, and go for a culture victory. Forget religion and focus on making sure you have as much culture and as many great works as possible. See how you do.