About the "if they just improved AI", if you are talking about AI combat, I'm thinking that this might be a classic be-careful-what-you-wish-for scenario. For instance, on Immortal/Deity, it is simply breathtaking sometimes the army numbers that an AI might field (I'm talking just pure roaming army). Not unusual to see 10+ units ready to do something. Now just imagine these units suddenly being used properly... you're probably just going to die. I'm talking about ranged units suddenly being used properly (esp mobile ranged like Camel Archers but really all of them), AI units actually columning properly, switching out as necessary, properly timed mass-attacks for taking cities, and so on. Even *great* players would be in a lot of trouble here.
Now you'd have to ask yourself how did the AI produce some scary armies in the 1st place- well the giant gold/science/pop/production bonus. But we're suggesting that in exchange for good combat AI, these bonuses either go away or get very toned down. So maybe it would even out. I really think that some of these AI combat improvements are actually achievable now but that's only my speculation.
--------
Back to the original question, watching players like Wainy, I get the impression that you really need to know just how much

you can possibly get for any AI transaction. Apparently there are adjustable formulas that he has on paper to use in various situations for selling luxuries, open borders, strategic resources, peace treaties, etc. Being in the negative

off and on for the first 100 turns should not panic you if you know the optimal trading strategies and you're using your military fairly aggressively to pillage the gold you need and/or find a favorable peace treaty.
Then there are advanced techniques like timing research agreements or calculating the timing of the conclusion of a RA with your own science plan. Planning your science route and making adjustments to that is important. Checking your military adviser before a major war decision or to check on the status of potential or current enemy is important. There's just so much more to say though.
Finally there are the "reckless techniques" that seem reckless or counter-productive but might be necessary. Like selling a city that an enemy AI is on the verge of taking from you to a different civ, selling a city you've just taken back to its original owner, ignoring a powerful AI that just DoW'ed you while your own army is entirely in a different region warring on another AI... just weird off-the-cuff things like this that you might need to do.
I think it's probably possible for most people to get "good" at this game, but realistically it's not attractive enough for the majority (including myself) to get there.