We must not forget that in some respects the AI in Civ V were done well. There were a lot of changes, yet the AI perform pretty well when it comes to workers, development, teching, wonders, settlement, etc. Many people turn their workers on automatic (I don't) because it's pretty good. These are all AI features. Some people would say that the AI sucks at settling--not picking the optimum spot or whatnot, but they still pick pretty good and make decent cities (not including the city spammers like Polynesia, Spain, Aztecs...) but even this is fine with me. Some cultures like cities and it's a perfectly legitimate strategy to try to grab up all the land (even the not so optimum places) to try to make the most out of population. It does pretty good for science and production long-term. What ticks people off is after all that, when the AI has done a good job starting their empire and even developed a nice military, that they can't fight with it properly. The main point of interaction with the AI is war and diplomacy. On both they act kinda like five-year-olds--however this is just one part of the AI.
I agree these two fronts could be improved, pretty easily actually, just by adding a few basic things. Since the AI is not the best at forward-calculating results based on the 1UPT rule have it know some cookie-cutter strategies that are cutting-edge. With their superior military this, alone, will make them way more competitive. Have them randomly choose, or better yet, have strategy sets unique to each culture to add a more exotic flavor. (aka Greece prefers melee units with high defense while Persia invests more heavily in horse and archers. Rome loves siege but has good diversity all around and protects them with formations)
1. Pre-program in some cookie-cutter strategies.
a) More sophisticated feints:
Use a damaged or weak-looking couple of units at a distance to draw out part of the human force. Ambush with hidden archers and kill with horse and then retreat. (more likely when outnumbered or the enemy has sizeable army)
b) Defense:
Don't be the first to invade, especially if you were the victim of a DoW. Instead have defensive strategies:
1 way: set your bombardment troops 3 squares back and behind cities, guard with a line of swordsman/pikeman/higher defense melee, have horse on line 4. Any incoming troops hit a wall of melee and take some damage. Bombard using city+3 archers behind. Mope up with pikeman and horse if needed. prioritize killing siege first and ranged second--especially with the powerful bombardment of the city. If the human tries to flank knowing there are archers behind, this is where the couple of horse on line 4 come in handy. use them to strike at flankers with the assistance of archers to prevent this.
(more likely when the enemy does not want war and just wants to stop you from killing them)
c) On crippling the economy (premonition to invasion)
First, gallop in horse to scout and pillage key roads/resources near the border isolating the target city. If they take damage withdraw if possible. Pillaging should keep them up in health though.
First: Capture all workers you can find (economic destruction tactics) If possible return them to your territory or disband if impossible. Second: Pillage resources: strategic first like horse/iron, luxuries second to make the enemy fall into unhappiness Both of these reduce fighting effectiveness of existing troops
Third: Destroy a few tiles of the incoming road as it delays reinforcements quickly coming up behind the city.
Fourth: Target high production tiles like mines/GP tiles/lumber mills/camps/etc. to cripple the city production and prevent building of walls/reinforcements (only necessary for high production cities)
d) On taking cities:
First: Horse to scout (see above)
Second: March in a formation of tight melee with archers behind, start demolishing surrounding defensive troops as quickly as possible. Prioritize killing ranged units.
Third: Move at least 2 siege into place right behind this wave (preferably 3+) (not 3 or 4 turns after the melee don't give the city time to kill all your melee. Have 1-2 melee or horse in reserve to capture the city if all others die (AI is ******** about this).
(more likely when the AI is aggressive, has superior military, and wants your cities)
Just adding these 4 cookie-cutter decisions depending on how threatened/aggressive the AI feels would make them more difficult, but obviously true sophistication would be the goal. If nothing else add some of these global strategies instead of wandering around and half-assing it on the spur of the moment each turn. I would put 3 variations on each tactic just so it wouldn't look the same all the time, but heck, even if it did it'd still be better than the way the AI currently fights. Right now they can only capture cities if they have an army large enough to surround it, survive for several turns, and manage to have so many siege you can't kill them all since they send them out unprotected and too quickly--and this is if they actually are targeting the city. I can't stand it when the AI surrounds and then sits there. What the heck is going on in those circuits?
For diplomacy: yes they are sometimes ******** and predictable but it isn't as bad. I think they did an OK job. The point isn't to be completely rational as many world leaders have not been. Adding more perception (aka reasons for wars and making connections) would be nice.
) and way behind me. 
