Saying the AI will never be good in Civ makes no sense. Yes the AI can be good. It may be not easy, but it can be done. Only two things are needed if firaxis wants to improve the AI. Resources and care.
The AI struggles in late war systems, cause probably it was not playtested enough or it was just rushed in developement, or not dedicated enough time. Combat AI in this civ is a weird thing, cause im convinced the AI is designed to not be at war. That effect goes up as the game advances, so the AI really never uses war much, it is just programmed to defend itself when the player attacks it. This is a big mistake, and ruins the best game system for me. But this also means that late combat behavior is so uncommon, that probably many needed logic was not implemented or tested. There are other issues with diplomacy, aid requests, city planning, use of siege weapons, pathfinding, coordination of groups of units, lack of some tactics such as try to heal units or cut enemy reinforcements...
All in all, a lot of things fail with the AI. Is that bad as many people think? No, i dont think so. The AI is competent in general in, and does some things remarcably well, such as trying to stop a player land invasion. All in all, I think firaxis can improve the AI significantly in the next patch if he wanted to. All that is really needed is to put people to work in the problem.