The reason you hear it a lot is because it was the stated goal of the developers. A quote from Jon Shafer himself back before the release of vanilla: "Our goal was to make diplomacy feel more like interacting with other players." Back in vanilla one of the diplomacy penalties that you'd get was "They think you're trying to win the game the same way they are." The AI was not playing in character, but explicitly playing a game. They would always eventually stab you in the back at some point.
This has changed quite a bit over the various patches and expansions, but the core Civ V diplomacy design that persists into BNW makes it still very difficult (and not very rewarding) to make and keep long term friends.
I understand that. But as you noted they've changed it quite a bit. The current system doesn't feel like playing against human players, it just feels like playing against in-character AI players who can scheme and has agency (to look after their own interest). Are you suggesting an AI backstab is a bad mechanic? I mean, that's actually something a lot of Civvers wanted.
The set DoF periods provide a pretty clear window when they are your friends. There's sometimes 'fake' friendships where they ask for a DoF then declare, but for the most part when geopolitical alignments conflict between friends they simply refuse to renew and you know things have soured. It's really not the surprising erratic relationship it's sometimes painted to be, just gradual ebb and flow of alliances as interest change.
As I noted, what's often left out when complaints are made about not being able to make long term friends, is that you can make new ones (shocking I know) as map realities change. It's not a one sided affaire.
I don't get the desire for static game long friendships. That's really boring and not really realistic. What you're essentially asking the AI to do is give up because they happen to be your friends even if they're the only one all along who could have challenged you.
But this terminology of "throwing the game" suggests a multiplayer mindframe. Not every country should be trying to conquer the world; it should be okay for some to want to just survive. Weak nations should be trying to ally with more powerful nations, instead of making suicide attacks against them in a vain hope to "win the game." It's very immersion-breaking.
Pretty sure Civ5 does this better than Civ4. Some Civs will focus on wonders, some on city-state alliances, some on warfare.
The breakdown in relations (or long-term friendships) occur when the human player is also pursuing one of these goals and the AI will naturally not like that. That is not really an immersion breaker, but creates more immersion.
On the point that the weak must submit to the strong you seem to suggest the AI must show some agency (something we agree on) to protect their position, yet your solution is to simply allow smaller AIs to be swallowed up in some opaque alliance with a larger Civ (we don't agree on this). AI's survival instincts must be tempered by the fact that Civ5 BNW is designed to be a fluid diplomatic game, and submitting to one country in fixed alliances is not ideal. Besides, what usually happens when a smaller Civ in Civ5 doesn't like you is they find other (larger) Civs to not like you and be friends with them. That's usually not hard to do when the human player is large enough to have a centre of gravity. There will undoubtedly be someone else on the map who doesn't like your empire's pull. It's a naturalistic way of alliance forming as opposed to the 'set' vassals system and set diplomacy you maybe want to see. Besides, strength is a fluid term in Civ5 with ideologies, tourism, and religion all providing key inflection points to the game. A small middling Civ could come out the winner in the late game by virtue of their city state alliances, ideology and culture. So we can't so easily define what is small and weak vs. what is strong. What the game ultimately tries to do is sort out groupings of Civs along national interests, and those interests change as the game progresses. That is actually how real geopolitical alignments work too and it's not something gamey they are trying to pull over us.
You misunderstand me; I'm not blaming the AI, I'm blaming the design. Of course diplomacy is fruitless in multiplayer, for the same reason it's fruitless in single-player with AI factions trying to act like human players: there's no point in making deals with someone who is only looking for a chance to stab you in the back. All the multiplayer 4X games I know of (Endless series in particular) have similarly useless diplomacy systems.
Ok,
I'd like to pick your brain on your idea diplomatic SP game. Forget for a moment there is Civ4, or Civ5.
Just describe the mechanics, how nations and rulers should interact. How international relations are kept. What are the flashpoints for war. etc.