The thing is, the AI doesn't actually get any smarter between levels - it just gets more bonuses.
That's always been my understanding, and it's the way all previous Civ AIs were programmed, but I feel like I do see better behaviour from the AI in various aspects of play at higher levels - they pay more attention to city-states (but maybe because they have more gold?), CSes themselves are more active beyond their borders, there is some improvement in the way they play combat (I'm not seeing the much-derided "lead with ranged" much these days).
EDIT: I'm now fairly certain the AI is programmed to make combat easier at lower levels. Played partway through a Warlord game in MP today (as I and my regular Civ partner were teaming up with one of his friends, who only just got Civ V). Attacking Darius (one of three AI players at the start of the game), he had an archer garrisoned in his city, but while he'd bombard my units with the city and had line of sight to them with the archer, he never used it to fire, while on Emperor the AI defender will always fire with his garrison, as a human would (and Bismarck did so annoyingly effectively too). Attacking Bucharest later in the same game, the same thing.
The AI on Emperor will fortify, reinforce, or back off from a melee that will certainly cost it a unit with no gain. Darius had no compunctions about running a slightly wounded Immortal into my full health Swordsmen (who were standing on a hill), in a combat he can't have calculated he'd have won. On Emperor I'm used to the AI being wary of attacking with equal-tech units if it's been damaged, sometimes even if it seems likely they'd win the fight, never mind more primitive ones.
Also, Darius tried to move out with settlers several times, and in no case were these ever protected. I always see settlers being herded by Warriors on Emperor - yes, well into the medieval era and later, but at least it's programmed with a basic instinct for preserving non-combat units.
The reason it's behaving more cooperatively, I think, is because I'm running at closer to parity with it. Not pulling so far ahead that the code prompts it to jump into desperation haymakers nor falling so far behind that the code evaluates me as easy-pickings.
Yes, I suspect this is likely to be the case. Certainly I got through the lower levels by having a big enough military to prevent early aggression - but at the same time I'm never close to the top in pointy sticks rankings even on those levels since I don't often play highly aggressive games.
EDIT: Again this Warlord experience. I tried to sign an RA on meeting Wu Zetian (the only surviving AI player - I didn't want an RA with the humans partly because they'd just made one and may not have had the funds, but mainly because I was trying to preserve my tech advantage over them, particularly my nearest rival who had the Porcelain Tower) and she wasn't having any of it; she wouldn't even accept a basic open borders deal. I did get it by trading her horses I didn't need, a token amount of cash, and one-way open borders which she'd get no use from being on another island. In this game I'd been aggressively expansionist, taking Persepolis before 1 AD and grabbing its ally Bucharest to gain points with Singapore soon afterwards. I had a large army with, at that time, three cavalry, a couple of Mandelaku, and assorted cannon, musketeers, crossbowmen, pikemen and longswords, as well as a couple of frigates, and could sustain much more since my strategy for this game had been, pure and simple, to maximise my gold income. I was the highest-scoring player, and ranked 500 points higher than poor Wu. I don't usually play that aggressively or with that large an army, and I don't normally encounter that intransigence over mutually beneficial deals, so there definitely seems to be something in your notion that a civ won't deal with you if it's too scared of you (in MP you don't get to see relationship modifiers with the AIs).
As an aside: he came close to victory in that game by 'declaring war left right and center.' Given his position - and choice of VC - it was the right move in that game. He just failed to build a navy, and left himself wide open to getting sniped.
I think Japan has to be an exception to that rule - after all, the Civ pretty much has to go for domination, and while I've seen leaders from Gandhi to Genghis go for non-domination victories, and indeed change strategies during a game, I'm not sure I ever have seen that with Oda (but then, Oda usually forces me or another player to reduce him to insignificance early on; I'm not sure I've seen him get close to victory). Japan is also the only domination-themed civ the AI can use moderately effectively, because of the bushido ability.
By contrast, my latest Emperor game, his slide from friend to enemy is much more paced out, and is more clearly related to my making friends with guys he considers "targets." There's more room to maneuver, and it seems much better odds that I could stay friendly with him until it's just him and me left if I wanted to.
(Not sure if I want to do that or not... if I did, it would be to set up another snipe. Otherwise, it's better to turn on him and preserve my trading partners.)
In my games with him, one or other of us hasn't lasted long enough to test his commitment to long-term friendship. Usually when I find him he's my neighbour (this is what's known as Sod's law, I think), and his desire for my territory tends to trump his other concerns even during friendly periods. Last time he showed up in one of my games, I declared war on him preemptively (trying unsuccessfully to recapture Delhi from him - not my own city, but I thought the Indians might be grateful), wiped out his army, and he was friendly from then on - but that game didn't go the distance either (can't recall why).