In my opinion, the only reason for the AI to seek techs getting them into the Modern Age is to attain a Victory Condition qualification. The UN being one, launching a Spaceship another. If both of these VCs are turned off, the priority of getting into the Modern Age for the AI is greatly diminished. The AI will already be able to calculate if it will stand a good chance of doubling your culture AND breaking 100K for a Culture Victory. In my games, they never stand a chance in the Culture Victory category. So for my games, if SS, UN, and culture are turned off, the only reason for the AI to strive for enough research to enter the Modern Age is for the Military units.
Let's say some of the more agressive tribes decide they want MIs. That's good news for me, as it means more competition. But what else, besides MAs are they there for? So military would be the main reason or cause to journey there.
For a while, I would not play with the UN enabled, because to me that lead to a better game. Then I started shutting off the Spaceship, because it prompted the AI to get off their butts and build a military to fight with. The reason for this was because they had more gold from lack of research. And they weren't trying to steal Wonders, as once I'd gotten the Theory of Evolution they were left in the dust. So their bigger builders were free to build military.
Keep in mind that the larger AI nations were already embroiled in war, with me or others for quite a while. I'm not one to let good Cavalry go to waste. So with priorities already on war or defense, the knowledge that they can't get the Cultural Victory, the UN Victory, or launch, the AI has to determine whether getting into the Modern Age is worth it or not. That's when I start gifting technologies to the slower nations, hoping they will push forward and put up a good fight. But a lot of "less-aggressive" nations won't.
I think I may have gotten off-track a bit, but I may have answered your question. Removing the reasons for getting to the Modern Age will affect the AIs drive. At which time it will most likely resort to the base strategy of war.