Those are some very interesting observations that you have made about the AI, this is really helpful stuff, as I am too tired to notice alot of these patterns. At htis popint a lot of the stuff in the game is still in the let's try it stage, many things (like ghoul farms) I threw in quick as an idea, and probably copy pasted over stuff that I did not want. (Like the fertile wasteland bit for G. Farms.)
Lets break it down some.
Roads:
This is one of those AI features that bugs me, more from how I want the gameplay to go, more than whether or not it is a good move.
I tried to make roads in such a way that the AI would no longer spam them, increased costs and I even added -1 food at one point, it may be that the upgraded versions don't have that negative so it goes back to it's old spammy behaviour.
I really want roads to be really expensive, and really powerful but fragile. The idea that connecting the wasteland together is a massive logistic undertaking, keeping them free of bandits and such would be really hard, I want them to be used in an absolute minimum way (because they are so expensive) but the benefits of having them to be massive (the connection of bonus resources to every city is very valuable, as it leads to those building benefits from resources).
In many ways I want safe road networks to be like the pinnacle of a wasteland society, connecting your empire together is hard work but massively rewarding. I need to get the pain and pleasure mix for routes just right, and is also linked to those other factors like the resources boosts, etc. Also getting the AI to value them intelligently and use them sparingly is going to be the challenge too.
It may be that I need to increase the negatives 'on tile' for roads, so that they only go where needed, rather than everywhere..
Buildings/Units:
I have noticed several odd 'valuations' by the ai when it comes to unit building, for example I saw it build stuff like a fighter when it had access to plasma armed power armour, and the turn time was negligible, so why not build power armour?!?
The Super Mutants are an exception right now, as they are an unfinished concept when it comes to their mutant units. They need to be 'super' but this means they need to be limited in some way. I have yet to come up with the best limitation, so for now I am not that bothered by their failure to work well.
The more pressing concern for now is that non of the AI factions are much good compared to the player.
The fact that units seem be of a higher value than buildings is a problem, as you say, it may be a simple matter of adjusting the ivalue and ipower and aiweight type tags (I think that is what they are called...) so that buildings are viewed as greater value than units, so that they get built first and the city then becomes more powerful allowing it to build units in good order.
We would need to do some side by side reviewing of those tags for buildings and units and see if the units are disproprotionately higher than buildings, if so it may just be a simple adjustment of the two relevant values. (That would be nice right?

)
Trees:
I think tree chopping was always a default AI move, with our new additions this is a pretty stupid move. For now we would probably have to 'turn off' tree chopping, or make it massively rewarding, so the AI at least gets something out of it.. Chances are it will use it early though and waste any benefit from it.
Perhaps the best solution is removing tree chopping for now, as trees have an ever increasing usefulness (with the tree hugger promos and logging improvements) thn chopping would only exist for rad weeds (jungle) which is just designed to be a nuissance anyway (or at least that is the future plan).
AI Expansion:
I am fairly convinced this is currently a nail to the head for the AI in terms of being competitive.
I don't think it has anything to do with settlers cost and such, as they aren't that expensive, and I know the AI builds them.
I think the current AI has something like a hard coded rule that says escort a settler with 2 units (or something similar to this) which is fine for a civ game as barbs are generally weaker or equal to the civs troops and they aren't that common, so the AI gets the job done and founds plenty of settlements.
I think also an AI settler stack has a thinking of 'take no risks' much like a worker, if it sees a bad guy, the stack is programmed to run away.
In FTTW it is a very different story, barbs are everywhere and they can be much stronger than civs.
So if you have a settler stack, with two defenders, it sees risks everywhere at it's borders and runs away, it repeats this cycle for let's say 20 turns, concludes it can't do anything and disbands the settler.
We need to make the ai take a very different approach to settling compared to civ.
Something like:
Take half th units from the city where the settler is built (most AI citys have 5-20 units stacked up) and add them to the settler stack.
Ignore dangerous units up to total number size of stack -3 (settler plus two units) (or move X squares outside of border, X being stack-3. )
Move to settlement site and found.
The idea being is that the AI/Player can afford to lose many of it's units, if the end result is a new settlement, as borders will protect it from animal attacks and a cities development will soon allow for even more units to replace the few that were lost, and the new city equals a big net gain in time.
It would probably also be good for the AI to add a worker to it's settler stack, so it canstart developing the area around the new city as soon as it has the funds.
I think the only reason it expands right now is because someimes a good site opens up inside it's borders or next to the border, we need to get that sphere expanded to about two or 3 squares, and in terms of the Vault civs, as far as it takes to get the next Vault.
Another approach may be creating a culture improvement that has borders around it (like a new city does) and then make the ai workers build 'culture paths' to potential new city sites, so that the settler is protected from animal threats.
It could be something cheap and quick to make, like a 'Patrol Lodge' or something..
Both solutions would need some kind of AI work though I think.
At the moment I had to switch off workers ability to leave borders, as the Superfort AI was seeing them suiciding trying to build forts outside borders. With some tweaks to that code, we may find a result we need.
Improvement Balance:
You are right, watchtowers should connect resources (just like forts). It should be working, I think I used it before...
Your comment about the choice between cheap improvement+Upgrade or Expensive improvement is an interesting one...
I wonder if we should consider removing the build option for later improvements and making them only improvable, then increase the time it takes to improve, so that civics with an improvement upgrade speed increase become more valuable at times when infrastructure needs improvement..
Not sure if that would make it better or worse for the AI...
They won't upgrade if the city is trash anyway, but if cities are not using their tiles because they are trash then the improvements themselves would be meaningless regardless...
The Ghoul Farm was really a half finished thought that I never went back to as I got distracted by more general things. (Like the SM's, Ghouls are a faction where I skipped ahead of my plans and added faction specific content before finsihing general design work)
So chances are it has loads of stupid mistakes like leaving in fertile land as a prereq or validator. It should be Rad waste and Feature Fallout that are two OR validators.
The health think may be from our switch to health as yields, I may need to move the health boost to the yield section, as yield heatlh was added after the Gfarm. (you can try this yourself by adding in the 5th iyield tag (4th is happiness) if you feel more productive than me!).
Village Vs Farm:
Eventaully I sort of want Maxed villages (calling them something like Suburbs/Districts as an upgrade path above towns.) to be the best improvement for standard flat ground. (ruins should probably benefit more from scav camp path, and hills from Mines (not least for the potential bonus resource pops) That will just be continuing to crunch the numbers and settle on the right ones. All these things are still very much WIP.
These ultimate villages should take a long time to get to, and be locked out by high end techs, but like a road network they are representative of a pinnacle of a recovered society.
These would also carry bombardable defense bonuses, so we can end up with some cool urban warfare and urban warfare specialists. (hopefully)
Resource Buildings:
These will continue to become more valuable as we move forward, right now Junk is only really good for getting access to melee troops, but we will continue to find places and buildings where junk (particularly multiple junk) will be more valuable and benficial.
The current Building bonus boost settings (that's some fun alliterration) are ported over from when each building could only use one resource, I now need to add in all the extra and alternate resources for each one. Like, for example, anything that benefits from building materials, could also benefit from junk.
I also eventually want to increase the resource/production system, so that 'base' resources are required to build quantities of production/manufactured resources which would then be consumed in the production of units and maybe buildings and such depending on how far we can take the system. (if we can take it at all)
So the overall fact that the AI prioritises resource buildings will be a good thing in the long run, as these will provide many more bonuses beyond the value of the building itself.
Platy's Manufactured Resources
That is a bare bones python version of the sort of thing I mean, but it would be an sdk version (hopefully) and much more advanced and deep. If I can convince people to help me do it

)
The Culture Flip Conundrum:
This may well have been achieved by a spy. I think spies can cause rebellions, which may in erm have made it susceptible to a chain reaction leading to a culture flip. I don't know for sure though.
The rebellion would cause a loss of culture tiles, which would push an enemy border up to it, and then... 'flip!'