Now that I think about it, this isn't really the case either. Remember that less than one percent of the population made it into the vaults when the bombs fell. That means the country (and the rest of the world) were severely depopulated. It's hard to rebuild when you don't have the people to do the rebuilding.
Also, there were definitely some attempts to rebuild that we can see, it's just that the people left (aside from the Brotherhood of Steel) simply do not have the know-how to make any significant progress. Think about it: if all of our industrial, communications, and logistical infrastructure/technology disappeared tomorrow and all the people that knew how to rebuild all of it died, how much of all that stuff could you rebuild on your own? Hell, how much of that do you think the people in your community could rebuild on their own? We can reasonable infer that no one is really left to design and build the advanced technology of the past by the weapons we see used. Any firearms or advanced armor we see being used are just leftovers from before the war or manufactured by the Brotherhood of Steel (one of the maybe two or three factions left that can actually build that stuff). Any new weapons and armor built by the survivors are extremely primitive improvised weapons and armor like spears, knives, and leather armor.
There's also the fact that humanity is in a state of total social collapse. Humanity is no longer organized and is just a smattering of scattered settlements that are just trying to secure enough food to survive and protect themselves from other humans who keep trying to kill and enslave them. There is no organized large scale government to direct and manage any serious effort to rebuild.
As for the vegetation not growing over all the old buildings: It simply doesn't grow. Every location we have ever seen in the Fallout universe, with a few exceptions, have always been shown to be barren wastelands where hardly anything is capable of growing. Even the Capital Wasteland was like this. Remember that one little bit Three Dog does when talking about Oasis. He is genuinely shocked and surprised that a place like that exists in the Capital Wasteland. That would seem to indicate that the people of the Capital Wasteland (or anywhere else for that matter) aren't accustomed to seeing living vegetation in the wasteland. I mean, it is called the wasteland for a reason.