Eh I don't get the impression that housing in New York or DC is vastly superior for it in comparison to say Germany. It sounds more like a lot of rip-offs. It is not like that there is no renovation and development going on in Germany. But of course you have to balance regulations with initiatives for investment.
Why do you think it is a 'huge failure'?
I can't speak for NYC, but here is why housing in DC is out of control expensive:
1) There are regulations that artificially limit housing supply within DC itself. For both national security reasons and traditions, there is a strict height cap on buildings within DC. There are no true residential high rises. Since DC itself is actually kind of small (physically), there isn't much room to build particularly dense housing. This is especially problematic for anybody seeking an apartment larger than one bedroom. Families face a HUGE supply squeeze.
2) DC's public transportation system removes incentives to live farther away. Unlike virtually every other subway in the US, DC does not sell monthly passes, and charges by distance and time, rather than a flat rate. If you live farther away to save on rent, your commuting costs will increase, often at a rate larger than the savings in rent. It's often actually cheaper to DRIVE, even with parking and insurance costing an arm and a leg. I pay over eight bucks a day to get to my office and back, as does my wife. That adds up!
3) DC, and the surrounding area, is almost exclusively white collar work. There's no manufacturing sector to speak of at all, so the people looking for housing are high wage consultants, contractors, academics, lawyers, etc. Developers have no real incentive to chase any other kind of demographic, and prices skyrocket.
It's a perfect storm. Young families who aren't high wage earners tend to leave pretty quickly. It's a lovely place if you are ambitious and single (or a college student), or absolutely loaded, but it sucks otherwise.
The distinction between "Middle Class" as a cultural identity and "middle class" as an income/wealth stratum is really interesting.
...that's all I really have to say right nowsorry.
Yeah, I agree! I could be persuaded a lot of different directions here, but I'm inclined to think it isn't JUST about purchasing power.