Servants had already begun to decline in numbers before the First World War, largely because increased industrialisation in the west meant that there were simply better jobs available for them. There is an interesting article
here from 1926 in which an American housewife describes her attempts to get her servant to work harder and to improve her working conditions, which gives a good insight into how they were treated and regarded by their employers: little wonder if they preferred to work for Mr Ford instead.
But I think in Britain at least it really changed during the Second World War. Labour shortages meant that servants suddenly became a luxury that few could afford. Indeed there was widespread talk of the "servant problem", which to our ears today seems so ludicrous, but then people really did think it was a problem. I think something similar happened in the US at the same time, with the added problem of far less immigration to the country during this period, which was a problem because in the US servants were typically poor immigrants. So the 1940s and 1950s saw a big cultural change in people's attitudes to housekeeping. That is why it was the age of the middle-class "housewife" who would do all the things that servants used to do, with the aid of wonderful new atomic age gadgets. The gadgets didn't always work out quite as planned, but it did turn out to be true that no-one really needed servants after all.
As others have pointed out, in many countries servants are still the norm - especially, I think, in the Middle East and Asia. I remember when I was at university knowing a girl from Saudi Arabia and being astonished to hear that she had servants back home, but when I lived in Singapore I found that it was completely normal for middle-class families to have "maids", almost all from the Philippines. These maids are effectively indentured slaves, usually living with their employers under strict regimes with curfews. Most have only one day off a week and many seem to spend that day either at church or, in extreme cases, working for free for former employers whom they like. They're all sending their wages back home to their husbands and children, or to siblings. I think it is common for a large family to choose one or two children who go to university or similar, and the siblings all work their knuckles to the bone to pay for it. If you ever get a chance to see the fantastic film
Singapore Dreaming, it's got some quite revealing stuff about the lot of maids in Singapore.
It is also usual for ex-pats living in Asia to have servants, and one striking thing I noticed is that cards advertising maids' services in the supermarkets almost always ask for an ex-pat employer rather than a Singaporean, because the ex-pats typically treat them much better.