I think Diablo 2 was better than 1. Civilization has done right by me as well. Age of Empires, the whole list is spotted with franchises I think made successful sequals. (I mean got better not made enough money to qualify as successful.)
I think alot of the problem is that say when Civilization first came out, you had nothing to compare it to. It was new, it was groundbreaking, and it was good. (*) After part 2, the idea isn't "fresh" anymore. Basically a sequel cannot draw the same reaction to a game that you have no previous expectations of. It is definatley possible to make a poor sequel to a game and everytime you change the core mechanics ou risk pissing off your fanbase. A new idea coming into a franchise is risky in that sense as most fans tend to get sunk into this idea that change is bad.
However, If Civ 4 had not evolved much from Civ 1 or Civ 2 what would be the point in making a Civ 4? You see it is a lose/lose situation for a developer. If you don't change much but simply update the graphics you get called out on pulling a Madden. (Madden'04,Madden'05, Madden'06, etc.) Fans get mad that you don't update anything but continue to spew out sequels that fans feel compelled to buy. Some who skip and pay every other sequel, or similar, is the smart consumer however this loses you money. So it would seem the answer is to try and make the game better by adding new stuff and taking out the stuff people didn't like.
Wrong, now fans are saying you went the wrong direction with it. Because they liked this part you removed, or they hate this part you added. Because the game didn't go in the direction they would have taken, they are disappointed from their expectations. I have been guilty of this before myself. This basically ties right back in to the (*) above.
But overall, I think that nostalgia for the original is stronger than a good quality sequal. That is why we can still go back and play earlier versions and to top it off say they are better. Have you played Civ 1 or 2 recently? Are you really going to say that neither 3 nor 4 is better than both of them? Even if you disregard modability, the core games have improved. Not only has balance improved the added features have grown better, the AI is smarter, etc. And IMO modability should be included as they were big features focused on in 3 and 4.
All that said, I will concede to the shooter titles. But the consumers have proven they can spew out crap and people will just lap it up and ask for more. MW2 is a good example of this. I loved MW1. MW2 is just mediocre but people act like it is cocaine. Hell, I will even admit that sometimes I go crawling back to it. But I suspect it is because there just isn't much else out there. Honestly, I prefer BF2142. But no one I know plays it because there reasoning is "Why don't we just play MW2 instead?"