Civ IV spies, IMO, are anything but worthless. As I've progressed through levels (currently whooping on Prince, almost ready for monarch), I find that there's always one or two civs that keep pace with me for the #1 slot, often they are my trading/religious partner, with whom I share incredible relations. So how do you get ahead of the guy who's helping you get ahead? Maintain a good relationship, have him your equal until building Scotland Yard, then boost ahead as your spies destroy improvements until the city is about to complete a production, then sabatoge it! In CivIV, friends often pose more of a threat than enemies do, and spies are pretty much the only way to deal with them.