The most important things for FFH performance are honestly probably RAM and hard drive speed, assuming you have a tolerable CPU and GPU. The primary performance concern is loading times, and those are the two things that will most directly affect that.
If you're buying a desktop and have a reasonable budget (i.e. you're not looking to buy some $90 computer from goodwill), you'd be hard-pressed to find one that _wouldn't_ run FFH pretty well. With laptops, you may run into very stingy default RAM and slow hard drives.
If you get a desktop, aim for a SATA 2 hard drive, at least 7200 RPM. The amount of space doesn't matter a whole lot but it's always a good idea to leave at least a few gigs open and games are getting more and more bloated each year, so you may want to aim higher than you can imagine actually needing. Hard drives are fairly cheap, anyway, and if another $20 keeps you from running out of space for another year, it's more than worth it.
As for RAM, get 4 gigs of it and you'll be in good shape. The type of RAM you can use will be dictated by your motherboard. Some manufacturers overcharge HORRIBLY for RAM, which is dirt cheap right now - I just got 4 gigs for my laptop for $50, shipped, and that's name brand RAM.
I'm a bit of a geek to put it lightly, so I could ramble on endlessly about this stuff. Instead, I'll close with a couple direct questions to help narrow down on what you're looking for:
1. Are you looking for a desktop or a laptop?
2. Approximately what budget range are you considering?
3. Anything in particular besides FFH you'd like advice based on?
4. Would you be comfortable making simple changes (e.g., opening up your case and replacing the RAM) on your own or would you prefer to pay a little more to have everything done beforehand by the manufacturer?
For the record, I play on my Macbook Pro running Windows XP, 2.2GHz Core 2 Duo (sadly the second core sits idle with Civ), now 4 gigs of RAM, a small 5400rpm SATA hard drive and a 6800GT 128MB graphics card. FF runs pretty well, although turns can take close to 30 seconds late game and I don't play huge maps. Upgrading from 2 to 4 gigs of RAM felt like I had a whole new machine.