I'm in the middle of building two PCs right now, so glad to share what I've learned. Haven't had a chance to research as much as I'd like though, as it's due to two breakdowns within days of each other, and we need 'em asap for business.
The most important question IMHO is what are you intending your new PC to do?
GravityWave is spot on with this comment. Our primary need is reliable business machines, which can also play games--but not the latest heavy-load games, more like CIV, RTS and casual games. We already have all the software [OS etc], I/O devices [monitors, keyboards, mice etc], and cases. I've spent around $1,100 total for the new parts for both machines--incl some at a local shop [ie a lot dearer] for emergency needs.
If you need a box to play the latest games, or do significant video editing or 3D work, and want to overclock, then you can spend a lot of money--that's the usage profile which = expense.
You should read a detailed guide to get a feel for what's involved. Google "build your own PC" and you'll find plenty of good ones. If there's anything you don't understand, ask here and we'll help you along.
I've always been told that integrated anything is bad.
Dido is right, it's not bad. Not good enough for high-end use, but otherwise perfectably usable. One of my machines [not my own one] will have the XFX 630i/7150 motherboard, with integrated graphics, sound and network. It'll do the job it's for just fine, and there's a graphics card slot if we ever want to bump it up a bit. My own motherboard won't have integrated graphics, because I want a bit extra there for gaming. That's the beauty of building your own machine, you can get exactly what you need.
How old is your current PC? Technology has moved on in the last few years. I've had to get adapters etc to be able to continue to use some old stuff like IDE drives, printer and a couple of serial devices. That could affect you if your drives aren't SATA and your peripherals all on USB.
One of the first choices you need to make is whether you'll go for:
1. 64bit Vista, quad-core CPU, 8gigs RAM, other high-end stuff;
2. 32bit XP, dual-core CPU, 2gigs RAM, and less expensive stuff.
This choice mainly hinges around your intended usage.
Tell us what you want the rig for, and what stuff you already have--then we can help you better.