Regedit is a component of Windows and is used for editing the registry. If you're running Windows Vista or 7 then type 'regedit' into the search box in the start menu and it will come up. There's a risk of seriously messing up your computer when using regedit though, if you're not sure what you're doing, so if you are going to change anything then make a backup of the registry first. (File->Export...)
To see the Vanilla icons I think you have to unpack the FPK files first. Launch the Civ5 SDK, go to Nexus, and then once Nexus starts go to the Nexus tab and click on 'script browser'. Click on 'Extract PAK (*.fpk) files' and press run script. It takes a long time to unpack the files, so pass the time watching TV or something.