There is no need to restart PC if playing on windowed mode (on fullscreen you can press ctrl+alt+del and click on task manager anyway).i now had to restart my PC 4 times. .
Just rightclick on task bar (or how that bar is called in English), click on Task Manager and select civ4 process to shutdown it.
I guess restarting PC is just simpler for you.