It is fairly simple, actually. To do this, you will need a text editing-tool because the files that decide the city names are XML files. I think you can use Notepad to do this, but I would use Notepad++ instead, as I think it is easier to use for these types of things.
First off, though, since you are new to modding, I will give you some advice. Always make backups of the files you are going to change, just in case you make an error. If you are going to be doing a lot of editing in one particular file, save a separate, fresh copy every so often so that you won't be overwhelmed with errors, and then have to try to fix 20 errors at the same time. To make changes to the default game, simply do all of your changes in the "Assets" folder in the main game directory.
Also, I am pretty sure that there is a way to program a shortcut to load up a mod without having to load up the default game, then load up the mod. If you search the website, I bet you can find it pretty easily. This could be very helpful if you want to get more into modding, and want to keep a fresh, working version available, and then build on it with your mod.
That said, city names can be changed in two different ways. The first way is through a file called "CIV4CivilizationInfos.xml". There is a section for each nation, and for each nation there is a bunch of tags for city names, looking something like this:
<City>TXT_KEY_CITY_NAME_WASHINGTON</City>
The first city in the cluster of "City" tags will be the capitol, and each city after that will be founded by the civ in descending order. To change the name, you must simply replace "TXT_KEY_CITY_NAME_WASHINGTON" part with the with new name. The reason the cities are named like this in this file is for language purposes. Those long names are actually linking over to a different file to find the different translations of the word. It is fine to replace them here, though, with your own names. This leads to the other way to change the spelling. You probably won't need this other way, but perhaps you simply want to change the spelling on a city, in which case, this is just as acceptable.
The second way is though a file called CIV4GameTextInfos_Cities.xml. In it, you can see all the cities' names, and how they will be translated when a player is using a different language in the game. Here you simply have to find the desired city, and then change the spelling in the correct tag that corresponds to the language you use in-game.
Happy modding!