Good detail; this makes it much easier to analyze your game. It seems you were in fact making some very common mistakes. That's the bad news. The good news is the mistakes are pretty easy to correct, and you'll notice the difference in you game immediately.
allin1joe said:
I never automate workers, but I tend to get lazy when managing cities. When there are a lot of cities, it's hard to keep up sometimes

I use CivReplay(mapstat or whatever that one tool is called) to help manage my empire so I'm not saddled with riots anymore, but I could use more work with MMing my cities.
Good on the workers; automation there'll kill you in the early game. With respect to cities, judicious use of the governors can be helpful in avoiding excessive micromanagement in your less important cities without hurting you very much.
How are you handling it when you do get riot conditions? Are you making specialists or using the luxury slider? A huge key to my own move up to regent was learning to use the luxury slider to maintain happiness in my capital and most productive cities instead of hiring specialists. This key becomes critical at higher levels: at emperor with no lux and two military police, cities can still only get to size 3 before they start to riot. (It's size 4 at regent, as I'm sure you've discovered.) You can't build an empire with only three citizens per city working tiles.
I think I did the following things wrong in my initial try with the Japanese (standard Arch 60% water, 5 random opponents)
1. I did a strict CxxC placement instead of spreading out a bit when I was going after resources. I probably should have used CxxxC to expand quickly to those areas, which would have also made it easier for me to fill in the gaps later. Instead, I ended up with an E shape which was dang tough to deal with. From what I've read though, a CxxC around my capital is a good idea, yes?
Either CxxC or CxxxC is fine. The former gives you on average 12 tiles per city -- enough, theoretically, to keep all of your citizens gainfully employed at size 12. Since you'll spend more time at size 12 in a typical game than at any other size, this works pretty well. I like to use a little more space to give me more flexibility in my tile use; about 15-16 tiles per city, on average. Much more than that is a waste.
Closer spacing around your capital is useful primarily for two reasons. It allows cities to share worked tiles, thus making more efficient use of you limited workers in the early game, and it allows a faster defense response to border cities if needed.
Sticking to a strict anything is generally a mistake, though -- place cities on water whereever you can rather than one tile away, for example.
2. I didn't build enough workers. I need to get a worker pump operational next game.
You don't *have* to have a worker pump. If you don't have a suitable city, or can't spare a suitable city to that particular duty, just build them whenever a city starts getting too big to be happy and you can't afford the necessary lux tax, or when a city has reached size 6 pre-aqueducts (later, also size 12 pre-hospitals) and stopped growing. I like to build workers in corrupt cities (1 shield per turn) with 2 or more food per turn; five of these will give you a worker every other turn, same as a typical worker pump.
3. I built too many improvements. Since I was the Japanese, EVERY city got a temple, barracks(because of the civ traits) and granary. Of course, when marketplaces came around, everyone had one of those too. Was it right for me to concentrate on those city improvements everywhere(due to my civ)? If I am agri, should I have a granary everywhere? If I am religious, should I have a temple everywhere? etc...
This one's huge. Prioritize your builds. Barracks are only needed in core cities that are currently building or about to build military units, and sometimes in border or choke-point cities that are in danger of attack. Anywhere else is a waste. A granary is indispensible in any settler- or worker-factory, but can usually be put off until later in other core cities, particularly if that city has a high food surplus or is going to come up against a no-growth barrier for a long time until aqueducts are built. Eventually, most core cities should have a granary, though, in order to get to size 12 as quickly as possible. In corrupt cities, granaries aren't worth the effort.
Temples are very useful, though, particularly as a religious civ, where even your corrupt cities can build them in a reasonable time frame. However, they're not critical during the expansion phase usually. Unless you need them to secure a border or a resource, save them for later when your cities are getting bigger and you need to lock up your culture gaps to avoid AI poaching. If you're scientific and have happiness under control, consider libraries instead for this purpose.
Marketplaces are vital. Build them as soon as you can in any productive city.
Courthouses are very useful in second- and third-ring cities, on average, but virtually unneeded closer in and virtually useless further out.
4. Didn't expand quickly enough in the early game. When the AI started getting close, I started building troops like crazy and "forgot" about settlers. I should have closed up the gaps created by me making my mad dash for the resources. I also had to declare war on the Chinese to get their iron(I had none, and had Immortals roaming around) before they built up too much so I was busy getting ready for that too. That left me stretched WAY too thin.
Never forget about settlers. And don't worry too much about the AI. If you give into tribute demands, trade a lot, and avoid settling cities that overlap an AI city's cultural borders, you will generally not be attacked while there's still open space anywhere on the map.
5. I ALWAYS have one defensive unit in each city. When I am in my build phase, I tend to build a spearman, then a settler, then move the pair to where the city will be. Wash, rinse, repeat unless I didnt recharge enough by building the defensive unit. So, the tip given that I have too much defense is probably correct. I have at least 1 defensive unit in each town. I need to work more on getting my settler pump city growing faster so I can afford to spit out 2-3 settlers before recharging a bit. I also think I need to get away from using my capital as my settler pump.
Build more warriors and fewer spearmen early on. There are several reasons for this. First, you can build nearly twice as many with the same production. Military police are so vital early on, and unit support so relatively high, that this makes much better economic sense. Second, in rating your military strength, the AI over-values unit numbers in comparison to unit strength, and values attackers over defenders. Two warriors impress them more than one spearman. One archer impresses them more than one spearman. 18 warriors and 3 archers at 500 BC will do more to deter aggressive AIs than would the 12 spearmen you could build for the same shields.
Depending on the situation, your settlers may not need escorts at all. (I've never ever seen an AI initiate a war by attacking a settler or settler pair. They go for the cities.) Until you get a better feel for what those situations are, though, a single warrior escort will usually suffice. If you lose your escort attacking barbs, and can't retreat, just plop your settler down wherever he is; all you'll lose is a handful of gold.
There's absolutely nothing wrong with using your capital as a settler pump, provided it has the right tiles, food being the most critical. It sounds like yours either did not, or you were not using them properly. Or maybe you lacked a granary. (You were buiding spear-settler -- 50 shields -- and in that time not generating enough food to hold population at equilibrium.) There's at least two articles in the war academy on settler factories, and just hanging around the SG forum will help you out a bit with it, too.
You can build settlers from towns other than your main settler factory if needed, in the same situations as you'd build a worker.
I have been reading SGs for a while now, and I need to get involved one of these eons. I think that would help my game immensly. It's already given me tons of tips, but I haven't been very good at implementing most of em. I read it at work, then don't play till the weekend, and I'll forget a lot of it

Old age stinks ya know
SGs are great! Reading them taught me almost everything I know about Civ. There's a monarch training game going on right now, I think, and I know there are more back in the archives (I played in one!): check those out in particular for tips. Regent and Monarch are very similar.
You'd better be older than me with that comment about 'old age'.
Good luck!
Renata