How to beat settler:
Generally alternate between building military units and settlers, while training a few workers and scouts/explorers. Let your workers build whatever, but make sure you have horses and iron. Send the settlers to the blue circles (if there aren't any, use your best judgement). When all of the land is built up, train only military units for about 20 turns and attack the weakest civilization. Recover as much as necessary and attack the next weakest civilization. Also, found a religion, spread it with missionaries, and adopt organized religion to improve your culture. Avoid going into debt.
How to beat chieftain:
Do it the same way, but with two military units defending each city, spend about 30-40 turns preparing your city, build walls in cities where you might be attacked, and bombard your enemies defenses before attacking.
How to beat warlord:
Playing as Japan and trying to conquer India, got 1 city, have a bad economy... (doing OK, work in progress)