In the ancient era, archers are the key. I remember doing some serious plowing with one hoplite, two companion cav, and just four archers to back them up. Like, serious kiss-your-empire goodbye damage.
Now, as you progress, siege equipment can cut through a city quickly, sure, but archers, bowman, or crossbowman can pop in and attack a city right away if you plan your attack right.
Like docbud said, terrain is really important. Taking a city on an open plain is easy, but if he's dug in like a tick in some hilly terrain, with some rivers acting as natural bulwarks, then you best come prepared.
Tactics = Situational.