I've tried investing everything in defence but it's still not enough. Is there likely to be anything I'm missing? Is it all down to luck? What are the advantages that AI have to make their army three times the size of mine by turn 60?
On Emperor AI starts with 3 warriors as opposed to your 1, so it is only natural for them to have an army 3 times the size of yours from the start

It is also the first difficulty, where AI gets a free settler upon founding their first city. They also have bigger research and production bonuses. Your task is to survive the beginning which may include an early AI rush or an unlucky barb onslaught from a triggered camp or even more camps.
Speaking about barbs, try not to let a barb scout to see your city (it gets an exclamation mark on top when he does), or don't let that scout to return to his camp, because that triggers a barb raid. Destroy or deflect him long enough. Triggered camp produces barb units at an increased rate, which could make your life more difficult. In any case, barbs can't take your capital (but they will raze your other cities if you let them).
Speaking about AI, it is easiest to avoid early war by handling them diplomatically. On standard speed they will accept your delegation on the contact turn (but likely not later), and that gives a positive modifier. Satisfying their agenda, signing open borders, sending trade routes or gifting will boost your relations even further and when they turn to friendly (smiley face) they will likely accept a Declaration of Friendship which means mutual immunity from war. Without a DoF they can and will happily attack you if they are stronger, even when friendly. If you have a DoF, you have guaranteed peace with them for 30 turns. Extending DoF on the turn it expired is also as good as guaranteed, unless, maybe, you're very close to victory.
Or else, if you have a close neighbour, often it is very profitable to conquer them. Found another city asap, and with two cities, or maybe three max, you can start taking over the world. Try to build units using respective policy cards for faster production. Agoge for early melee and ranged, Maneuver for early light and heavy cavalry. Three-four warriors and then three-four slingers into archers should help you to survive early attack and make your first conquests, then add chariots and horsemen if you have or can trade horses. Upgrading earlier units into later ones with gold is more effective than building those from scratch. When cities get walls, bring battering rams and catapults or more modern siege for more modern walls.
Cities strength is based on the best melee unit that AI (or you for your cities) has ever built (including ships), they fire based on the best range unit they've ever built.
If you're at war, don't forget to pillage your enemies for gold, science, culture and faith.
Builders are also vital for improving your land and getting early eurekas to help you progress faster in tech.
Don't forget scouting - scouts get all sorts of useful stuff from tribal villages and make contact with city states - they are amazing. Try to adapt your game to make use of city state bonuses.
Civ 4 has slavery for speeding up production, Civ 6 has chopping and harvesting of resources. With governor Magnus this becomes exceptionally strong. At the start, if in emergency, you can just chop out your starting army, settlers, or essential districts. See other governors and their promotions too, they also have very useful bonuses.