First problem is units. I believe I seen in discussions here that to reliably conquer a city you need like 10 units. If you build each unit for 4 turns, that's 40 turns your starting city doing nothing else? Maybe a bit less if you also burn all your gold. So, I assume you may want to build some infrastructure first, but doesn't that mean that you'll have an army only closer to the age end?
Second problem is the distance. There are some rare occasions of AIs starting pretty close, but on standard maps you usually have a distance of 2 settlements between your capital cities. So, even if AI settles in your direction, you also need at least one settlement before conquest will start making sense?
You look for and stack different bonuses for your benefit. I'll give example of my current game.
I decided to do some Antiquity warfare, so I chose Continents and Islands map, to have some land area, and paired Bolivar with Persia - strong military traits with support for expansion and economy, awesome traditions.
Then Mementos: Inscribed Bullet for 1 Militaristic attribute, and Scythian Battle Axe for gold from every commander xp - this extra gold does come handy.
My closest neighbours were Rizal Of the Maya, 12 tiles between our capitals, and he did not settle towards me, so we're keeping it neutral; and Lafayette of Aksum, 18 tiles between our capitals, but he settled one town towards me, 12 tiles away, and I settled my three towns in his direction. Close to me was a commercial IP Meroe, which I befriended winning the race with Lafayette, so he got miffed about it and very conveniently denounced me, and later DoWed me.
In the meanwhile, civics wise, I went Discipline for the free commander, Spada for the troops building boost, then Discipline mastery. My unique commander got his first promotions and some respective gold by fighting a hostile IP's units in the north, and got back just in time to repel Lafayette's initial attack. I did not disperse the IP, though, as it was Expansionist one, so I thought it would be better to befriend it.
First celebration came in just in time to stack Kara from Spada for +50% towards building infantry units and +30% towards the same from Despotism government bonus. Infantry units took mostly 1 turn to build.
So it is turn 51 of this standard Deity game, I have 1 city and 3 towns, 2 scouts running about, 1 commander, 10 infantry units and 2 slingers, and my invasion force is 3-4 tiles away from the nearest Lafayette city. He does not have walls yet, it's just the terrain that's a bit tricky, but he should fall without big problems. I have +2 war weariness advantage form Bolivar and militaristic attribute, and Gates of All Nations are 7 turns away, I have hardly seen an AI building it early, so I should be safe here.
By the way for the movement, your commander can speed up the movement for more units than he can pack. Walk 2 tiles out of 3, unpack a unit forward and walk 2 tiles forward, pack a lagger from behind, walk the last tile with the commander. And so on. This is just in case of unobstructed terrain and absence of enemy, but it does help to cross distances faster.