Actually I was playing byz as Emperor and I was struggling to wipe out the arabs, but the best tactic was to wait away 1 turn, they moved the whole army out of Damascus, leaving there only 3 settlers. A skimirsher took the city and arabs were gone. Then they had units all over the region (from the barbs?), and they had units without city for a few turns, and I made peace with them. Jerusalem became independant, and a few turns later it became arab by culture switch. A few turns later, the Bulgars appeared, moved all their army south to attack me, one horse archer mercenary took their capital and they were gone for good. Eventually the arabs declare war again and conquer Jerusalem as any unit in the area would desert to them, but it can be easily conquered back in this case if it's just with one horse archer for example.
I reloaded and relived similar situations, e.g. with Tyros becoming independant I could reconquer it quickly and leave the arabs still without city when I destroyed the Bulgars