Sounds like you have to wait for him to attack, as that's when you generally get the bonuses instead of him.
If you have archer bombard on, a stack of archers can attack him with impunity (until he attacks them back of course) and do 'guaranteed' damage - depending on their accuracy rating, but guaranteed in the sense of not needing to win a round of combat to inflict it.