I think that sometimes barbarian units disappear in the same fashion that they appear in areas you can't see. So if you lost sight of them, they may have just vanished. I don't know if being triggered by an "event" affects this. Just a guess, really, not sure how it works. My only basis is...