AI units don't consider being able to actually see the tile a unit is on, just being able to see the unit itself if it is invisible in some manner. They just see an area around themselves at each step in their consideration of what to do next, starting with what they can reach in one turn, then expanding out to what they can reach quickly, and finally ranging out to a longer term objective.
I suppose I could fix it up so that they have to be able to actually see you, but then it would be nearly impossible to ever have a barbarian chase you down (maybe they never should though?)
I'd say, if you're in the line of sight of a unit, and you move into somewhere it wouldn't be able to see (like behind a mountain) it shouldn't just forget you, but it shouldn't home in on you like a heat seeking missile either.
How about, making it so that if you move out of a unit's line of sight, it treats the your last known location as where it thinks you are, and if intending to chase you, will move there. If it can't see you when or before it reaches that tile, then it should give up the chase
Also, it'd be nice if barbarians would have some sort of limit on how far from "home" they'd stray to chase you down. It does make sense that the guardians of Prstinus Pass would go after whoever disturbed them, but it seems a little silly when they'll chase a lowly scout to the ends of the earth because they can't catch him. Until his nation becomes powerful enough to kill the gargoyles.
For barbarians spawned from dungeons/villages, they should consider that tile as "home", and not wander more than 10 tiles or so from there. Giving up the chase if their quarry moves outside that range. Likewise, the aforementioned gargoyles should treat pristinus pass as home.
As for randomly spawned wandering barbs, I'd say they should not bother ever chasing down units, except as targets of opportunity. Thy should focus on lemming rushing any cities they can find, and maybe pillaging stuff.