I have avatars turned off by default, a habit I picked up when I first started forum-browsing years ago. So, changing avatars don't confuse me. I might remember which poster posted what more if I did have them on, but I know the familiar faces fairly well (especially in the Civ3 forums, but also to a lesser degree in OT, and a few prominent posters in other fora, mostly those who used to be active in Civ3 or are moderators). There's only a few avatars that I associate with users - Thunderfall with the battleship and a couple former Civ3 posters such as TheRat (capitalization incorrect).
I recently switched back to my original avatar after more than a year with a different one. I hadn't intended to switch away for so long, but wasn't often reminded that I'd switched.