Yeh, the rebellion units just kinda parade around as they see fit. Generally they like to attack whatever unit they see, against which the AI deems it has a statistical advantage.
It happens to the AI opponents, too, especially in periods of war. A few days ago, I was attacking China (I forget who I was) and had plans after taking one of his satellite cities to take his capitol. I took the first city, then his city to the east spawned 2 knights. So these things come running at me because I have units out in the open, and before I could focus fire them down, they'd killed an archer, a catapult, and caught one of my swordsmen so he couldn't get away from between them at one point. He died valiantly, though. Then 2 more spawned at his capitol down south, so I avoided them by 4 tiles circling around up north and liberated a city state from China instead. Totally messed up my plans, his insurrection killed 5 total units before the war ended. He had succeeded in killing only one, himself.