On the left side of the unit action bar is an arrow, if you click that it opens some additional options.
One of them is the option to sleep the unit, which took me halve of my first game to find.
But when you are able to upgrade a unit, an up arrow appears here (which looks more like a hut). Also note that the action is only possible in your own territory (unless you certain promotions) and costs money, although I think very little money.