Several possibilities:
Recon units cannot attack at all. The unit isn't an explorer is it?
Ships cannot attack cities, though some of them can be used to weaken the city's defenses before the assault. Similarly, aircraft and missiles cannot attack cities... at least, not in the same way land units do. Instead they conduct "missions." To attack the units in a city with aircraft, select the "air strike" mission from their action bar.
Is it a siege weapon such as a catapult or artillery? Siege weapons cannot deal damage to a unit if they are at or below 25% strength. If every unit in the city is damaged below this threshold - perhaps because of collateral damage from other siege attacks - your siege weapons will not be able to attack.
Are you attacking with a Helicopter (gunship)? Gunships cannot capture cities. I cannot remember if this means they cannot attack the last defender at all, or if they can attack but just won't capture the city if they win; I don't use them that much. If the units remaining in the victim city are all defenseless (workers, settlers, docked ships, grounded aircraft) you definitely can't attack with a chopper. Those units are automatically captured/destroyed when an attacker enters the city.
Finally, even though a unit may have more than one movement point, it can only attack once per turn unless it has the "blitz" promotion. Capturing a worker counts as attacking, as does pillaging. Blitz requires Combat 3 as a prereq, and isn't even available until you research military science, so you don't see it that often (except on tanks, which start with it).
So, those are the reasons you might not be able to attack the city with whatever unit you have selected. If it is not something on this list you are leaving something out of your description or you have encountered a weird bug.
