The Civilopedia's statement that a city without water access can "receive up to +6 Housing" from an aqueduct is misleadingly phrased; what they really mean is that a city without water access will be "topped up to +6 Housing."
If your city is founded with access to "full" fresh water (adjacent to river, lake or oasis), your city starts with 5 Housing from water (base 2 Housing from water that every city gets, plus 3 Housing for access to the fresh water itself) and all you can get from an aqueduct is +2 Housing from water (for a total of 7 Housing from water).
If your city is founded with coastal fresh water access, your city starts with 3 Housing from water (base 2 plus 1 Housing from the coastal water) and you can get another +3 Housing from an aqueduct (for a total of 6 Housing from water).
And if you found in a location with no fresh water access at all, your city starts with just the base 2 Housing from water that every city gets, and you can gain +4 Housing by building the aqueduct (for an eventual total of 6 Housing from water).
So that means it's better to build aqueducts on coastal cities and cities that are on top of plains or deserts surrounding it?