Is it possible to do this? It becomes frustrating every time I research a new tech and have to go through each city and add the building.

If you hold down the Alt key and select a single city, it will select all cities you own. Then, press either Shift and a build item to send it to the end of the queue or Alt and a build item to send it to the start of the queue (pausing other buildings). I use it quite often doing research cities--a large number of small but self-sufficient cities that use limited hammers to research.

Wow, thank you!

Also, if you hold down Ctrl when you click a city, it'll select all your cities on that continent.

while in the city screen, press [CTRL+#key] to save the production queue. This queue can be loaded in any other city by pressing #key... (civilopedia)
#key means a digit i think :)