You have absolutely no control over puppet city production. They will build just about anything they want including armories, walls, etc., although I'm still trying to figure out if you are responsible for having to pay the maintenance costs on everything.
So far it looks like the buildings that they do build do have an influence on your empire, as I've seen them complete colosseums and other happiness related buildings which seemed to increase my empire happiness as well.
They will not build units or wonders, nor will they ever build a courthouse. If you annex, is it up to you to build a courthouse. If you don't, you can build whatever you'd like but the city is severely hampered until you do so. Also, you cannot rush courthouses with gold or a great engineer.
If you click on a puppet city as mentioned a window pops up asking you if you want to annex, so you cannot enter the city production screen at all until then. You can however treat puppet territory as your own (including upgrading units inside the borders), and even place a unit in the city tile. You can also control city defense by choosing which targets to fire at if an enemy gets too close.
If you're wondering why they chose the icon they did for puppets, think back to the old marionette puppets that had strings attached to their limbs. There was usually a wooden cross mechanism above that helped puppeteers control the strings.
http://sp.life123.com/bm.pix/puppet2.s600x600.jpg
The chain means that I've annexed the city, but haven't made a courthouse yet.