Forts already have a pretty decent defensive advantage, no need to pour more worker turns to make a fort a "castle" and give it an even higher defensive bonus. Fighting will become impossible if the AI ever built a successful Maginot Line of these "castles."
Besides, the medival forts look a lot like castles anyway, like others have said.