I don't think you need to worry about stacking fort AOE and citadel AOE, they should always just use the strongest effect without stacking. So if forts were 5 and citadels were 10, it would just deal 10 damage. That's a pretty consistent pattern across the game (think of Medic I and Medic II from different units).
There will be a stacking problem and it is unknown whether the AI will be able to take it into account.
If you take this layout Citadel +3 forts, then the tile with raw stone will always receive -10..-30 damage until the last fort is destroyed. If the attacking AI stands as an infantryman on this hill, will it be able to calculate that it needs to destroy 4 defense points instead of 1 (Citadel) to neutralize the aura?
In addition, after destroying the Citadel and 1 fort, the attacking unit will receive -10 damage from the aura of one of the two remaining forts.
If the forts' aura is activated only when a unit is present, then this will only mean that the attacker will have to additionally suffer from the forts' aura the entire time he is trying to kill the rotating fort garrison.
In addition, border forts cause damage in enemy territory, where he should be able to heal. It is possible to cause damage with the aura of a Citadel on enemy territory, but to do this, both empires must install Citadels after 1 tile so that the intermediate tiles change their owner - that is, two generals are required for such a special action of the Citadels. The price of 1 fort and two Citadels is incomparable.
It seems to me that adding an aura to forts will further reduce the AI's desire to attack Citadels in such dense defenses.
Here the Citadel of Japan to the left of the arrow is capable of causing damage to 3 tiles in Zulu territory. But it took 3 Great Generals to achieve such an effect. First, Japan set up its Citadel, and then the Zulus set up 2 of their Citadels.
But, if Japan places the proposed fort on the tile indicated by the arrow, then this fort will be able to deal damage to a unit in the Zulu Citadel. How should the Zulu defend now? Any garrison of them will automatically receive 10 damage regardless of combat strength. I very much doubt that AI Zulu will be able to solve the problem of destroying such a fort between the two Citadels of Japan. It must be remembered that when attacking from their Citadel, the Zulu mounted unit will not move to the fort tile and thus the attack is only available from below, so that the tile can be plundered immediately after the kill.
The Ottomans, having built a fort on a hill, will be able to inflict 10 damage to the Tunsberg garrison every turn. An infantryman with the Stalwart promo and +60% defense (50% fort, 10% hill without forest) is very difficult to kill. A city is unlikely to be able to deal 10 damage to such a unit. And in an earlier era, the city would not be able to deal even 5 damage to the Conquistador.
Technically, the Ottomans can install a new Citadel on this hill if they build a farm on the site of the current one, and inflict 30 damage to the city garrison. But the AI does not move Citadels in this way.