I've used them, but only along specific tiles that make sense, and against a neighbor that continuously remains as "annoyed" no matter how you treat them diplomatically. You really have a lot of factors to weigh before you waste your time;
Do you plan to attack soon farther from this point, or will it be a forever defensive point against an always "annoyed" enemy? A well planned attack by you will render your Fortress/Barricades a waste of time.
NEVER build them along a border that is in question by cultural superiority, especially if your opponent has higher culture, or simply doesn't "like you".
Remember, if you build a Fortress and Barricade, it has to be protected, and this should be done by your highest defense unit, but they're "stuck" there.
If you need and can afford time for a Fortess, then ALWAYS build a barricade.
Ultimately, all these bonuses do to is turn your own grasslands, plains, & hills into mountains defensively, but you have to do the math, whether it's worth it.