I know in civ5 if you changed a context xml file this would happen wrt the associated UI.
I stumbled on this when following William Howard's tutorial(s) on UI modding. Freaked me out, really.
I think (but am not positive) you only get it withUI lua as a general rule. I seem to remember some method to hotload a change to a 'gameplay' InGameUIAddin, but never tried it.