It is just an increasing number of unhappy citizens, there are no other indications that I know of.
On the easiest levels it is manageable, as you say, with luxuries turned up, temples, Wonders etc, but on the higher levels it is much harder.
War weariness seems to me to increase in...