The term originally meant adherents of the old Roman religion and came into use in the later fourth century CE. Fashionable Christians in the cities of the empire looked down on them as dreadful backwards bucolic types who lived out in the boondocks and had religious views to match. Of course there was an element to this of deliberately turning the tables, since in the past adherents of the old Roman religion had adopted precisely that attitude towards Christians. So it was a reference to the rural areas of the empire and to Roman religion, rather than to areas outside the empire and Germanic religion.