Part of the problem I have is that my early strategy whilst play on demi god is that because I cannot militarily compete until the end of the ancient age, I tend to build few units at all. I also fear that I lack the competence to face a demi god level enemy in an early war. I can't outsmart like 10-30 units when I've got 1-3 cities and no infrastructure or whatever.
But I do see the point, and that in that situation, being a builder is plain stupid (and I'm a builder)