Moral relativism is obviously anti-realist, so you will find most disagreement with moral realists. Moral realism is much less problematic than moral anti-realism, IMO, but moral relativism is a particularly problematic form of moral anti-realism.
Au Contraire. Moral relativism is firmly realist. Realist's hold that moral facts exist and that moral propositions can be true or false. Moral disagreement can be, and usually is, disagreement about a factual issue. But realism is a meta-ethical position; realism does not constrict one in characterizing what makes a moral proposition true or false.
Moral relativists can accept all the commitments of realism. They believe that there are moral facts; moral propositions can be true and false. They just give an obviously false account of the truth conditions of such propositions. The relativist believes that what makes a moral proposition true or false is the beliefs of the community in which that proposition is uttered (or about, or from which the utterer heralds, something like that). The truth conditions of moral propositions are given by some facts about the context in which the proposition was uttered. Moral facts are thus relative to their contexts - usually relative to the belief of each community. In Rome around the birth of Christ slavery was genuinely morally permissible, two thousand years later in the same place it is not. Through most of human history oppressing women has been fine, now, in much of the world, it isn't. The moral facts change but, according to the relativist, they are still facts. Note that disagreement tends also to be disagreement about issues of fact; most moral disagreement happens within communities rather than between communities. This means most disagreement is a disagreement over some factual issue.
Note that moral relativism is still obviously false. The two examples above establish that.
I rather suspect you are confusing moral relativism with a form of moral scepticism, like that described in the OP. This is understandable; most people who proclaim themselves moral sceptics and talk about the arbitrariness of morality are philosophically unsophisticated. They tend to believe that moral relativity is just what they are claiming, when it is not. Their ethical viewpoint is often, to put it bluntly, incoherent. People believe that there is no morality, but also that morality is relative to each community.
Nonetheless, moral scepticism itself is much more credible then moral relativism. Specifically, accounts like moral error theory are often compellingly argued. This is a theory that denies Integral's second premise; moral propositions are factual propositions, but all are false. There are no moral facts.
I do not think error theory is correct; I am a realist. But it raises substantive point which need to be seriously engaged. Something relativism all too often fails to do.