You really think that polytheistic religions aren't violent towards outside denominations? What about the Japanese campaigns against Christians, the treatment of missionaries by, well, pretty much every non-Christian tribe they met, the anti-Jewish pogroms by the pre-Christian Roman Empire, etc.? Religion tends to be violent towards outsiders, because people tend to be violent towards outsiders. It's simply the overall success of the two Abrahamic religions, Christianity and Islam, that gives the appearance of some sort of superiority - through force or persuastion - possessed by monotheism. And that success has next to nothing to do with religion.
If you look long enough you can find violence nearly everywhere, of course. But I do think that overall monotheistic religion have been more violent. You are right that groups of people will be wary and often violent towards any outsiders, but when considering the role of religion in facilitating or providing motives for that
there is a difference between polytheistic and monotheistic religions. Polytheistic religions can more easily incorporate foreign divinities into their pantheons, or find "similar" gods. And because they do not invest everything on the characteristics of any single divinity, and they are already used to a diversity of rituals, they can be more accommodating to differences in worship and ritual towards any particular divinity, more curious and willing to learn about new ones.
Monotheistic religions, especially the abrahamic ones, will take any other different monotheistic religion to be dangerous heretics, because the mere existence of another monotheistic religion is a denial of the correctness of their own rituals and practices: there can be only one! And they will take any other polytheistic religion to be false and misleading, and at the very least feel free to attack and insult it. My impression is that the anti-jewish pogroms in the roman empire happened because the jewish would happily denounce any other religions, instead of just going along with the motions of imperial worship
and basically ignoring the whole thing in practice as other sects did. But I'll also say that trying to force statues of other "gods" on their temple was pushing it!
As for the japanese, it was a political thing, as the japanese realized that missionaries could be laying the ground for military attacks. They tolerated missionary activity well enough, allowing the number of converts to swell even despite the serious lack of missionaries trained in japanese, until the missionaries own spats (Franciscans vs. Dominicans) and portuguese preference for allying with and selling weapons to christian Daimyos showed that the possibility of political interference or even outright attack using a local "5th column" was real. An attack could not succeed, of course, the portuguese and the spanish were overstretched as it was, but interference was certainty on the cards: the mere fact that the portuguese king, not the pope, was the one appointing the bishops and priests for the east made it clear to the japanese that they were political tools of another polity with bases nearby. they might not have realized that detail initially, but the spanish Dominicans who arrived later made sure they did!
Oh, and missionaries' success with non-christian tribes was mixed: some attacked them on sight as they did all strangers, some accepted them and the missionaries went on to become essential bridges between colonial governments and tribes. In some cases they even set themselves and "their" tribes against those governments.