Well,
they also fought in WW2,
but on the side of Germany and not russia,
because they hated the communists

There were whole cossack regiments of the Red army which flipped sides because of this.
At he end of the war, most cossack formations surrendered themselves (and often also their families) to the american and british troops (knowing what would await them if they got in the hands of stalin).
But to no avail. During the post war times lots of them (more than 100.000) were handed over to the sowjets and much of them ended either executed or dying in the Gulags.
So, well, maybe we could implement it like this:
Stalin is able to build cossacks. But if they ever, in case of a war, enter the cultural borders of the enemy civilization, they instantly flip sides