@MortalD
Yes I am sure!
Russia went into battle with me but I took over moscow. Because of that, another russian city became the new capital (perhaps that's why it worked!?).
Then, after a period of ceasefire, my allies and I went into war with russia to wipe them out.
I bribed the nearby CSs (after I donated some units to them: see above) and they went into war with russia, too.
After taking and burning down the last russian cities on my continent, the capital moved to a small island with a nearby CS. I bribed this CS too, weakened the russian capital with my battleships and the CS conquered and burned it down.
So technically it wasn't me who finished the russian player...