I'd say that since the PC was junk when you got it, try carefully drilling out the rivets. You lose nothing if you fail.
Yeah, that's what I'm going to do. I don't really need any part of it, but it's just fun to play around with it, and perhaps I'll try to make a computer if I'll find another one in the thrash.