Ahh the old days of the failed console wars. That generation between the NES and the PS2 was absolutely littered with the corpses of failed consoles... why was that? Turbografx 16, CD-i, Atari Jaguar, Neo Geo, Saturn, DreamCast, Sega CD, 3DO... so many. Now we have Nintendo, MS, and Sony. (Remember when Nintendo and Sony were supposed to team up for their next gen console? Oops.)
Not sure if any of them were the worst though, some actually had good games such as the Neo Geo and Turbografx from what I have read and remember. (I never played them but IIRC those two had a lot of the popular arcade games on them. I really wanted a Neo-Geo as a kid actually, those SNK arcade games were loads of fun. Too bad the games themselves cost hundreds of dollars and were ridiculously huge.)
I actually forgot about the Virtual Boy until this thread, and reading up on it that seems to qualify both in terms of bad games and completely god-awful playability. I mean rarely is a video game console so bad that it actually induces physical pain to actually play.
If I had to pick one--besides the Virtual Boy--I'd say it's a toss up between the 3DO and the CD-i as they seemed to be the most obscure, random, and poorly supported by anyone.