Absurd? I think not. I was already a hacker, then. Yes, theoretically 'everybody' had heard of Commodore. But 'everybody' thought of them primarily as game machines. Even though some of the military/government offices I worked in used them (and Ataris) as work machines (much like modern PCs). The IBM was, as you said, really aimed at the business market. But that just reaffirmed people's thinking that 'real' computers were made by IBM.
Home users/hobbyists went for the Amiga. But the trend had already started: "If I have a computer at home, I want to be able to work on it / use the same programs I'm familiar with from work / etc." Since the breadwinner controlled the purse strings, that meant he (or, occasionally she) would buy the computer he wanted, a 'real' computer, not just a 'kids toy'.

And there went the home PC market....