There is quite a bit of good Scandinavian music.
Regarding Kim Larsen, it is certainly not true that everybody loves him. While I don't think he's horrible or anything, it's just not the kind of music I listen to.
From Denmark:
Under Byen(or, "Below the City"), probably my favourite Danish band. Very interesting and unique music. A song from their most recent album can be found here:
http://www.youtube.com/watch?v=xy-66WGK5EE
The lyrics are so obscure that it doesn't really matter if you understand Danish or not, it still makes little sense.
Efterklang A mix of classical and electronica, somewhat reminiscent of Sigur Rös at times. English lyrics.
Mew Progressive rock/pop. The lead singer has a very unique voice. One of their best songs can be heard here, the video is a bit...weird, though:
http://www.youtube.com/watch?v=jFjKALF454g English lyrics.
Windermere Space rock. Their first album was quite nice, haven't heard all of the newest one yet. Many of their songs can be heard on their website
www.windermere.dk "Trailer Park" is probably the best one. English lyrics.
From Sweden:
Anglägard: I believe the genre is called symphonic prog, very epic, nice guitarplay. Swedish lyrics.
The Flower Kings: Neo-prog, VERY much inspired by Yes. English lyrics.
From Norway:
Kaizer's Orchestra: Polka-rock. It is as crazy as it sounds.
http://www.youtube.com/watch?v=65Y3UuwWftM Norwegian lyrics.
From Iceland:
Sigur Rós: Probably well-known already, one of my favourite bands. Especially Agætis Byrjun. Hopelandic/Icelandic lyrics.
Múm: Nice and relaxing electronica. English lyrics.
From Finland:
Nightwish: Little introduction required, I guess.