In anticipation of the release of “Collapse Into Now” next week, Paste Magazine took a unique look at R.E.M.’s legacy by posting a list of the 20 best cover versions of the band’s work. I loved this piece because I had not heard many of them, and they managed to have audio for about half the list.

Even though I listen to the awesome podcast Coverville a lot, popular R.E.M. covers are not that easy to find. The anonymity of the efforts on the Paste list underscores this even though many of them are amazing.

While listening to those songs and bugging Brian Ibbot from Coverville to do another R.E.M. cover story, I started thinking about a different angle: What are the best covers R.E.M. has performed?

The official studio albums released by the band have only included two covers, but their B-sides and bonus tracks – some of them compiled on “Dead Letter Office” – give me plenty to choose from. I cannot fully choose from their yearly Fan Club Christmas release because I only belonged to the club for a year or two and have not heard most of them. I will, however, pull from bootlegs I have heard from the band’s early years.

So here goes with my 10 favorites.

Superman: Hearing this song played at college parties in 1986 played a huge role in my introduction to the band. I know they kind of shy away from it, but the song really is a fun little jaunt which showcases a lot of what makes R.E.M. so awesome. Watch them perform “Superman” in 1986 with one of the song’s writers from the group The Clique.

There She Goes Again: As I got deeper and deeper into R.E.M. in the late 1980s, I read a lot about how much The Velvet Underground influenced them. I knew little about VU except that it was Lou Reed’s band so bought some of their stuff. What an education. It took me a while to figure out how such a dark band inspired R.E.M., which first made its name as an upbeat live act playing poppy music. But they owned this song in the early years, often playing it live and releasing it as an early B-side.

Crazy: Peter Buck has often talked about how he felt humbled when he heard the original version of this song by fellow Athens band Pylon. I can’t see how anything could make anyone think the original exceeded anything R.E.M. ever did, including this cover, which appeared on “Dead Letter Office.” Here’s a live version.

King of the Road: Tacked onto the end of Dead Letter Office after an upbeat number which was created as an unofficial commercial jingle for Weaver D’s, the soul food joint which inspired the title for “Automatic for the People,” this is a hidden gem, right down to the fact that Mike Mills doesn’t seem to know tape is rolling.

Moon River: A version of this appears tacked onto the first re-release of “Reckoning,” but I have heard Michael Stipe do an a capella version on some bootlegs. It’s mesmerizing.

First We Take Manhattan: I had never heard of Leonard Cohen until R.E.M. recorded this for a tribute album in the early 1990s. They really do a nice job putting their own touch on this song while opening the eyes and ears of people like me to the legendary performer.

Favorite Writer: Linda Hopper’s relationship with the members of R.E.M. pre-dates the band’s formation so it seemed appropriate to have her on stage to sing this song first performed by her band Magnapop when the R.E.M. toured Holland (where Magnapop is based) when supporting the 2003 release of “In Time.” The band then released their own version as a bonus track on a single, but I love the version with Hopper.

Yellow River: Included as a bonus track to one of the singles from “Reveal,” this 1970 song by The Christies shows off some fun guitar work and Mike Mills’ sunny vocal. I wish more people knew about this.

Love Is All Around: Mills again takes the reins here, this time on a sweet number from The Troggs. He semi-explained in a solo appearance years later that Stipe is not a fan of the ban, which may be why the bass player handled the lead vocal. This song probably got most of its R.E.M. notoriety because they did it in their first appearance on MTV’s Unplugged in support of “Out of Time.”

Baby Baby: One of the few Christmas Fan Club singles I received. You’ll notice a trend here as Mills handles the vocals of a British band from the past, this time The Vibrators. Just fun.

Little Girl: I only have heard one version of this Syndicate of Sound cover, from the epic April 10, 1981 show at Tyrone’s. You could totally see this as an early R.E.M. song, and it totally works when you imagine them playing in front of a group of college kids dancing around.

There are a lot of other songs which could have slipped on this list, like their version of Aerosmith’s “Toys in the Attic,” early concert songs like “Shakin’ All Over” and “Route 66” or the many other Velvet Underground songs they did, but I had to cut things off somewhere. I also recommend the Hindu Love Gods album, which is the musical part of R.E.M. along with Warren Zevon. Their version of “Raspberry Beret” kills.