I think you're right. It might also partially be an acquired skill. As a musician I hear a lot more details in music than a non-trained ear normally would. This naturally makes it easier for me to recall music with more detail. If you aren't trained and have a low level of musicality then I would assume you'd only memorize/remember the main parts of the song/passage, like the "sound" of the guitar playing that solo, whereas I hear the piano comping (or is it just another guitar? I remember piano :D), bass & drums, all while picturing how to actually play the solo on the guitar.

Like a non-musician hearing a guitar riff would just replay it as "nana nana na, na-naa" or something simple whereas I would be able to hear it in exact detail and how it interacts with the rest of the band melodically as well as rhythmically.