Previous research suggests that listening to music can enhance memory and well-being. However, what is often missing from this analysis is consideration of the social dimensions of music—for example, its capacity to affirm or threaten listeners’ social identities. This study examined whether (ir)religious music that was potentially identity-affirming or identity-threatening (Christian hymns, Buddhist chants, classical, or no music) would affect Christians’ and Atheists’ (N = 267) well-being and memory performance while listening. Analyses revealed significant interactions between (ir)religious group and music type on memory, self-esteem, and mood. Listening to music that potentially threatened one's religious identity appeared to undermine both performance self-esteem and actual memory performance, while increasing feelings of hostility. This pattern was found for Christians (vs. Atheists) who listened to Buddhist chants. Conversely, Atheists’ performance self-esteem (and to some degree their memory performance) was lowest, and their hostility highest, when they listened to Christian hymns. In this way, listening to music that potentially threatened one's religious group identity (or lack thereof) appeared to be detrimental for memory, self-esteem, and mood. These results bridge research on the psychology of religion, music psychology, and social identity theorizing by demonstrating that the effects of music on memory and well-being may reflect important (even sacred) social identities, with potential implications for individual well-being and intergroup relations.