Background The disappearance of a loved one is a unique type of loss, also termed “ambiguous loss”, which may heighten the risk for developing prolonged grief (PG), depression, and posttraumatic stress (PTS) symptoms. Little is known about protective and risk factors for psychopathology among relatives of missing persons. A potential protective factor is self-compassion, referring to openness toward, and acceptance of one’s own pain, failures, and inadequacies. One could reason that self-compassion is associated with lower levels of emotional distress following ambiguous loss, because it might serve as a buffer for getting entangled in ruminative thinking about the causes and consequences of the disappearance (coined “grief rumination”). Objective The current study sought to test this mediational model. In a sample of relatives of missing persons we aimed to examine (1) the prediction that greater self-compassion is related to lower symptom-levels of PG, depression, and PTS and (2) to what extent these associations are mediated by grief rumination. Method Dutch and Belgian relatives of long-term missing persons (N = 137) completed self-report measures tapping self-compassion, grief rumination, PG, depression, and PTS. Mediation analyses were conducted. Results Self-compassion was significantly, negatively, and moderately associated with PG, depression, and PTS levels. Grief rumination significantly mediated the associations of higher levels of self-compassion with lower levels of PG (a*b = -0.11), depression (a*b = -0.07), and PTS (a*b = -0.11). Specifically, 50%, 32%, and 32% of the effect of self-compassion on PG, depression, and PTS levels respectively, was accounted for by grief rumination. Conclusions Findings suggest that people with more self-compassion experience less severe psychopathology, in part, because these people are less strongly inclined to engage in ruminative thinking related to the disappearance. Strengthening a self-compassionate attitude using, for instance, mindfulness-based interventions, may therefore be a useful intervention to reduce emotional distress associated with the disappearance of a loved one.