The current scholarly literature on the international mediation of civil wars draws predominantly on a rationalist-materialist perspective. This perspective suggests that the ticket to mediation success is the material manipulation of the bargaining environment by third parties with a high degree of economic and military resources. I argue that legitimacy also determines outcomes of mediation because if a mediator has legitimacy, it can continue to look for a mutually satisfactory outcome and try to pull the conflict parties toward compliance. I show that legitimacy matters by systematically comparing the effectiveness of African and non-African third parties. African third parties are typically considered ineffective because of a low degree of economic and military capacity. However, they effectively mediate civil wars in Africa because of a high degree of legitimacy, which is a result of a strong conviction within the African society of states that African mediation is the most desirable type in conflicts there. Drawing on data from the Uppsala Conflict Data Program supplemented with unique data, which together cover all mediation efforts in Africa between 1960 and 2017, I find quantitative evidence supporting the effectiveness of African third parties. Compared to non-African ones, African third parties are far more likely to conclude negotiated settlements that are more likely to be durable. African third parties are especially effective if the conflict parties are highly committed to the African solutions norm. Theoretically, this study deviates from much of the literature that puts forward solely rationalist-materialist explanations of mediation success. By bringing legitimacy to the forefront, this article supplements the current mediation literature that emphasizes material sources of power and ignores social structures.