Mitigating Algorithm Aversion in Recruiting: A Study on Explainable AI for Conversational Agents

The use of conversational agents (CAs) based on artificial intelligence (AI) is becoming more common in the field of recruiting. Organizations are now adopting AI-based CAs for applicant (pre-)selection, but negative news coverage and especially the black box character of AI, has hindered adoption. So far, little is known about the contextual factors influencing users’ perception of AI-based CAs in general and the effect of provided explanations by explainable AI (XAI) in particular. While research on algorithm aversion provides some initial explanations, information regarding the effects of different XAI approaches on different types of decisions on the attitudes of (potential) applicants is scarce. Therefore, in this study, we use a quantitative, quota-representative study (n = 490) to assess the acceptance of CAs in recruiting. By applying an experimental within-subject design, we provide a more nuanced perspective on why and when providing explanations increases user acceptance. We also show that contextual factors such as the type of assessed skills are major determinants of this effect, and we conclude that XAI is not a “one-size-fits-all approach.” Based on the insight that contextual factors of the decision problem are more important than the type of XAI approach itself, we argue that the use and the effects of explainability in recruiting need a more nuanced perspective, focusing on the fit of explanations with the user’s characteristics and preferences.