Don't Be Fooled: The Misinformation Effect of Explanations in Human–AI Collaboration

Titelangaben

Spitzer, Philipp ; Holstein, Joshua ; Morrison, Katelyn ; Holstein, Kenneth ; Satzger, Gerhard ; Kühl, Niklas:
Don't Be Fooled: The Misinformation Effect of Explanations in Human–AI Collaboration.
In: International Journal of Human-Computer Interaction. (November 2025) .
ISSN 1532-7590
DOI: https://doi.org/10.1080/10447318.2025.2574511

Volltext

Link zum Volltext (externe URL):

Abstract

Across various applications, humans increasingly use black-box artificial intelligence (AI) systems without insight into these systems’ reasoning. To counter this opacity, explainable AI (XAI) methods promise enhanced transparency and interpretability. While recent studies have explored how XAI affects human–AI collaboration, few have examined the potential pitfalls caused by incorrect explanations. The implications for humans can be far-reaching but have not been explored extensively. To investigate this, we conducted a study (n = 160) on AI-assisted decision-making in which humans were supported by XAI. Our findings reveal a misinformation effect when incorrect explanations accompany correct AI advice with implications post-collaboration. This effect causes humans to infer flawed reasoning strategies, hindering task execution and demonstrating impaired procedural knowledge. Additionally, incorrect explanations compromise human–AI team performance during collaboration. With our work, we contribute to HCI by providing empirical evidence for the negative consequences of incorrect explanations on humans post-collaboration and outlining guidelines for designers of AI.