Social and Behavioral Sciences Commons^™

Measures For Explainable Ai: Explanation Goodness, User Satisfaction, Mental Models, Curiosity, Trust, And Human-Ai Performance, Robert R. Hoffman, Shane Mueller, Gary Klein, Jordan Litman

Michigan Tech Publications

If a user is presented an AI system that portends to explain how it works, how do we know whether the explanation works and the user has achieved a pragmatic understanding of the AI? This question entails some key concepts of measurement such as explanation goodness and trust. We present methods for enabling developers and researchers to: (1) Assess the a priori goodness of explanations, (2) Assess users' satisfaction with explanations, (3) Reveal user's mental model of an AI system, (4) Assess user's curiosity or need for explanations, (5) Assess whether the user's trust and reliance on the AI are …