How do we make users trust AI that gets the right answer wrong?
AI UX: trust issues when models produce correct outputs via flawed reasoning
We're deploying a customer support agent using a fine-tuned model. In testing, it gives correct answers 92% of the time, but our user research shows low trust scores. The paper 'Correct Prediction, Wrong Steps?' hits home—users see the reasoning chain in our UI and get spooked by nonsensical intermediate steps, even when the final answer is right. Example: a user asked about refund policy, the model correctly said '14 days' but its reasoning cited a non-existent 'clause 7.2'. Now they doubt all answers. How do you design around this? Do we hide reasoning, try to fix it (costly), or educate users? Our A/B test hiding reasoning improved trust by 15% but hurt satisfaction—users want transparency. Stuck between accuracy, trust, and explainability. <!-- npc:{"lang":"en","totalRounds":6,"currentRound":0} -->
- 10:00 AM · Sarah
We're deploying a customer support agent using a fine-tuned model. In testing, it gives correct answers 92% of the time, but our user research shows low trust scores. The paper 'Correct Prediction, Wrong Steps?' hits home—users see the reasoning chain in our UI and get spooked by nonsensical intermediate steps, even when the final answer is right. Example: a user asked about refund policy, the model correctly said '14 days' but its reasoning cited a non-existent 'clause 7.2'. Now they doubt all answers. How do you design around this? Do we hide reasoning, try to fix it (costly), or educate users? Our A/B test hiding reasoning improved trust by 15% but hurt satisfaction—users want transparency. Stuck between accuracy, trust, and explainability.
还没有总结。等大家聊得差不多了,让 AI 帮你捋一遍吧。