XChoice: 用于基于LLM的受限选择决策中的可解释性AI-人类对齐评估
XChoice: Explainable Evaluation of AI-Human Alignment in LLM-based Constrained Choice Decision Making
AI总结：
XChoice是一种用于评估基于LLM的受限选择决策中AI-人类对齐的可解释框架，通过机制模型恢复可解释参数以诊断不一致并支持改进。
摘要：
我们提出了XChoice，一种用于评估AI-人类对齐的可解释框架，在受限决策中。超越结果一致性的评估，如准确率和F1分数，XChoice将基于机制的决策模型拟合到人类数据和LLM生成的决策上，恢复可解释的参数，这些参数捕捉了决策因素的相对重要性、约束敏感性和隐含的权衡。通过比较这些参数向量在不同模型、选项和子群体之间的差异来评估对齐程度。我们利用美国时间使用调查（ATUS）作为人类真实数据，展示了XChoice在美国人日常时间分配中的应用，揭示了不同模型和活动之间的异质对齐情况，并在黑人和已婚群体中发现了显著的不一致。我们进一步通过不变性分析验证了XChoice的鲁棒性，并通过检索增强生成（RAG）干预评估了针对性的缓解措施。总体而言，XChoice提供了基于机制的度量标准，能够诊断不一致并支持有根据的改进，而不仅仅是表面结果匹配。
英文摘要：
We present XChoice, an explainable framework for evaluating AI-human alignment in constrained decision making. Moving beyond outcome agreement such as accuracy and F1 score, XChoice fits a mechanism-based decision model to human data and LLM-generated decisions, recovering interpretable parameters that capture the relative importance of decision factors, constraint sensitivity, and implied trade-offs. Alignment is assessed by comparing these parameter vectors across models, options, and subgroups. We demonstrate XChoice on Americans' daily time allocation using the American Time Use Survey (ATUS) as human ground truth, revealing heterogeneous alignment across models and activities and salient misalignment concentrated in Black and married groups. We further validate robustness of XChoice via an invariance analysis and evaluate targeted mitigation with a retrieval augmented generation (RAG) intervention. Overall, XChoice provides mechanism-based metrics that diagnose misalignment and support informed improvements beyond surface outcome matching.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
