上下文胜过计算 人类在环优于迭代思维链提示在面试回答质量上的表现
Context Over Compute Human-in-the-Loop Outperforms Iterative Chain-of-Thought Prompting in Interview Answer Quality
AI总结：
本文通过对比人类在环和自动思维链提示方法，发现人类在环在面试回答质量评估中表现更优，且迭代次数更少，同时具有更高的训练效果。
摘要：
使用大语言模型进行行为面试评估存在独特挑战，需要结构化评估、现实面试官行为模拟和候选人培训的教育价值。我们通过两个受控实验研究思维链提示在面试回答评估和改进中的应用，使用50对行为面试问题和回答对。我们的贡献有三方面：首先，我们提供了人类在环和自动思维链改进的定量比较。使用配对设计，n等于50，两种方法均显示出积极的评分改进。人类在环方法提供了显著的培训效益。信心从3.16提高到4.16（p小于0.001），真实性从2.94提高到4.53（p小于0.001，Cohen's d是3.21）。人类在环方法还要求五次迭代更少（1.0对5.0，p小于0.001）并实现了完整的个人细节整合。其次，我们分析了收敛行为。两种方法都快速收敛，平均迭代次数低于1次，其中人类在环方法在最初较弱的回答中达到100%的成功率，而自动方法为84%（Cohen's h是0.82，大效应）。额外的迭代提供 diminishing returns，表明主要限制是上下文可用性而非计算资源。第三，我们提出了一种基于负面偏见模型的对抗性挑战机制，称为bar raiser，以模拟现实的面试官行为，尽管定量验证仍需未来工作。我们的发现表明，尽管思维链提示为面试评估提供了有用的基石，但领域特定的增强和上下文感知的方法选择对于现实和具有教育价值的结果至关重要。
英文摘要：
Behavioral interview evaluation using large language models presents unique challenges that require structured assessment, realistic interviewer behavior simulation, and pedagogical value for candidate training. We investigate chain of thought prompting for interview answer evaluation and improvement through two controlled experiments with 50 behavioral interview question and answer pairs. Our contributions are threefold. First, we provide a quantitative comparison between human in the loop and automated chain of thought improvement. Using a within subject paired design with n equals 50, both approaches show positive rating improvements. The human in the loop approach provides significant training benefits. Confidence improves from 3.16 to 4.16 (p less than 0.001) and authenticity improves from 2.94 to 4.53 (p less than 0.001, Cohen's d is 3.21). The human in the loop method also requires five times fewer iterations (1.0 versus 5.0, p less than 0.001) and achieves full personal detail integration. Second, we analyze convergence behavior. Both methods converge rapidly with mean iterations below one, with the human in the loop approach achieving a 100 percent success rate compared to 84 percent for automated approaches among initially weak answers (Cohen's h is 0.82, large effect). Additional iterations provide diminishing returns, indicating that the primary limitation is context availability rather than computational resources. Third, we propose an adversarial challenging mechanism based on a negativity bias model, named bar raiser, to simulate realistic interviewer behavior, although quantitative validation remains future work. Our findings demonstrate that while chain of thought prompting provides a useful foundation for interview evaluation, domain specific enhancements and context aware approach selection are essential for realistic and pedagogically valuable results.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
