保持能力不变的简历扰动揭示LLM筛选中的呈现敏感性
Competence-Preserving Resume Perturbations Expose Presentation Sensitivity in LLM Screening
浏览论文内容
中文总结 AI 辅助
本研究通过受控简历扰动审计发现，LLM在简历筛选中存在呈现敏感性，即使能力证据不变，决策仍可能大幅翻转，呼吁评估应同时关注有效性与稳定性。
中文摘要 AI 辅助
简历筛选者必须从简历中推断与工作相关的能力，而简历的呈现方式在措辞、结构、文体润色和文档提取质量上可能有很大差异。理想情况下，当潜在资格证据不变时，这种表面变化不应改变决策。我们引入了对这一属性的受控审计，在受控能力水平上构建基于职业的候选人档案，并将每个档案渲染为多种简历呈现形式。一个确定性验证门在评分前排除改变潜在证据的变体。在六种开放指令微调LLM条件下，我们发现筛选有效性与呈现稳定性之间存在明显脱节。Llama-3.1-8B及其原生聊天模板实现了最强的有效性（0.781），但在保持能力不变的呈现变化下，有29.6%的配对决策发生反转；Mistral-7B-v0.3达到有效性0.644，翻转率为41.4%。原生聊天格式提高了几种聊天微调模型的有效性，但并未消除这种不稳定性。这些结果表明，简历筛选评估不仅应评估系统是否能识别更强的候选人，还应评估当相同能力证据以不同方式呈现时，这些决策是否保持稳定。
英文摘要
Resume screeners must infer job-relevant competence from resumes whose presentation can vary substantially in wording, structure, stylistic polish, and document extraction quality. Ideally, such surface variation should not change decisions when the underlying qualification evidence is unchanged. We introduce a controlled audit of this property, constructing occupation-grounded candidate profiles at controlled competence levels and rendering each profile into multiple resume presentations. A deterministic validation gate excludes variants that alter the underlying evidence before scoring. Across six open instruction-tuned LLM conditions, we find a clear disconnect between screening validity and presentation stability. Llama-3.1-8B with its native chat template achieves the strongest validity ($0.781$) yet reverses $29.6\%$ of matched pairwise decisions under competence-preserving presentation changes; Mistral-7B-v0.3 reaches validity $0.644$ with a $41.4\%$ flip rate. Native chat formatting improves validity for several chat-tuned models but does not remove this instability. These results show that resume-screening evaluations should assess not only whether a system identifies stronger candidates, but also whether those decisions remain stable when the same competence evidence is presented differently.
发表机构
- Macquarie University（麦考瑞大学）
- The University of Melbourne（墨尔本大学）
机构由 AI 辅助整理，请以论文原文为准。
