arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2607.05545cs.CLcs.AI

大多数大语言模型的从众现象无需说话者:在同伴压力基准测试中测量无说话者底线

Most LLM Conformity Needs No Speaker: Measuring the Speaker-Free Floor in Peer-Pressure Benchmarks

Illinois Institute of Technology(伊利诺伊理工学院);Amazon(亚马逊)

Yibo Hu, Jiaming Qu

AI总结:

研究大语言模型从众现象,发现多数从众在去除同伴后仍存。因标准提示混杂线索致现有基准无法分辨。引入无来源条件测试,发现其在多数案例中致有害修正,还揭示来源框架对底线的调节作用及相关问题,强调从众基准测试应先测无说话者底线。

摘要:

大语言模型(LLM)的从众现象常指模型将正确答案改为同伴或群体的答案。研究表明,即使去除同伴,大部分明显的从众现象依然存在。原因在于标准从众提示同时混杂了说话者的存在和重复的错误答案这两个线索。现有基准测试同时改变这些线索,无法分辨修正实际有多少依赖于说话者。引入无来源条件(去除明确说话者的相同断言答案)后发现,在六个开放权重的大语言模型和七个问答及推理数据集上,仅该条件就在66.5%的初始正确案例中导致有害修正,而普通重新提问时为10.3%。当重复答案被释义以及在开放式设置中隐藏答案选项时,该效应依然存在。来源框架主要调节这个底线:专家小组框架提高它,而最少的人物标签不能可靠地提高它。当模型翻转时,通常错得很自信,简单重新校准无法恢复原始答案。来源归因仍然重要,但应作为高于这个无说话者底线的增量来衡量。方法上的教训是,从众基准测试应首先测量去除说话者后剩下的部分;没有这一步,基准测试可能将重复文本误认为社会影响。

英文摘要:

LLM conformity is often used to describe cases where a model changes a correct answer toward a peer or group response. We show that most of this apparent conformity survives even after the peer is removed. The reason is a confound: standard conformity prompts mix two cues at once, the presence of a speaker and the repeated wrong answer itself. Existing benchmarks vary these cues together, so they cannot tell how much of the revision actually depends on the speaker. We introduce a no-source condition: the same asserted answer with the explicit speaker removed. Across six open-weight LLMs and seven QA and reasoning datasets, this condition alone causes harmful revision in $66.5\%$ of initially correct cases, compared with $10.3\%$ under a plain re-ask. The effect also remains when the repeated answer is paraphrased and when answer options are hidden in an open-ended setting. Source framing mainly modulates this floor: expert-panel framing raises it, while minimal person labels do not reliably raise it. When models flip, they are usually confidently wrong, and simple recalibration does not recover the original answer. Source attribution still matters, but it should be measured as an increment above this speaker-free floor. The methodological lesson is that conformity benchmarks should first measure what remains after the speaker is removed; without this step, benchmarks may mistake repeated text for social influence.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑