arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2610.22249cs.CL
收藏

不是X,而是Y:通过语境恰当性诊断中文大语言模型的写作习惯

Not X, but When: Diagnosing a Mandarin LLM Writing Tic through Contextual Appropriateness

Tsinghua University(清华大学);Fudan University(复旦大学)

Yihang Peng, Yi Jing, Dewen Liu, Weihan Meng, Ying Liu

AI总结:

本研究以中文修正结构“不是X,而是Y”为对象,引入决策树分类方法,发现多款大语言模型在该结构的使用上存在恰当性不足问题,呼吁开发关注表达恰当性的模型。

摘要:

模型生成文本中的重复表达常被视为机器作者身份的标志,但多数这类表达具有合理功能。这类表达是否为写作习惯,取决于模型是否知晓何时该使用它;这种恰当性判断是交际能力的核心,却在很大程度上未被针对语言模型研究。我们通过中文修正结构“bushi X,ershi Y”(“不是X,而是Y”)研究该问题。基于格赖斯语用学和修正否定理论,我们引入一种决策树,通过测试语境是否使X成为可行替代项、拒绝X是否具有信息性,将该结构的使用分为实质性、可选或有缺陷三类。五款商业模型使用该结构的频率是人类参考语料的2.5至9.2倍;在307个标注用例中,仅1.30%为有缺陷使用,52.8%为可选使用(即正确但非必要),因此模型在恰当性上的不足远多于正确性上的不足。多款奖励模型甚至偏好该结构甚于质量匹配的变体,大语言模型评判器在语境不允许的情况下仍偏好该结构,且监督微调与偏好优化均倾向于提升开放权重模型中该结构的使用频率。我们希望本研究能推动关注表达恰当性的模型开发。

英文摘要:

Recurring expressions in model-written text are often treated as signatures of machine authorship, yet most of them serve legitimate functions. Whether such an expression is a tic depends on whether the model knows when it is called for. This judgment of appropriateness, central to communicative competence, remains largely unexamined for language models. We study it through the Mandarin corrective construction bushi X, ershi Y ("not X, but Y"). Drawing on Gricean pragmatics and corrective negation, we introduce a decision tree that classifies uses as substantive, optional, or defective by testing whether the context makes X a live alternative and whether rejecting X is informative. Five commercial models use the construction 2.5 to 9.2 times as densely as a human reference. Of 307 annotated uses, only 1.30% are defective while 52.8% are optional, that is, correct but unneeded, so models fall short in appropriateness far more often than in correctness. Several reward models favor the construction even over quality-matched variants, an LLM judge favors it even where the context does not license it, and both supervised fine-tuning and preference optimization tend to raise its frequency in open-weight models. We hope this work encourages the development of models attentive to the appropriateness of their expressions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑