发表机构
University of Bremen（不来梅大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
研究自然语言推理中形式语义结构对人类标签差异的解释程度，通过在ChaosNLI项目上运用特定方法和预注册分析，得出组级界限、项目级上限及组合不变性等结果，揭示形式语义结构对标注分歧的影响有限。
AI 中文摘要
自然语言推理中的人类标签差异越来越被视为信号而非噪声，但形式语义结构能解释多少差异尚未得到直接测量。我们在ChaosNLI的3113个SNLI和MNLI项目上进行测量，使用基于规则的运算符和经过MED验证的单调性标记器，进行三个预注册分析块，并全面报告负面结果。得出三个界限：一是组级界限，非纯向上单调的假设标签熵更高；二是项目级上限，相同形式特征解释的熵方差仅3.3%至3.6%；三是组合不变性，跨边界的对比结果均为无效。在该样本中，形式语义结构对注释者分歧程度影响小，且未改变分歧内容。ChaosNLI - S/M由低原始一致性项目组成，所有分析均在版本控制的研究日志中预注册，论文披露了审计记录，包括一条修正的解释规则。
英文摘要
Human label variation in natural language inference (NLI) is increasingly treated as a signal to be measured rather than as noise to be removed. We ask whether one candidate source of that signal, formal semantic structure such as negation, quantification, and monotonicity, changes how much annotators disagree and what they disagree about. We tag the SNLI and MNLI items of ChaosNLI, each labeled by 100 annotators, with a rule-based monotonicity tagger, check the tagger by hand on a sample of the same items, and answer three questions. At the group level, hypotheses that are not purely upward monotone attract somewhat more disagreement, but an error sensitivity analysis shows that this difference is sensitive to tagger error. At the item level, formal structure explains only a few percent of the variation and cannot pick out the items that attract high disagreement. In composition, the kinds of disagreement recorded by VariErr and LiTEx do not differ detectably across the formal boundary. Formal structure therefore belongs in the inventory of disagreement sources, with a small and bounded weight. ChaosNLI was built from low-agreement items, and every claim holds within that scope. Analysis decisions were written in a version-controlled research log before the corresponding results were computed, and negative results are reported in full.
Comments16 pages, 3 figures. v2 is a substantial revision; v3 updates only the abstract and comments to match v2, the text is unchanged. Code and preregistered analysis log: https://github.com/oudeis01/nli-hlv-structure
