SwanNLP 在 SemEval-2026 任务 5：一种基于 LLM 的叙事词义消歧可信度评分框架
SwanNLP at SemEval-2026 Task 5: An LLM-based Framework for Plausibility Scoring in Narrative Word Sense Disambiguation
AI总结：
本文提出基于 LLM 的框架，用于叙事文本中同义词消歧的可信度评分，通过结构化推理机制评估词义的可信度，实验表明动态少样本提示的大型参数 LLM 能有效模拟人类判断。
摘要：
近年来，语言模型的进步显著提升了自然语言理解（NLU）。尽管广泛使用的基准测试表明，大型语言模型（LLMs）能有效进行词义消歧，但其在真实叙事场景中的实际应用仍待探索。SemEval-2026 任务 5 通过引入预测短故事中词义可信度的任务填补了这一空白。本文提出一种基于 LLM 的框架，利用结构化推理机制对叙事文本中的同义词词义进行可信度评分。我们研究了通过多样化推理策略微调低参数 LLM 和动态少样本提示大型参数模型对准确词义识别和可信度评估的影响。实验结果表明，使用动态少样本提示的商业大型参数 LLM 能接近人类的可信度判断。此外，模型集成略微提升了性能，比单一模型预测更好地模拟了五名人类标注员的一致性模式。
英文摘要：
Recent advances in language models have substantially improved Natural Language Understanding (NLU). Although widely used benchmarks suggest that Large Language Models (LLMs) can effectively disambiguate, their practical applicability in real-world narrative contexts remains underexplored. SemEval-2026 Task 5 addresses this gap by introducing a task that predicts the human-perceived plausibility of a word sense within a short story. In this work, we propose an LLM-based framework for plausibility scoring of homonymous word senses in narrative texts using a structured reasoning mechanism. We examine the impact of fine-tuning low-parameter LLMs with diverse reasoning strategies, alongside dynamic few-shot prompting for large-parameter models, on accurate sense identification and plausibility estimation. Our results show that commercial large-parameter LLMs with dynamic few-shot prompting closely replicate human-like plausibility judgments. Furthermore, model ensembling slightly improves performance, better simulating the agreement patterns of five human annotators compared to single-model predictions
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
