大型语言模型能否辅助标注者识别道德框架？——基于社交媒体疫苗辩论的案例研究
Can LLMs Assist Annotators in Identifying Morality Frames? -- Case Study on Vaccination Debate on Social Media
AI总结：
本研究通过少样本学习利用LLMs生成概念与解释，辅助人工标注者识别社交媒体疫苗辩论中的道德框架，实验表明该方法能提高标注准确性并降低认知负荷。
摘要：
如今，社交媒体在塑造公共话语方面发挥着关键作用，尤其是在疫苗接种等极化议题上，多元的道德视角会影响个体观点。在自然语言处理（NLP）领域，数据稀缺以及心理语言学任务（如识别道德框架）的复杂性，使得仅依赖人工标注者成本高昂、耗时较长，且由于认知负荷容易产生不一致。为解决这些问题，我们利用大型语言模型（LLMs），它们擅长通过少样本学习适应新任务，利用少量上下文示例并结合将示例与任务原则相联系的解释。我们的研究探索了LLMs在辅助人工标注者识别社交媒体疫苗辩论中道德框架方面的潜力。我们采用两步流程：首先用LLMs生成概念和解释，然后使用“出声思维”工具进行人工评估。我们的研究表明，将LLMs整合到标注流程中可以提高准确性、降低任务难度、减轻认知负荷，这为复杂心理语言学任务中的人机协作提供了一条有前景的途径。
英文摘要：
Nowadays, social media is pivotal in shaping public discourse, especially on polarizing issues like vaccination, where diverse moral perspectives influence individual opinions. In NLP, data scarcity and complexity of psycholinguistic tasks, such as identifying morality frames, make relying solely on human annotators costly, time-consuming, and prone to inconsistency due to cognitive load. To address these issues, we leverage large language models (LLMs), which are adept at adapting new tasks through few-shot learning, utilizing a handful of in-context examples coupled with explanations that connect examples to task principles. Our research explores LLMs' potential to assist human annotators in identifying morality frames within vaccination debates on social media. We employ a two-step process: generating concepts and explanations with LLMs, followed by human evaluation using a "think-aloud" tool. Our study shows that integrating LLMs into the annotation process enhances accuracy, reduces task difficulty, lowers cognitive load, suggesting a promising avenue for human-AI collaboration in complex psycholinguistic tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
