arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2302.14828cs.CL
收藏

基于大语言模型的梦境情感内容自动评分

Automatic Scoring of Dream Reports' Emotional Content with Large Language Models

University of Sussex(萨塞克斯大学);IMT School for Advanced Studies(IMT高等研究院)

Lorenzo Bertolini, Valentina Elce, Adriana Michalak, Giulio Bernardi, Julie Weeds

摘要:

在梦的研究领域,对梦的内容的研究通常依赖于对做梦者从睡眠中醒来时提供的口头报告的分析。这个任务通常是由经过训练的注释者通过手动评分来完成的,这需要花费大量的时间。虽然一致的工作表明,自然语言处理(NLP)工具可以支持梦报告的自动分析,但提出的方法缺乏对报告的完整上下文进行推理的能力,并且需要大量的数据预处理。此外,在大多数情况下,这些方法没有经过标准手动评分方法的验证。在这项工作中,我们通过采用大语言模型(LLM)来研究和复制梦报告的手动注释来解决这些限制,使用现成和定制方法的混合物,重点关注报告的情感参考。我们的研究结果表明,现成的方法实现了低性能可能是鉴于内在的语言差异,在不同的(组)个人收集的报告。另一方面,所提出的定制文本分类方法实现了高性能,这是对潜在的偏见的鲁棒性。总的来说,这些观察结果表明,我们的方法可以在大型梦境数据集的分析中找到应用,并且可能有利于研究结果的可重复性和可比性。

英文摘要:

In the field of dream research, the study of dream content typically relies on the analysis of verbal reports provided by dreamers upon awakening from their sleep. This task is classically performed through manual scoring provided by trained annotators, at a great time expense. While a consistent body of work suggests that natural language processing (NLP) tools can support the automatic analysis of dream reports, proposed methods lacked the ability to reason over a report's full context and required extensive data pre-processing. Furthermore, in most cases, these methods were not validated against standard manual scoring approaches. In this work, we address these limitations by adopting large language models (LLMs) to study and replicate the manual annotation of dream reports, using a mixture of off-the-shelf and bespoke approaches, with a focus on references to reports' emotions. Our results show that the off-the-shelf method achieves a low performance probably in light of inherent linguistic differences between reports collected in different (groups of) individuals. On the other hand, the proposed bespoke text classification method achieves a high performance, which is robust against potential biases. Overall, these observations indicate that our approach could find application in the analysis of large dream datasets and may favour reproducibility and comparability of results across studies.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑