arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2402.14453cs.CL
收藏

大语言模型能否隐式地为用户确定合适的文本难度?

Do LLMs Implicitly Determine the Suitable Text Difficulty for Users?

Nara Institute of Science and Technology(奈良科学技术研究所)

Seiji Gobara, Hidetaka Kamigaito, Taro Watanabe

AI总结:

本文通过构建Stack-Overflow问答数据集并结合TSCC多轮对话数据,研究大语言模型在用户输入与生成回复之间隐式调整文本难度的能力,发现部分LLMs可超越人类且指令微调至关重要。

摘要:

适合个体学习水平的教育对于提高学生的理解能力是必要的。利用大语言模型(LLMs)实现这一目标的第一步,是使回复的文本难度适应学生。本文分析了LLMs如何在用户输入与其生成文本之间隐式地调整文本难度。为开展实验,我们基于Stack-Overflow构建了一个新数据集,以探究基于问答的对话性能。在Stack-Overflow数据集和包含多轮对话的TSCC数据集上的实验结果表明,LLMs能够在用户输入与其生成回复之间隐式地处理文本难度。我们还观察到,一些LLMs在处理文本难度方面可以超越人类,并且指令微调(instruction-tuning)具有重要作用。

英文摘要:

Education that suits the individual learning level is necessary to improve students' understanding. The first step in achieving this purpose by using large language models (LLMs) is to adjust the textual difficulty of the response to students. This work analyzes how LLMs can implicitly adjust text difficulty between user input and its generated text. To conduct the experiments, we created a new dataset from Stack-Overflow to explore the performance of question-answering-based conversation. Experimental results on the Stack-Overflow dataset and the TSCC dataset, including multi-turn conversation show that LLMs can implicitly handle text difficulty between user input and its generated response. We also observed that some LLMs can surpass humans in handling text difficulty and the importance of instruction-tuning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑