Education that suits the individual learning level is necessary to improve students' understanding. The first step in achieving this purpose by using large language models (LLMs) is to adjust the textual difficulty of the response to students. This work analyzes how LLMs can implicitly adjust text difficulty between user input and its generated text. To conduct the experiments, we created a new dataset from Stack-Overflow to explore the performance of question-answering-based conversation. Experimental results on the Stack-Overflow dataset and the TSCC dataset, including multi-turn conversation show that LLMs can implicitly handle text difficulty between user input and its generated response. We also observed that some LLMs can surpass humans in handling text difficulty and the importance of instruction-tuning.
Probing the Difficulty Perception Mechanism of Large Language Models
Institute of Digital Twin, EIT(数字孪生研究所,EIT);Wuhan University of Science and Technology(武汉科技大学);Zhejiang University(浙江大学);Hong Kong Polytechnic University(香港理工大学);Shenzhen Institutes of Advanced Technology, CAS(深圳先进技术研究院,中国科学院)
RankLLM: Weighted Ranking of LLMs by Quantifying Question Difficulty
Lehigh University(莱维大学);University of Notre Dame(诺特大学);Zhejiang Wanli University(浙江万里大学);Squirrel Ai Learning;City University of Hong Kong(香港城市大学);Duke University(杜克大学)
Can LLMs Estimate Cognitive Complexity of Reading Comprehension Items?
Graduate School of Artificial Intelligence, POSTECH, Republic of Korea(韩国立科学技术院人工智能研究生院);Department of Computer Science and Engineering, POSTECH, Republic of Korea(韩国立科学技术院计算机科学与工程系)
Success is in the Details: Evaluate and Enhance Details Sensitivity of Code LLMs through Counterfactuals
Harbin Institute of Technology(哈尔滨工业大学);Fudan University(复旦大学);University of Science and Technology of China(中国科学技术大学);Beijing University of Posts and Telecommunications(北京邮电大学)
Estimating LLM Grading Ability and Response Difficulty in Automatic Short Answer Grading via Item Response Theory
DIPF | Leibniz Institute for Research and Information in Education(德意志教育研究所 | 列支敦士登教育研究所);Faculty of Computer Science, Goethe University Frankfurt(法兰克福歌德大学计算机学院);Chemnitz University of Technology(化学工业大学)
A Multi-Agent Framework for Feature-Constrained Difficulty Control in Reading Comprehension Item Generation
Graduate School of Artificial Intelligence, POSTECH, Republic of Korea(韩国POSTECH人工智能研究生院);Department of Computer Science and Engineering, POSTECH, Republic of Korea(韩国POSTECH计算机科学与工程系)
Exploring the Potential of Large Language Models for Estimating the Reading Comprehension Question Difficulty
Siebel School of Computing and Data Science, University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校西贝尔计算与数据科学学院);Arkansas State University(阿肯色州立大学);School of Computer Science, Carnegie Mellon University(卡内基梅隆大学计算机科学学院);Heinz College of Information Systems and Public Policy, Carnegie Mellon University(卡内基梅隆大学海因茨信息系统与公共政策学院);University of Memphis(孟菲斯大学);Institute for Intelligent Systems, University of Memphis(孟菲斯大学智能系统研究所);Department of Psychology, University of Memphis(孟菲斯大学心理学系)
LLMs Encode Their Failures: Predicting Success from Pre-Generation Activations
Oxford Internet Institute, University of Oxford(牛津大学牛津互联网研究所);FLAIR, University of Oxford(牛津大学FLAIR);Department of Computer Science, University College London(伦敦大学学院计算机科学系)
LLMs Struggle to Measure What Distinguishes Students of Different Proficiency Levels: A Study of Item Discrimination in Reading Comprehension Assessment
MBZUAI(穆罕默德·本·扎耶德人工智能大学);University of Maryland(马里兰大学);Virginia Tech(弗吉尼亚理工大学)
What's Taboo for You? - An Empirical Evaluation of LLMs Behavior Toward Sensitive Content
Università degli Studi di Milano, Department of Computer Science(米兰大学计算机科学系);Università degli Studi di Milano, Department of Languages, Literatures, Cultures and Mediations(米兰大学语言、文学、文化与媒介系);Università degli Studi di Milano, Department of Historical Studies(米兰大学历史学系)
Farther the Shift, Sparser the Representation: Analyzing OOD Mechanisms in LLMs
Rutgers University(罗格斯大学);Northwestern University(西北大学);UKP Lab, TU Darmstadt(德累斯顿技术大学UKP实验室);New Jersey Institute of Technology(新泽西理工学院);NEC Lab American(美国NEC实验室);University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校)