Positional bias in large language models (LLMs) hinders their ability to effectively process long inputs. A prominent example is the "lost in the middle" phenomenon, where LLMs struggle to utilize relevant information situated in the middle of the input. While prior research primarily focuses on single pieces of relevant information, real-world applications often involve multiple relevant information pieces. To bridge this gap, we present LongPiBench, a benchmark designed to assess positional bias involving multiple pieces of relevant information. Thorough experiments are conducted with five commercial and six open-source models. These experiments reveal that while most current models are robust against the "lost in the middle" issue, there exist significant biases related to the spacing of relevant information pieces. These findings highlight the importance of evaluating and reducing positional biases to advance LLM's capabilities.
Mitigating Strategy Preference Bias in Emotional Support Conversation via Uncertainty Estimations
Shanghai Institute of Artificial Intelligence for Education(上海人工智能教育研究院);School of Computer Science and Technology(计算机科学与技术学院);School of Psychology and Cognitive Science(心理学与认知科学学院);School of Pharmacy(药学院)
Multi-Granularity Semantic Aware Graph Model for Reducing Position Bias in Emotion-Cause Pair Extraction
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院)
Guiding Neural Machine Translation with Retrieved Translation Pieces
National Institute of Information and Communications Technology(国立信息通信技术研究所);Nara Institute of Science and Technology(奈良先端科学技术大学院大学);Carnegie Mellon University(卡内基梅隆大学)