学术同行评审仍然是学术验证的基石，但该领域在数据和方法上面临一些挑战。从数据角度看，现有研究受到大规模、经过验证的基准不足以及过于简化的评估指标的限制，这些指标无法反映现实中的编辑工作流程。为弥补这一差距，我们提出了OmniReview，一个通过整合多源学术平台构建的综合数据集，通过歧义消除流程获得202,756条经过验证的审稿记录。基于此数据，我们引入了一个三级层次评估框架，以评估从召回率到精确专家识别的推荐。从方法角度看，现有基于嵌入的方法受到语义压缩的信息瓶颈和有限可解释性的限制。为解决这些方法限制，我们提出了利用大语言模型（LLM）与多任务学习相结合的“Profiling Scholars with Multi-gate Mixture-of-Experts（Pro-MMoE）”新框架。具体而言，它利用LLM生成的语义资料来保留细粒度的专业知识细微差别和可解释性，同时采用任务自适应的MMoE架构动态平衡冲突的评估目标。全面的实验表明，Pro-MMoE在六个指标上均达到最先进的性能，建立了现实审稿人推荐的新基准。
英文摘要：
Academic peer review remains the cornerstone of scholarly validation, yet the field faces some challenges in data and methods. From the data perspective, existing research is hindered by the scarcity of large-scale, verified benchmarks and oversimplified evaluation metrics that fail to reflect real-world editorial workflows. To bridge this gap, we present OmniReview, a comprehensive dataset constructed by integrating multi-source academic platforms encompassing comprehensive scholarly profiles through the disambiguation pipeline, yielding 202, 756 verified review records. Based on this data, we introduce a three-tier hierarchical evaluaion framework to assess recommendations from recall to precise expert identification. From the method perspective, existing embedding-based approaches suffer from the information bottleneck of semantic compression and limited interpretability. To resolve these method limitations, we propose Profiling Scholars with Multi-gate Mixture-of-Experts (Pro-MMoE), a novel framework that synergizes Large Language Models (LLMs) with Multi-task Learning. Specifically, it utilizes LLM-generated semantic profiles to preserve fine-grained expertise nuances and interpretability, while employing a Task-Adaptive MMoE architecture to dynamically balance conflicting evaluation goals. Comprehensive experiments demonstrate that Pro-MMoE achieves state-of-the-art performance across six of seven metrics, establishing a new benchmark for realistic reviewer recommendation.
School of Computer Science and Technology, Beijing Institute of Technology（北京理工大学计算机科学与技术学院）；School of Computer Science, Beijing University of Posts and Telecommunications（北京邮电大学计算机学院）；Department of Computer Science, Tongji University（同济大学计算机科学系）；State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications（北京邮电大学网络与交换技术国家重点实验室）
Bridge-AI Lab（Bridge-AI实验室）；University of Central Florida（中佛罗里达大学）；Auburn University（奥本大学）；Wichita State University（威奇托州立大学）；Gustavus Adolphus College（古斯塔夫斯阿道夫学院）
Knowledge-Guided Prompt Learning for Request Quality Assurance in Public Code Review
School of Computer Science（计算机科学学院）；Artificial Intelligence, Wuhan University of Technology（人工智能，武汉理工大学）；China Mobile Design Institute Co., Ltd.（中国移动设计研究院有限公司）；School of Computer Science, Wuhan University（计算机科学学院，武汉大学）
Towards Understanding and Measuring COGNITIVE ATROPHY in LLM Behaviour
York University（约克大学）；Vector Institute（向量研究所）；Rotman Research Institute（罗特曼研究学院）；Dalhousie University（达尔豪斯大学）；Centre for Addiction & Mental Health（成瘾与心理健康中心）；KITE Research Institute（KITE研究机构）
Measuring what Matters: Construct Validity in Large Language Model Benchmarks
University of Oxford（牛津大学）；EPFL（苏黎世联邦理工学院）；Weizenbaum Institute Berlin（柏林Weizenbaum研究所）；Technical University Munich（慕尼黑技术大学）；Centre for Digital Governance, Hertie School（赫尔姆霍兹学院数字治理中心）；Stanford University（斯坦福大学）；UK AI Security Institute（英国人工智能安全研究所）；SomosNLP；Universdad Politécnica de Madrid（马德里理工大学）；Yale University（耶鲁大学）；Allen Institute for AI（人工智能研究所）；University of Washington（华盛顿大学）；Meedan；UC Berkeley（加州大学伯克利分校）
Traditional statistical representations outperform generative AI in identifying expert peer reviewers
Michigan State University（密歇根州立大学）；Masaryk University（马萨里克大学）；Max Planck Institute for Astrophysics（马克斯·普朗克天体物理研究所）；Joint ALMA Observatory（ALMA联合天文台）；Stanford University（斯坦福大学）；European Southern Observatory（欧洲南方天文台）；Space Telescope Science Institute（太空望远镜科学研究所）
