通知：联邦节假日调休：哥伦布日 / 原住民日（Columbus Day / Indigenous Peoples' Day），今日（13日）无更新。通知：联邦节假日调休：哥伦布日 / 原住民日（Columbus Day / Indigenous Peoples' Day），今日（13日）无更新。
arXiv 2507.13881cs.CLcs.AIcs.CY
使用大语言模型在开放式情境判断测试中识别人际与职业技能特征
Using LLMs to identify features of personal and professional skills in an open-response situational judgment test
Acuity Insights Inc.（Acuity Insights公司）
Cole Walsh, Rodica Ivan, Muhammad Zafar Iqbal, Colleen Robb
备注：
10 pages, 2 figures, 4 tables; this work was accepted for presentation at the 2025 Artificial Intelligence in Measurement and Education Conference in Pittsburgh, Pennsylvania, United States
Academic programs are increasingly recognizing the importance of personal and professional skills and their critical role alongside technical expertise in preparing students for future success in diverse career paths. With this growing demand comes the need for scalable systems to measure, evaluate, and develop these skills. Situational Judgment Tests (SJTs) offer one potential avenue for measuring these skills in a standardized and reliable way, but open-response SJTs have traditionally relied on trained human raters for evaluation, presenting operational challenges to delivering SJTs at scale. Past attempts at developing NLP-based scoring systems for SJTs have fallen short due to issues with construct validity of these systems. In this article, we explore a novel approach to extracting construct-relevant features from SJT responses using large language models (LLMs). We use the Casper SJT to demonstrate the efficacy of this approach. This study sets the foundation for future developments in automated scoring for personal and professional skills.
Automatic Item Generation for Personality Situational Judgment Tests with Large Language Models
Beijing Normal University（北京师范大学）；Central China Normal University（华中师范大学）；Key Laboratory of Adolescent Cyberpsychology and Behavior, Ministry of Education（教育部青少年网络心理与行为重点实验室）；Beijing Key Laboratory of Applied Experimental Psychology（北京市应用实验心理学重点实验室）；Research Center for Capacity Building in Educational Assessment and Evaluation (Beijing Higher Education Innovation Center for Philosophy and Social Sciences)（教育评价与评估能力建设研究中心（北京高等教育哲学社会科学创新中心））
Can AI Reason Like an Urban Planner? Benchmarking Large Language Models Against Professional Judgment
School of Architecture and Urban Planning, Shenzhen University（深圳大学建筑与城市规划学院）；Shenzhen Key Laboratory of Urban Spatial Information and Intelligent Modeling（深圳市城市空间信息与智能建模重点实验室）；Department of Urban Planning and Design, The University of Hong Kong（香港大学城市规划与设计系）
AUV-Bench: Aesthetic Understanding and Generation Evaluation for User Interfaces
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Alibaba Group（阿里巴巴集团）；The Hong Kong University of Science and Technology（香港科技大学）
Legal Syllogism Prompting: Teaching Large Language Models for Legal Judgment Prediction
Peking University Law School（北京大学法学院）；Institute for Artificial Intelligence, Peking University（北京大学人工智能研究院）；PKU-WUHAN Institute for Artificial Intelligence（北大-武汉人工智能研究院）
A Data Science Approach to Calcutta High Court Judgments: An Efficient LLM and RAG-powered Framework for Summarization and Similar Cases Retrieval
A.K.Choudhury School of IT（A.K.Choudhury信息科技学院）；University of Calcutta（加尔各答大学）；Department of Computer Science（计算机科学系）；Bangabasi Morning College（Bangabasi晨学院）
ADAPT Research Centre, DCU（ADAPT研究中心，都柏林城市大学）；Logrus Global
Human evaluation has always been expensive while researchers struggle to trust the automatic metrics. To address this, we propose to customise traditional metrics by taking advantages of the pre-trained language models (PLMs) and the limited available human labelled scores. We first re-introduce the hLEPOR metric factors, followed by the Python version we developed (ported) which achieved the automatic tuning of the weighting parameters in hLEPOR metric. Then we present the customised hLEPOR (cushLEPOR) which uses Optuna hyper-parameter optimisation framework to fine-tune hLEPOR weighting para…
In this paper, we study the problem of legal domain adaptation problem from an imbalanced source domain to a partial target domain. The task aims to improve legal judgment predictions for non-professional fact descriptions. We formulate this task as a partial-and-imbalanced domain adaptation problem. Though deep domain adaptation has achieved cutting-edge performance in many unsupervised domain adaptation tasks. However, due to the negative transfer of samples in non-shared classes, it is hard for current domain adaptation model to solve the partial-and-imbalanced transfer problem. In this wor…
Transforming the Voice of the Customer: Large Language Models for Identifying Customer Needs
Kellogg School of Management, Northwestern University（西北大学凯洛格管理学院）；MIT Management School, Massachusetts Institute of Technology（麻省理工学院斯隆管理学院）；Applied Marketing Science, Inc.（应用营销科学公司）
Truth Knows No Language: Evaluating Truthfulness Beyond English
HiTZ Center - Ixa, University of the Basque Country, UPV/EHU（希茨中心-ixa，巴斯克大学，UPV/EHU）；Elhuyar（埃尔胡亚尔）；Centro de Investigación en Tecnoloxías Intelixentes (CiTIUS), Universidade de Santiago de Compostela（智能技术研究中心（CiTIUS），圣地亚哥-德孔波斯特拉大学）；Departament de Traducció i Ciències del Llenguatge, Universitat Pompeu Fabra（翻译与语言科学部门，庞培法布拉大学）
Human services organizations and the responsible integration of AI: Considering ethics and contextualizing risk(s)
School of Social Work, University of Michigan（密歇根大学社会工作学院）；Graduate School of Social Service, Fordham University（福特汉姆大学社会工作研究生院）；School of Social Work, Wayne State University（韦恩州立大学社会工作学院）
When Ethics and Payoffs Diverge: LLM Agents in Morally Charged Social Dilemmas
ETH Zürich（苏黎世联邦理工学院）；University of Zurich（苏黎世大学）；Carnegie Mellon University（卡内基梅隆大学）；University of Michigan（密歇根大学）；Max Planck Institute for Intelligent Systems, Tübingen（图宾根人工智能研究所）；University of Toronto（多伦多大学）；Vector Institute（向量研究所）
Rare Diseases, Common Dilemmas: LLMs Prioritize Equal Resource Distribution over Patient Benefit in Decision-Making
Harvard T.H. Chan School of Public Health（哈佛陈曾熙公共卫生学院）；Harvard Medical School（哈佛医学院）；Clalit Health Services（克拉利特医疗服务机构）；Harvard College（哈佛学院）；Harvard-MIT Program in Health Sciences and Technology（哈佛-麻省理工健康科学与技术项目）
Speech-based Psychological Crisis Assessment using LLMs
Tsinghua University（清华大学）；Peking University Huilongguan Clinical Medical School（北京大学回龙guan临床医学院）；WHO Collaborating Centre for Research and Training in Suicide Prevention（世界卫生组织自杀预防研究与培训协作中心）
NurValues: Real-World Nursing Values Evaluation for Large Language Models in Clinical Context
The Hong Kong Polytechnic University（香港理工大学）；University of Copenhagen（哥本哈根大学）；Tianjin University（天津大学）；Tongji Hospital of Tongji Medical College of Huazhong University of Science and Technology（华中科技大学同济医学院同济医院）；Beijing University of Chinese Medicine（北京中医药大学）；Halmstad University（哈马尔大学）
AI-Press: A Multi-Agent News Generating and Feedback Simulation System Powered by Large Language Models
School of Data Science, Fudan University（复旦大学数据科学学院）；Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University（复旦大学类脑智能科学与技术研究院）；School of Computer Science, Fudan University（复旦大学计算机科学学院）；Research Institute of Intelligent Complex Systems（智能复杂体系研究院）
Inteligencia Artificial jurídica y el desafío de la veracidad: análisis de alucinaciones, optimización de RAG y principios para una integración responsable
