Summary assessment involves evaluating how well a generated summary reflects the key ideas and meaning of the source text, requiring a deep understanding of the content. Large Language Models (LLMs) have been used to automate this process, acting as judges to evaluate summaries with respect to the original text. While previous research investigated the alignment between LLMs and Human responses, it is not yet well understood what properties or features are exploited by them when asked to evaluate based on a particular quality dimension, and there has not been much attention towards mapping between evaluation scores and metrics. In this paper, we address this issue and discover features aligned with Human and Generative Pre-trained Transformers (GPTs) responses by studying statistical and machine learning metrics. Furthermore, we show that instructing GPTs to employ metrics used by Human can improve their judgment and conforming them better with human responses.
Exploring Combinatorial Problem Solving with Large Language Models: A Case Study on the Travelling Salesman Problem Using GPT-3.5 Turbo
King Fahd University of Petroleum & Minerals（法赫德国王石油矿产大学）；SDAIA-KFUPM Joint Research Center for Artificial Intelligence（SDAIA-法赫德国王石油矿产大学人工智能联合研究中心）；Center for Smart Mobility and Logistics, King Fahd University of Petroleum & Minerals（法赫德国王石油矿产大学智能移动与物流中心）；Queensland University of Technology（昆士兰科技大学）
GPTKB v1.5: A Massive Knowledge Base for Exploring Factual LLM Knowledge
ScaDS.AI Dresden/Leipzig & TU Dresden, Germany（德国德累斯顿/莱比锡ScaDS.AI & 德累斯顿工业大学）；VNU University of Engineering and Technology, Hanoi, Vietnam（越南河内国立大学工程与技术大学）
Evaluating Text Summaries Generated by Large Language Models Using OpenAI's GPT
University of Colorado - Colorado Springs（科罗拉多大学科罗拉多斯普林斯分校）；University of Massachusetts Lowell（马萨诸塞大学洛厄尔分校）；University of Kansas（堪萨斯大学）；DataCrunch Lab, LLC（DataCrunch实验室有限责任公司）；University of Florida（佛罗里达大学）
Summary of ChatGPT-Related Research and Perspective Towards the Future of Large Language Models
School of Physics and Information Technology, Shaanxi Normal University（陕西师范大学物理与信息技术学院）；School of Life and Technology Biomedical-Engineering, Xi’an Jiaotong University（西安交通大学生命与技术学院生物医学工程系）；School of Computing, The University of Georgia（佐治亚大学计算机学院）；Department of Computer Science and Engineering, The University of Texas at Arlington（德克萨斯大学阿灵顿分校计算机科学与工程系）；Department of Radiology, Massachusetts General Hospital and Harvard Medical School（麻省总医院放射科与哈佛医学院）；School of Biomedical Engineering, ShanghaiTech University（上海科技大学生物医学工程学院）；Shanghai United Imaging Intelligence Co., Ltd.（上海联影智能有限公司）；Shanghai Clinical Research and Trial Center（上海临床研究与试验中心）
Exploring the Trade-Offs: Unified Large Language Models vs Local Fine-Tuned Models for Highly-Specific Radiology NLI Task
The University of Georgia（佐治亚大学）；The University of Texas at Arlington（德克萨斯大学阿灵顿分校）；Northwestern Polytechnical University（西北工业大学）；University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；Mayo Clinic（梅奥诊所）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；ShanghaiTech University（上海科技大学）；Shanghai United Imaging Intelligence Co., Ltd.（上海联影智能医疗科技有限公司）；Shanghai Clinical Research and Trial Center（上海临床研究与试验中心）
Exploring the Limits of ChatGPT in Software Security Applications
University of Michigan, Ann Arbor（密歇根大学安娜堡分校）；ASU（亚利桑那州立大学）；Washington University in St. Louis（圣路易斯华盛顿大学）；University of Wisconsin, Madison（威斯康星大学麦迪逊分校）
The use of GPT-4o and Other Large Language Models for the Improvement and Design of Self-Assessment Scales for Measurement of Interpersonal Communication Skills
University of Zagreb（萨格勒布大学）；Faculty of Organization and Informatics（组织与信息学院）
Dr. GPT Will See You Now, but Should It? Exploring the Benefits and Harms of Large Language Models in Medical Diagnosis using Crowdsourced Clinical Cases
