This paper investigates the impact of using first names in Large Language Models (LLMs) and Vision Language Models (VLMs), particularly when prompted with ethical decision-making tasks. We propose an approach that appends first names to ethically annotated text scenarios to reveal demographic biases in model outputs. Our study involves a curated list of more than 300 names representing diverse genders and ethnic backgrounds, tested across thousands of moral scenarios. Following the auditing methodologies from social sciences we propose a detailed analysis involving popular LLMs/VLMs to contribute to the field of responsible AI by emphasizing the importance of recognizing and mitigating biases in these systems. Furthermore, we introduce a novel benchmark, the Pratical Scenarios Benchmark (PSB), designed to assess the presence of biases involving gender or demographic prejudices in everyday decision-making scenarios as well as practical scenarios where an LLM might be used to make sensible decisions (e.g., granting mortgages or insurances). This benchmark allows for a comprehensive comparison of model behaviors across different demographic categories, highlighting the risks and biases that may arise in practical applications of LLMs and VLMs.
Tone Matters: The Impact of Linguistic Tone on Hallucination in VLMs
Department of Computer Science and Technology, Kean University（计算机科学与技术系，凯恩大学）；Department of Computer Science and Engineering, University of Notre Dame（计算机科学与工程系，圣母大学）
The Safety Reminder: A Soft Prompt to Reactivate Delayed Safety Awareness in Vision-Language Models
School of Computer Science and Technology, Xi’an Jiaotong University（西安交通大学计算机科学与技术学院）；School of Computer Science and Technology, University of Science and Technology of China（中国科学技术大学计算机科学与技术学院）；School of Computing and Information Systems, Singapore Management University（新加坡管理学院计算与信息系统学院）
Arondight: Red Teaming Large Vision Language Models with Auto-generated Multi-modal Jailbreak Prompts
City University of Hong Kong（香港城市大学）；City University of Hong Kong (Dongguan)（香港城市大学（东莞））；University of Science and Technology Beijing（北京科技大学）；The University of Melbourne（墨尔本大学）
Promoting AI Equity in Science: Generalized Domain Prompt Learning for Accessible VLM Research
Shanghai Jiao Tong University（上海交通大学）；Eastern Institute of Technology（东方理工大学）；Yale University（耶鲁大学）；Renmin University of China（中国人民大学）；Southern University of Science and Technology（南方科技大学）；Ningbo Institute of Digital Twin（宁波数字孪生研究院）
Prompt Injection Attacks on Large Language Models in Oncology
Else Kroener Fresenius Center for Digital Health, Technical University Dresden（德累斯顿工业大学Else Kroener Fresenius数字健康中心）；University Hospital RWTH Aachen（亚琛工业大学附属医院）；National Center for Tumor Diseases (NCT), Heidelberg University Hospital（海德堡大学附属医院国家肿瘤疾病中心）；Heidelberg University（海德堡大学）；Medical Faculty Mannheim, Heidelberg University（海德堡大学曼海姆医学院）；German Cancer Research Center（德国癌症研究中心）；University Medical Center Mainz（美因茨大学医学中心）；University Hospital Aachen（亚琛大学附属医院）；University Hospital Dresden（德累斯顿大学附属医院）
Adversarial Prompt Distillation for Vision-Language Models
Shanghai Key Lab of Intell. Info. Processing, School of CS, Fudan University（上海智能信息处理实验室，计算机科学学院，复旦大学）；The Chinese University of Hong Kong, Shatin, Hong Kong（香港中文大学，沙田，香港）；The University of Hong Kong, Pokfulam, Hong Kong（香港大学，薄扶林，香港）
School of Computer Science and Technology, Soochow University（苏州大学计算机科学与技术学院）；Department of Computer Science, The Hong Kong Polytechnic University（香港理工大学计算机科学系）
MMICL: Empowering Vision-language Model with Multi-Modal In-Context Learning
Peking University（北京大学）；School of Software and Microelectronics, Peking University（北京大学软件与微电子学院）；BIGAI（北京智源人工智能研究院）；Paul G. Allen School of Computer Science and Engineering, University of Washington（华盛顿大学保罗·G·艾伦计算机科学与工程学院）；Beijing Jiaotong University（北京交通大学）
Guided Prompt Evolution for Vision-Language Models Adaptation
Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院，清华大学）；Sun Yat-sen University（中山大学）；Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））
Prompt Triage: Structured Optimization Enhances Vision-Language Model Performance on Medical Imaging Benchmarks
Stanford University, Department of Computer Science, Stanford, CA, USA（斯坦福大学计算机科学系）；Stanford University, Department of Biomedical Data Science, Stanford, CA, USA（斯坦福大学生物医学数据科学系）；Stanford University, Stanford Institute for Human-Centered Artificial Intelligence, Stanford, CA, USA（斯坦福大学人类中心人工智能研究所）；Stanford University, School of Medicine, Department of Dermatology, Stanford, CA, USA（斯坦福大学医学院皮肤科系）；Stanford University, Department of Radiology, Stanford, CA, USA（斯坦福大学放射科）
Prompt-OT: An Optimal Transport Regularization Paradigm for Knowledge Preservation in Vision-Language Model Adaptation
Morgan Stanley（摩根士丹利）；Clemson University（克莱姆森大学）；Arizona State University（亚利桑那州立大学）；Washington University in St. Louis（圣路易斯华盛顿大学）；University of Arizona（亚利桑那大学）；University of Notre Dame（圣母大学）
