National University of Singapore(新加坡国立大学);University of Warwick(华威大学);Ant Group(蚂蚁集团);Shandong University(山东大学);Harbin Institute of Technology (Shenzhen)(哈尔滨工业大学(深圳))
Harry Cheng, Yangyang Guo, Qingpei Guo, Ming Yang, Tian Gan, Weili Guan, Liqiang Nie
Multi-modal Large Language Models (MLLMs) have dramatically advanced the research field and delivered powerful vision-language understanding capabilities. However, these models often inherit deep-rooted social biases from their training data, leading to uncomfortable responses with respect to attributes such as race and gender. This paper addresses the issue of social biases in MLLMs by i) introducing a comprehensive counterfactual dataset with multiple social concepts (CMSC), which complements existing datasets by providing 18 diverse and balanced social concepts; and ii) proposing a counter-stereotype debiasing (CSD) strategy that mitigates social biases in MLLMs by leveraging the opposites of prevalent stereotypes. CSD incorporates both a novel bias-aware data sampling method and a loss rescaling method, enabling the model to effectively reduce biases. We conduct extensive experiments with four prevalent MLLM architectures. The results demonstrate the advantage of the CMSC dataset and the edge of CSD strategy in reducing social biases compared to existing competing methods, without compromising the overall performance on general multi-modal reasoning benchmarks.
Identifying and Mitigating Social Bias Knowledge in Language Models
Zhejiang University(浙江大学);Nanyang Technological University(南洋理工大学);Angelalign Technology Inc.(时代天使科技有限公司);A*STAR Centre for Frontier AI Research(新加坡科研局前沿人工智能研究中心);ZJU-Angelalign R&D Center for Intelligence Healthcare(浙江大学时代天使智能医疗研发中心)
Latent space bias directions in LLMs capture confidence, not fairness
Institute of Astronomy & Kavli Institute for Cosmology, University of Cambridge(剑桥大学天文学研究所及卡夫利宇宙学研究所);Risk and Security AI Lab, Visa Inc.(Visa公司风险与安全人工智能实验室)
Debiasing Multimodal Large Language Models via Noise-Aware Preference Optimization
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院);Baidu Inc.(百度公司)
Beyond Spurious Signals: Debiasing Multimodal Large Language Models via Counterfactual Inference and Adaptive Expert Routing
School of Computer Science, Peking University(北京大学计算机科学学院);MOE Key Laboratory of Computational Linguistics, Peking University(北京大学教育部计算语言学重点实验室);National Key Laboratory for Multimedia Information Processing, Peking University(北京大学国家多媒体信息处理重点实验室)
Prompting Fairness: Integrating Causality to Debias Large Language Models
Google DeepMind(谷歌DeepMind);Carnegie Mellon University(卡内基梅隆大学);University of Maryland, College Park(马里兰大学帕克分校);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);University of Texas at Austin(德克萨斯大学奥斯汀分校);University of California, Santa Cruz(加州大学圣克鲁兹分校)
Bi-directional Bias Attribution: Debiasing Large Language Models without Modifying Prompts
School of Informatics, Xiamen University, China(厦门大学信息学院);vivo AI Lab, China(vivo AI实验室);Key Laboratory of Digital Protection and Intelligent Processing of Intangible Cultural Heritage of Fujian and Taiwan (Xiamen University), Ministry of Culture and Tourism, China(福建省和台湾非物质文化遗产数字化保护与智能处理重点实验室(厦门大学),文化部,中国)
TriCon-Fair: Triplet Contrastive Learning for Mitigating Social Bias in Pre-trained Language Models
School of Computer Science(计算机科学学院);Artificial Intelligence, Wuhan University of Technology, Wuhan, China(武汉理工大学人工智能学院);Faculty of Data Science, City University of Macau, Macau, China(澳门城市大学数据科学学院)
Bridging Semantic Understanding and Popularity Bias with LLMs
Jilin University Changchun China;Dalian University of Technology Dalian China;Jinan University \& TAL Education Group Guangzhou China;Jilin University;Dalian University of Technology;Jinan University \& TAL Education Group
Unified Detoxifying and Debiasing in Language Generation via Inference-time Adaptive Optimization
Tsinghua University(清华大学);Beijing Academy of(北京学院(文本不完整,仅提取现有内容));Institute for AI Industry Research, Tsinghua University(清华大学人工智能产业研究院);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);National Research Center for Information Science Technology(国家信息科学技术研究中心);Microsoft Research Asia(微软亚洲研究院);International Innovation Center of Tsinghua University(清华大学国际创新中心)
LLM4Rec: Large Language Models for Multimodal Generative Recommendation with Causal Debiasing
Department of Software \& Microelectronics Peking University Beijing, China;Department of Software \& Microelectronics Peking University Beijing, China hangli\;Department of Software \& Microelectronics Peking University Beijing, China zehua\;Department of Software \& Microelectronics Peking University Beijing, China xiaofan\;Economic Law School China University of Political Science;Peking University Changsha, China
Take its Essence, Discard its Dross! Debiasing for Toxic Language Detection via Counterfactual Causal Effect
School of Computer Science and Technology, Dalian University of Technology(大连理工大学计算机科学与技术学院);School of Foreign Chinese, Dalian University of Technology(大连理工大学对外汉语学院)