Mitigating biases in generative AI and, particularly in text-to-image models, is of high importance given their growing implications in society. The biased datasets used for training pose challenges in ensuring the responsible development of these models, and mitigation through hard prompting or embedding alteration, are the most common present solutions. Our work introduces a novel approach to achieve diverse and inclusive synthetic images by learning a direction in the latent space and solely modifying the initial Gaussian noise provided for the diffusion process. Maintaining a neutral prompt and untouched embeddings, this approach successfully adapts to diverse debiasing scenarios, such as geographical biases. Moreover, our work proves it is possible to linearly combine these learned latent directions to introduce new mitigations, and if desired, integrate it with text embedding adjustments. Furthermore, text-to-image models lack transparency for assessing bias in outputs, unless visually inspected. Thus, we provide a tool to empower developers to select their desired concepts to mitigate. The project page with code is available online.
BiasMap: Leveraging Cross-Attentions to Discover and Mitigate Hidden Social Biases in Text-to-Image Generation
Institute for Clarity in Documentation（清晰文档研究所）；Inria Paris-Rocquencourt（巴黎-罗克琴court研究所）；Rajiv Gandhi University（拉吉夫·甘地大学）；Tsinghua University（清华大学）；Palmer Research Laboratories（帕勒尔研究实验室）；University of North Carolina at Charlotte（北卡罗来纳大学教堂山分校）；Utah State University（犹他州立大学）
Peking University（北京大学）；Bytedance AI Lab（字节跳动人工智能实验室）
This paper introduces the Attribute-Decomposed GAN, a novel generative model for controllable person image synthesis, which can produce realistic person images with desired human attributes (e.g., pose, head, upper clothes and pants) provided in various source inputs. The core idea of the proposed model is to embed human attributes into the latent space as independent codes and thus achieve flexible and continuous control of attributes via mixing and interpolation operations in explicit style representations. Specifically, a new architecture consisting of two encoding pathways with style block…
Peking-Tsinghua Center for Life Sciences, Academy for Advanced Interdisciplinary Studies, IDG/McGovern Institute for Brain Research, Peking University（北京大学-清华生命科学中心，先进跨学科研究院，IDG/麦克戈文脑科学研究院，北京大学）；Center of Quantitative Biology, Peking University（北京大学定量生物学中心）；School of Psychological and Cognitive Sciences, Key Laboratory of Machine Perception (Ministry of Education), Peking University（心理与认知科学学院，机器感知重点实验室（教育部），北京大学）
UniNDM: A Unified Noise-driven Detection and Mitigation Framework Against Sexual Content in Text-to-Image Generation
Institute of Artificial Intelligence, State Key Laboratory of Virtual Reality Technology and Systems, Beihang University（北京航空航天大学虚拟现实技术与系统国家重点实验室人工智能研究院）；College of Artificial Intelligence, Tsinghua University（清华大学人工智能学院）；Security Department, Alibaba Group（阿里巴巴集团安全部）；School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-Sen University（中山大学深圳校区网络科学与技术学院）
Reverse the auditory processing pathway: Coarse-to-fine audio reconstruction from fMRI
Key Laboratory of Brain Cognition and Brain-inspired Intelligence Technology（脑认知与类脑智能技术重点实验室）；State Key Laboratory of Multimodal Artificial Intelligence Systems（多模态人工智能系统国家重点实验室）；School of Future Technology, University of Chinese Academy of Sciences（中国科学院大学未来技术学院）
BM-CL: Bias Mitigation through the lens of Continual Learning
Research Institute for Signals, Systems and Computational Intelligence, sinc(i), CONICET - Universidad Nacional del Litoral（信号、系统与计算智能研究所以及CONICET - 国家沿海大学）；AI Development and Evaluation (AIDE) Lab, Stanford University（人工智能发展与评估实验室，斯坦福大学）；Institute of Computer Sciences (ICC), CONICET - Universidad de Buenos Aires（计算机科学研究所（ICC），CONICET - 阿根廷布宜诺斯艾利斯大学）
Context-aware Multimodal AI Reveals Hidden Pathways in Five Centuries of Art Evolution
Soongsil University（崇实大学）；Korea Advanced Institute of Science and Technology（韩国科学技术院）；Indiana University（印第安纳大学）；Pohang University of Science and Technology（浦项科技大学）
Learning Enabled Components (LECs) are widely being used in a variety of perception based autonomy tasks like image segmentation, object detection, end-to-end driving, etc. These components are trained with large image datasets with multimodal factors like weather conditions, time-of-day, traffic-density, etc. The LECs learn from these factors during training, and while testing if there is variation in any of these factors, the components get confused resulting in low confidence predictions. The images with factors not seen during training is commonly referred to as Out-of-Distribution (OOD). …
Steering LVLMs via Sparse Autoencoder for Hallucination Mitigation
School of Computer Science and Engineering, Southeast University（东南大学计算机科学与工程学院）；Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University)（东南大学新一代人工智能技术及其交叉应用关键实验室）；Foundation Model Research Center, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所基础模型研究中心）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Department of Computer Science and Technology, Tsinghua University（清华大学计算机科学与技术系）；Wuhan University of Technology（武汉理工大学）；National University of Singapore（新加坡国立大学）
Siamese-Driven Optimization for Low-Resolution Image Latent Embedding in Image Captioning
Department of Mechatronics and Biomedical Engineering, Lee Kong Chian Faculty of Engineering and Science（机械与生物医学工程系，李光前工程与科学学院）；Laboratoire de traitement et transport de l'information, Université Sorbonne Paris Nord（信息处理与传输实验室，巴黎-索邦大学）
