Learning Fine-grained Domain Generalization via Hyperbolic State Space Hallucination
School of Artificial Intelligence, Wuhan University（武汉大学人工智能学院）；Faculty of Information Technology, Monash University（莫纳什大学信息技术学院）；School of Medicine, Yale University（耶鲁大学医学院）
Qi Bi, Jingjun Yi, Haolan Zhan, Wei Ji, Gui-Song Xia
细粒度域泛化（FGDG）旨在仅使用源域数据训练时，学习一种能够良好泛化到未见目标域的细粒度表示。与通用域泛化相比，FGDG 尤其具有挑战性，因为细粒度类别只能通过一些细微且微小的模式加以区分。这些模式在由光照、颜色等引起的跨域风格变化下尤为脆弱。为推进这一前沿方向，本文提出了一种新颖的双曲状态空间幻觉（Hyperbolic State Space Hallucination，HSSH）方法。该方法包含两个关键组件，即状态空间幻觉（state space hallucination，SSH）和双曲流形一致性（hyperbolic manifold consistency，HMC）。SSH 通过首先对源图像进行外推、然后进行幻觉化，丰富状态嵌入的风格多样性。随后，将风格幻觉前后的状态嵌入投影到双曲流形中。双曲状态空间对高阶统计量进行建模，从而能够更好地区分细粒度模式。最后，最小化双曲距离，以消除风格变化对细粒度模式的影响。在三个 FGDG 基准上的实验证明了该方法的当前最优性能。
英文摘要：
Fine-grained domain generalization (FGDG) aims to learn a fine-grained representation that can be well generalized to unseen target domains when only trained on the source domain data. Compared with generic domain generalization, FGDG is particularly challenging in that the fine-grained category can be only discerned by some subtle and tiny patterns. Such patterns are particularly fragile under the cross-domain style shifts caused by illumination, color and etc. To push this frontier, this paper presents a novel Hyperbolic State Space Hallucination (HSSH) method. It consists of two key components, namely, state space hallucination (SSH) and hyperbolic manifold consistency (HMC). SSH enriches the style diversity for the state embeddings by firstly extrapolating and then hallucinating the source images. Then, the pre- and post- style hallucinate state embeddings are projected into the hyperbolic manifold. The hyperbolic state space models the high-order statistics, and allows a better discernment of the fine-grained patterns. Finally, the hyperbolic distance is minimized, so that the impact of style variation on fine-grained patterns can be eliminated. Experiments on three FGDG benchmarks demonstrate its state-of-the-art performance.
Can Humans Dream of Electric Sheep? Human-Written Samples for Fine-Grained Vision-and-Language Hallucination Benchmarking
University of Helsinki（赫尔辛基大学）；Politecnico di Torino（都灵理工大学）；Universiteit Utrecht（乌得勒支大学）；Université Bretagne Sud（南布列塔尼大学）；University of Copenhagen（哥本哈根大学）；University Grenoble Alpes（格勒诺布尔阿尔卑斯大学）；University of Lorraine（洛林大学）
SHALE: A Scalable Benchmark for Fine-grained Hallucination Evaluation in LVLMs
Key Laboratory of AI Safety of CAS, Institute of Computing Technology, Chinese Academy of Sciences (CAS)（中国科学院计算技术研究所人工智能安全重点实验室）；University of Chinese Academy of Sciences（中国科学院大学）；Trustworthy Technology and Engineering Laboratory, Huawei（华为可信技术与工程实验室）
FINER: MLLMs Hallucinate under Fine-grained Negative Queries
Technical University of Munich（慕尼黑技术大学）；Munich Center for Machine Learning（慕尼黑机器学习中心）；Helmholtz Munich（海德堡-慕尼黑亥姆霍尔茨中心）；Google（谷歌）；LTCI, Télécom Paris, Institut Polytechnique de Paris（LTCI，巴黎电信学院，巴黎理工学院）
Beyond the Global Scores: Fine-Grained Token Grounding as a Robust Detector of LVLM Hallucinations
Hanoi University of Science and Technology（河内科技大学）；Australian Institute for Machine Learning, University of Adelaide（澳大利亚机器学习研究所，阿德莱德大学）；Nanjing University（南京大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Hanoi-Amsterdam High School for the Gifted（河内阿姆斯特丹天才高中）
Seeing Clearly without Training: Mitigating Hallucinations in Multimodal LLMs for Remote Sensing
School of Computer Science, Wuhan University, China（武汉大学计算机学院）；Zhongguancun Academy, China（中关村学院）；School of Computer Science, Chongqing University, China（重庆大学计算机学院）
SAP SE（SAP公司）；Humboldt Universität zu Berlin（柏林洪堡大学）
State-of-the-art deep learning algorithms generally require large amounts of data for model training. Lack thereof can severely deteriorate the performance, particularly in scenarios with fine-grained boundaries between categories. To this end, we propose a multimodal approach that facilitates bridging the information gap by means of meaningful joint embeddings. Specifically, we present a benchmark that is multimodal during training (i.e. images and texts) and single-modal in testing time (i.e. images), with the associated task to utilize multimodal data in base classes (with many samples), to…
H3Former: Hypergraph-based Semantic-Aware Aggregation via Hyperbolic Hierarchical Contrastive Loss for Fine-Grained Visual Classification
College of Computer Science and Technology, Jilin University（吉林大学计算机科学与技术学院）；College of Software, Jilin University（吉林大学软件学院）；School of Software, Tsinghua University（清华大学软件学院）
Beyond Saying Less: Fine-Grained Alignment for Informative and Faithful Vision-Language Models
State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Zhongguancun Academy（中关村学院）
Fine-Grained Rib Fracture Diagnosis with Hyperbolic Embeddings: A Detailed Annotation Framework and Multi-Label Classification Model
Indian Institute of Technology Jodhpur（印度理工学院贾姆pur分校）；All India Institute of Medical Sciences Delhi（全印度医学科学研究所德里）；Government Medical College Srinagar（政府医学院斯里 Nagar）
ICT: Image-Object Cross-Level Trusted Intervention for Mitigating Object Hallucination in Large Vision-Language Models
Tsinghua University（清华大学）；Zhipu AI（智谱AI）；Chongqing University（重庆大学）；Shanghai Jiao Tong University（上海交通大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
Patch Matters: Training-free Fine-grained Image Caption Enhancement via Local Perception
School of Future Technology, South China University of Technology（华南理工大学未来技术学院）；College of Science, China Agricultural University（中国农业大学理学院）；Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学高瓴人工智能学院）；School of Engineering, Westlake University（西湖大学工程学院）
