Large Multimodal Models (LMMs) achieve state-of-the-art performance in high-stakes domains like healthcare, yet their reasoning remains opaque. Current interpretability methods, such as attention mechanisms or post-hoc saliency, often fail to faithfully represent the model's decision-making process, particularly when integrating heterogeneous modalities like time-series and text. We introduce Tree-of-Evidence (ToE), an inference-time search algorithm that frames interpretability as a discrete optimization problem. Rather than relying on soft attention weights, ToE employs lightweight Evidence Bottlenecks that score coarse groups or units of data (e.g., vital-sign windows, report sentences) and performs a beam search to identify the compact evidence set required to reproduce the model's prediction. We evaluate ToE across six tasks spanning three datasets and two domains: four clinical prediction tasks on MIMIC-IV, cross-center validation on eICU, and non-clinical fault detection on LEMMA-RCA. ToE produces auditable evidence traces while maintaining predictive performance, retaining over 0.98 of full-model AUROC with as few as five evidence units across all settings. Under sparse evidence budgets, ToE achieves higher decision agreement and lower probability fidelity error than other approaches. Qualitative analyses show that ToE adapts its search strategy: it often resolves straightforward cases using only vitals, while selectively incorporating text when physiological signals are ambiguous. ToE therefore provides a practical mechanism for auditing multimodal models by revealing which discrete evidence units support each prediction.
Grounded Decoding: Retrieval-Anchored Probability Fusion for Faithful RAG
Department of Computer Science, Iowa State University(爱荷华州立大学计算机科学系);Department of Computer Science, Kalinga Institute of Industrial Technology(卡林加工业技术学院计算机科学系);Department of Civil, Construction & Environmental Engineering, Iowa State University(爱荷华州立大学土木、建设与环境工程系)
From Concept Alignment to Causal Grounding: An Intervention Test of Chain-of-Thought Faithfulness
German Research Center for Artificial Intelligence (DFKI)(德国人工智能研究中心(DFKI));Saarland Informatics Campus(萨尔兰信息学园区);Centre for European Research in Trusted AI (CERTAIN)(欧洲可信人工智能研究中心(CERTAIN));Technische Universität Berlin(柏林工业大学);IBM Research(IBM研究院);University of Copenhagen(哥本哈根大学);University of Groningen(格罗宁根大学)
Affective Multimodal Agents with Proactive Knowledge Grounding for Emotionally Aligned Marketing Dialogue
Hunan Police Academy, Department of Criminal Investigation(湖南警察学院犯罪侦查系);Business College, California State University(加州州立大学长滩分校商学院);Northwestern University(西北大学);AI Agent Lab, Vokram Group(Vokram集团AI代理实验室);Beijing University of Technology(北京理工大学);Zheng Zhou University of Light Industry(郑州轻工业大学)