Lyapunov-Stable Adaptive Control for Multimodal Concept Drift
Department of Electrical and Computer Engineering(电气与计算机工程系);Florida Institute of National Security(佛罗里达国家安全研究所);Applied Artificial Intelligence Group(应用人工智能组);University of Florida(佛罗里达大学)
Tianyu Bell Pan, Mengdi Zhu, Alexa Jordyn Cole, Ronald Wilson, Damon L. Woodard
Multimodal learning systems often struggle in non-stationary environments due to concept drift, where changing data distributions can degrade performance. Modality-specific drifts and the lack of mechanisms for continuous, stable adaptation compound this challenge. This paper introduces LS-OGD, a novel adaptive control framework for robust multimodal learning in the presence of concept drift. LS-OGD uses an online controller that dynamically adjusts the model's learning rate and the fusion weights between different data modalities in response to detected drift and evolving prediction errors. We prove that under bounded drift conditions, the LS-OGD system's prediction error is uniformly ultimately bounded and converges to zero if the drift ceases. Additionally, we demonstrate that the adaptive fusion strategy effectively isolates and mitigates the impact of severe modality-specific drift, thereby ensuring system resilience and fault tolerance. These theoretical guarantees establish a principled foundation for developing reliable and continuously adapting multimodal learning systems.
Diverse via bounded Agreement: Geometric Regularization for Multimodal Fusion
Department of Informatics University of Bern(伯尔尼大学信息学院);College of Computing and Data Science Nanyang Technological University(南洋理工大学计算机与数据科学学院);School of Software Engineering Xi’an Jiaotong University(西安交通大学软件工程学院)
Silence and Noise: Resolving Suppression and Overvaluation in Multimodal Entity Alignment
School of Computer Science, The University of Auckland(奥克兰大学计算机科学学院);Institute of Trustworthy Embodied AI, Fudan University(复旦大学可信具身智能研究院);College of Electrical and Computer Engineering, National Yang Ming Chiao Tung University(国立阳明交通大学电机与计算机工程学院)
MIRPALab, Department of Engineering, University of Palermo(MIRPAL实验室、工程系、巴勒莫大学);Laboratoire LTI, Université de Picardie Jules Verne(LTI实验室、皮卡第大学朱利斯·韦尔纳大学);University of Palermo(巴勒莫大学)