开放世界机器学习中的信息论:基础、框架与未来方向
Information Theory in Open-world Machine Learning Foundations, Frameworks, and Future Direction
AI总结:
本文综述信息论在开放世界机器学习中的应用,通过熵、互信息等概念统一开放集识别、新颖性发现与持续学习,并探讨其与可证明学习框架的联系及未来方向。
摘要:
开放世界机器学习(OWML)旨在开发能够识别已知类别、拒绝未知样本并持续从新信息中学习的智能系统。尽管在开放集识别、新颖性检测和持续学习方面取得了显著进展,该领域仍缺乏统一的理论基础来量化不确定性、表征信息传递并解释动态非平稳环境中的学习适应性。本文全面综述了开放世界机器学习中的信息论方法,强调熵、互信息和Kullback-Leibler散度等核心概念如何为描述开放世界条件下的知识获取、不确定性抑制和风险控制提供数学语言。我们将近期研究综合为三大研究方向:实现未知安全拒绝的信息论开放集识别、引导新概念形成的信息驱动新颖性发现,以及确保稳定长期适应的信息保持持续学习。此外,我们讨论了信息论与可证明学习框架之间的理论联系,包括PAC-Bayes界、开放空间风险理论和因果信息流,以建立通往可证明且可信的开放世界智能的路径。最后,本综述指出了关键开放问题和未来研究方向,如信息风险的量化、动态互信息界的开发、多模态信息融合,以及信息论与因果推理和世界模型学习的整合。
英文摘要:
Open world Machine Learning (OWML) aims to develop intelligent systems capable of recognizing known categories, rejecting unknown samples, and continually learning from novel information. Despite significant progress in open set recognition, novelty detection, and continual learning, the field still lacks a unified theoretical foundation that can quantify uncertainty, characterize information transfer, and explain learning adaptability in dynamic, nonstationary environments. This paper presents a comprehensive review of information theoretic approaches in open world machine learning, emphasizing how core concepts such as entropy, mutual information, and Kullback Leibler divergence provide a mathematical language for describing knowledge acquisition, uncertainty suppression, and risk control under open world conditions. We synthesize recent studies into three major research axes: information theoretic open set recognition enabling safe rejection of unknowns, information driven novelty discovery guiding new concept formation, and information retentive continual learning ensuring stable long term adaptation. Furthermore, we discuss theoretical connections between information theory and provable learning frameworks, including PAC Bayes bounds, open-space risk theory, and causal information flow, to establish a pathway toward provable and trustworthy open world intelligence. Finally, the review identifies key open problems and future research directions, such as the quantification of information risk, development of dynamic mutual information bounds, multimodal information fusion, and integration of information theory with causal reasoning and world model learning.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。