从教中学正则化：可泛化的相关性应当易于模仿
Learning from Teaching Regularization: Generalizable Correlations Should be Easy to Imitate
AI总结：
针对机器学习模型泛化难题，本文提出新型正则化技术LoT，通过辅助学生学习器与主模型的教学反馈循环捕捉可泛化相关性，在多领域实验中效果显著，可有效剔除虚假相关性。
摘要：
泛化能力仍是机器学习领域的核心挑战。本文提出了“从教中学（Learning from Teaching，LoT）”这一面向深度神经网络的新型正则化技术，用以提升模型泛化能力。受人类捕捉简洁抽象模式的能力启发，我们提出假设：可泛化的相关性通常更易于被模仿。LoT将这一理念落地，通过辅助学生学习器来提升主模型的泛化能力：学生学习器由主模型进行训练，反过来又提供反馈，帮助主模型捕捉更具泛化性、更易模仿的相关性。我们在计算机视觉、自然语言处理以及强化学习等方法学的多个领域开展了实验，结果表明，与仅在原始数据集上训练模型相比，引入LoT能带来显著收益。实验结果证实，LoT能有效且高效地识别恰当尺度下的可泛化信息，同时剔除数据中的虚假相关性，是现有机器学习方法的有益补充。代码已开源至https://github.com/jincan333/LoT。
英文摘要：
Generalization remains a central challenge in machine learning. In this work, we propose Learning from Teaching (LoT), a novel regularization technique for deep neural networks to enhance generalization. Inspired by the human ability to capture concise and abstract patterns, we hypothesize that generalizable correlations are expected to be easier to imitate. LoT operationalizes this concept to improve the generalization of the main model with auxiliary student learners. The student learners are trained by the main model and, in turn, provide feedback to help the main model capture more generalizable and imitable correlations. Our experimental results across several domains, including Computer Vision, Natural Language Processing, and methodologies like Reinforcement Learning, demonstrate that the introduction of LoT brings significant benefits compared to training models on the original dataset. The results suggest the effectiveness and efficiency of LoT in identifying generalizable information at the right scales while discarding spurious data correlations, thus making LoT a valuable addition to current machine learning. Code is available at https://github.com/jincan333/LoT.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
