解耦与稀疏性之间的协同作用：多任务学习中的泛化与可辨识性
Synergies between Disentanglement and Sparsity: Generalization and Identifiability in Multi-Task Learning
AI总结：
本文研究多任务学习中解耦表示与稀疏基预测器的协同关系，证明稀疏性带来可辨识性的条件，并提出双层优化与元学习算法，在小样本分类基准上取得竞争力结果。
摘要：
尽管解耦表示通常被认为有利于下游任务，但目前的实证和理论理解仍然有限。在本工作中，我们提供证据表明，解耦表示与稀疏基预测器相结合可以改善泛化。在多任务学习背景下，我们证明了一个新的可辨识性结果，给出了使最大稀疏基预测器产生解耦表示的条件。受这一理论结果启发，我们提出了一种基于促进稀疏性的双层优化问题来学习解耦表示的实用方法。最后，我们探索了该算法的元学习版本，其基于 group Lasso 多类 SVM 基预测器，并为此推导了易于处理的对偶形式。它在标准小样本分类基准上取得了具有竞争力的结果，同时每个任务仅使用所学表示的一小部分。
英文摘要：
Although disentangled representations are often said to be beneficial for downstream tasks, current empirical and theoretical understanding is limited. In this work, we provide evidence that disentangled representations coupled with sparse base-predictors improve generalization. In the context of multi-task learning, we prove a new identifiability result that provides conditions under which maximally sparse base-predictors yield disentangled representations. Motivated by this theoretical result, we propose a practical approach to learn disentangled representations based on a sparsity-promoting bi-level optimization problem. Finally, we explore a meta-learning version of this algorithm based on group Lasso multiclass SVM base-predictors, for which we derive a tractable dual formulation. It obtains competitive results on standard few-shot classification benchmarks, while each task is using only a fraction of the learned representations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
