学习树结构的数据增强组合
Learning Tree-Structured Composition of Data Augmentation
AI总结：
提出一种二叉树结构的数据增强组合搜索算法，将复杂度从$O(k^d)$降至$O(2^d k)$，在图和图像数据集上计算成本降低43%且性能提升4.3%，并能解释变换的相对重要性。
摘要：
数据增强被广泛用于在标注数据较少时训练神经网络。增强训练的一种常见做法是将多个变换的序列组合依次应用于数据。现有的增强方法如 RandAugment 从预选的变换列表中随机采样，而像 AutoAugment 这样的方法则应用高级搜索来优化大小为 $k^d$ 的增强集合，即在给定 $k$ 个变换的列表时，长度为 $d$ 的变换序列的数量。在本文中，我们设计了高效的算法，可证明其运行时间复杂度远快于最坏情况复杂度 $O(k^d)$。我们提出了一种新算法来搜索 $k$ 个变换的二叉树结构组合，其中每个树节点对应一个变换。该二叉树泛化了序列增强，例如用于对比学习的 SimCLR 增强方案。使用自顶向下的递归搜索过程，我们的算法实现了 $O(2^d k)$ 的运行时复杂度，当 $k$ 增加到 2 以上时，这比 $O(k^d)$ 快得多。我们应用该算法处理具有异质子群的数据分布，方法是在每个子群中搜索一棵树，然后学习加权组合，从而形成树的森林。我们在众多图和图像数据集上验证了提出的算法，包括我们收集的一个多标签图分类数据集。该数据集在图的大小及其平均度上表现出显著差异，使其成为研究数据增强的理想选择。我们表明，与现有搜索方法相比，我们的方法可以将计算成本降低 43%，同时将性能提高 4.3%。树结构可用于解释每个变换的相对重要性，例如识别小图与大图上的重要变换。
英文摘要：
Data augmentation is widely used for training a neural network given little labeled data. A common practice of augmentation training is applying a composition of multiple transformations sequentially to the data. Existing augmentation methods such as RandAugment randomly sample from a list of pre-selected transformations, while methods such as AutoAugment apply advanced search to optimize over an augmentation set of size $k^d$, which is the number of transformation sequences of length $d$, given a list of $k$ transformations. In this paper, we design efficient algorithms whose running time complexity is much faster than the worst-case complexity of $O(k^d)$, provably. We propose a new algorithm to search for a binary tree-structured composition of $k$ transformations, where each tree node corresponds to one transformation. The binary tree generalizes sequential augmentations, such as the SimCLR augmentation scheme for contrastive learning. Using a top-down, recursive search procedure, our algorithm achieves a runtime complexity of $O(2^d k)$, which is much faster than $O(k^d)$ as $k$ increases above $2$. We apply our algorithm to tackle data distributions with heterogeneous subpopulations by searching for one tree in each subpopulation and then learning a weighted combination, resulting in a forest of trees. We validate our proposed algorithms on numerous graph and image datasets, including a multi-label graph classification dataset we collected. The dataset exhibits significant variations in the sizes of graphs and their average degrees, making it ideal for studying data augmentation. We show that our approach can reduce the computation cost by 43% over existing search methods while improving performance by 4.3%. The tree structures can be used to interpret the relative importance of each transformation, such as identifying the important transformations on small vs. large graphs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
