In experimental design, we are given a large collection of vectors, each with a hidden response value that we assume derives from an underlying linear model, and we wish to pick a small subset of the vectors such that querying the corresponding responses will lead to a good estimator of the model. A classical approach in statistics is to assume the responses are linear, plus zero-mean i.i.d. Gaussian noise, in which case the goal is to provide an unbiased estimator with smallest mean squared error (A-optimal design). A related approach, more common in computer science, is to assume the responses are arbitrary but fixed, in which case the goal is to estimate the least squares solution using few responses, as quickly as possible, for worst-case inputs. Despite many attempts, characterizing the relationship between these two approaches has proven elusive. We address this by proposing a framework for experimental design where the responses are produced by an arbitrary unknown distribution. We show that there is an efficient randomized experimental design procedure that achieves strong variance bounds for an unbiased estimator using few responses in this general model. Nearly tight bounds for the classical A-optimality criterion, as well as improved bounds for worst-case responses, emerge as special cases of this result. In the process, we develop a new algorithm for a joint sampling distribution called volume sampling, and we propose a new i.i.d. importance sampling method: inverse score sampling. A key novelty of our analysis is in developing new expected error bounds for worst-case regression by controlling the tail behavior of i.i.d. sampling via the jointness of volume sampling. Our result motivates a new minimax-optimality criterion for experimental design which can be viewed as an extension of both A-optimal design and sampling for worst-case regression.
Worst-case generation via minimax optimization in Wasserstein space
Department of Mathematics, Duke University（杜克大学数学系）；H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology（佐治亚理工学院H. Milton Stewart工业与系统工程学院）
Bridging Theory and Algorithm for Domain Adaptation
Tsinghua University（清华大学）；University of California, Berkeley（加州大学伯克利分校）
本文从理论和算法两个方面研究了无监督域自适应问题。现有的领域自适应理论自然包含极大极小优化算法，极大极小优化算法与基于对抗学习的领域自适应方法有很好的结合。然而，一些脱节仍然存在，形成理论和算法之间的差距。我们扩展了以前的理论（Mansour et al.，2009 c; Ben-David等人，2010）到领域自适应中的多类分类，其中基于评分函数和边际损失的分类器是算法设计中的标准选择。我们介绍了利润差距离散，一种新的测量与严格的推广界，量身定制的分布比较与不对称的利润损失，和最小最大优化更容易训练。我们的理论可以无缝地转化为一个对抗学习算法的领域自适应，成功地弥合了理论和算法之间的差距。一系列的实证研究表明，我们的算法达到了最先进的精度上具有挑战性的领域自适应任务。
Clustering under Local Stability: Bridging the Gap between Worst-Case and Beyond Worst-Case Analysis
最近，人们对聚类研究产生了浓厚的兴趣，这种研究采用了超越最坏情况的方法来分析算法。典型的想法是设计一个聚类算法，输出一个接近最优的解决方案，提供的数据满足自然的稳定性概念。例如，Bilu and Linial（2010）和Awasthi et al.（2012）提出了输出近似最优解的算法，假设最优解在输入距离的小扰动下保持不变。这种方法的一个缺点是，算法通常是根据稳定性假设显式构建的，并且在最坏的情况下没有保证;事实上，最近的几个算法输出任意坏的解决方案，即使只有一小部分数据不满足给定的稳定性概念。在这项工作中，我们以两种方式解决这一问题。首先，我们提供的算法，继承聚类近似算法的最坏情况下的保证，同时保证接近最优的解决方案时，数据是稳定的。我们的算法是对现有最先进的近似算法的自然修改。其次，我们开始研究局部稳定性，这是一个单一的最优集群，而不是整个最优解的属性。我们表明，我们的算法输出所有的最佳集群，满足稳定的局部。具体而言，我们在最近的稳定性概念下，在我们的局部框架中实现了强有力的积极结果，包括度量扰动弹性（Angelidakis et al. 2017）和稳健的扰动弹性（Balcan and Liang 2012），用于$k$ -中位数，$k$ -均值和对称/非对称$k$ -中心目标。
Generalized Neyman Allocation for Locally Minimax Optimal Best-Arm Identification
Graduate School of Arts and Sciences, The University of Tokyo（东京大学文学部学院）；Data Analytics Department, Mizuho–DL Financial Technology, Co., Ltd.（瑞穗数字金融科技有限公司数据分析部）
Online Algorithms via Minimax and Posterior Matching
University of Bonn（波恩大学）；Institute of Computer Science, University of Bonn（波恩大学计算机科学学院）；Microsoft Research（微软研究院）；PUC-Rio（里约热内卢天主教大学）；Georgia Tech（佐治亚理工学院）；School of Mathematics, Georgia Tech（佐治亚理工学院数学学院）；School of Computer Science, Georgia Tech（佐治亚理工学院计算机科学学院）
MaxMatch: Semi-Supervised Learning with Worst-Case Consistency
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Alibaba Group（阿里巴巴集团）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；Sun Yat-sen University（中山大学）；Peng Cheng Laboratory（鹏城实验室）
FlatMatch: Bridging Labeled Data and Unlabeled Data with Cross-Sharpness for Semi-Supervised Learning
Sydney AI Centre, The University of Sydney（悉尼大学悉尼人工智能中心）；JD Explore Academy（京东探索研究院）；University of Science and Technology of China（中国科学技术大学）；Hong Kong Baptist University（香港浸会大学）
