研究比较概率满足强随机传递性和随机三角不等式的一般概率模型的$(ε,δ)$ -PAC极大选择和排序问题。修改流行的淘汰赛，我们提出了一个最大选择算法，使用$\mathcal{O}\left(\frac{n}{ε^2}\log \frac{1}δ\right)$比较，一个数字紧到一个常数因子。然后，我们推导出一个通用框架，该框架可以提高许多排名算法的性能，并将其与合并排序和二分搜索相结合，以获得对任何$δ\ge\frac1n$使用$\mathcal{O}\left(\frac{n\log n (\log \log n)^3}{ε^2}\right)$比较的排名算法，该$δ\ge\frac1n$是最佳数字，最高可达$(\log \log n)^3$因子。
英文摘要：
We consider $(ε,δ)$-PAC maximum-selection and ranking for general probabilistic models whose comparisons probabilities satisfy strong stochastic transitivity and stochastic triangle inequality. Modifying the popular knockout tournament, we propose a maximum-selection algorithm that uses $\mathcal{O}\left(\frac{n}{ε^2}\log \frac{1}δ\right)$ comparisons, a number tight up to a constant factor. We then derive a general framework that improves the performance of many ranking algorithms, and combine it with merge sort and binary search to obtain a ranking algorithm that uses $\mathcal{O}\left(\frac{n\log n (\log \log n)^3}{ε^2}\right)$ comparisons for any $δ\ge\frac1n$, a number optimal up to a $(\log \log n)^3$ factor.
We consider the problem of learning the true ordering of a set of alternatives from largely incomplete and noisy rankings. We introduce a natural generalization of both the classical Mallows model of ranking distributions and the extensively studied model of noisy pairwise comparisons. Our selective Mallows model outputs a noisy ranking on any given subset of alternatives, based on an underlying Mallows distribution. Assuming a sequence of subsets where each pair of alternatives appears frequently enough, we obtain strong asymptotically tight upper and lower bounds on the sample complexity of …
This paper studies the problem of finding the exact ranking from noisy comparisons. A comparison over a set of $m$ items produces a noisy outcome about the most preferred item, and reveals some information about the ranking. By repeatedly and adaptively choosing items to compare, we want to fully rank the items with a certain confidence, and use as few comparisons as possible. Different from most previous works, in this paper, we have three main novelties: (i) compared to prior works, our upper bounds (algorithms) and lower bounds on the sample complexity (aka number of comparisons) require th…
Paderborn University（帕德博恩大学）；Leibniz University Hannover（汉诺威莱布尼茨大学）
It is well known that different algorithms perform differently well on an instance of an algorithmic problem, motivating algorithm selection (AS): Given an instance of an algorithmic problem, which is the most suitable algorithm to solve it? As such, the AS problem has received considerable attention resulting in various approaches - many of which either solve a regression or ranking problem under the hood. Although both of these formulations yield very natural ways to tackle AS, they have considerable weaknesses. On the one hand, correctly predicting the performance of an algorithm on an inst…
We study the problem of learning an unknown mixture of $k$ rankings over $n$ elements, given access to noisy samples drawn from the unknown mixture. We consider a range of different noise models, including natural variants of the "heat kernel" noise framework and the Mallows model. For each of these noise models we give an algorithm which, under mild assumptions, learns the unknown mixture to high accuracy and runs in $n^{O(\log k)}$ time. The best previous algorithms for closely related problems have running times which are exponential in $k$.
University of Virginia（弗吉尼亚大学）；University of California, Los Angeles（加州大学洛杉矶分校）；California Institute of Technology（加州理工学院）
In heterogeneous rank aggregation problems, users often exhibit various accuracy levels when comparing pairs of items. Thus a uniform querying strategy over users may not be optimal. To address this issue, we propose an elimination-based active sampling strategy, which estimates the ranking of items via noisy pairwise comparisons from users and improves the users' average accuracy by maintaining an active set of users. We prove that our algorithm can return the true ranking of items with high probability. We also provide a sample complexity bound for the proposed algorithm which is better than…
Meta AI；CREST, ENSAE（法国国立统计与经济管理学院经济与统计研究中心）；Core Data Science, Meta（Meta核心数据科学部）
We consider a multi-armed bandit setting where, at the beginning of each round, the learner receives noisy independent, and possibly biased, \emph{evaluations} of the true reward of each arm and it selects $K$ arms with the objective of accumulating as much reward as possible over $T$ rounds. Under the assumption that at each round the true reward of each arm is drawn from a fixed distribution, we derive different algorithmic approaches and theoretical guarantees depending on how the evaluations are generated. First, we show a $\widetilde{O}(T^{2/3})$ regret in the general case when the observ…
We introduce supervised feature ranking and feature subset selection algorithms for multivariate time series (MTS) classification. Unlike most existing supervised/unsupervised feature selection algorithms for MTS our techniques do not require a feature extraction step to generate a one-dimensional feature vector from the time series. Instead it is based on directly computing similarity between individual time series and assessing how well the resulting cluster structure matches the labels. The techniques are amenable to heterogeneous MTS data, where the time series measurements may have differ…
Pohang University of Science and Technology（浦项科技大学）；Seoul National University of Science and Technology (SeoulTech)（首尔国立科技大学（首尔科技大学））
Because deep learning is vulnerable to noisy labels, sample selection techniques, which train networks with only clean labeled data, have attracted a great attention. However, if the labels are dominantly corrupted by few classes, these noisy samples are called dominant-noisy-labeled samples, the network also learns dominant-noisy-labeled samples rapidly via content-aware optimization. In this study, we propose a compelling criteria to penalize dominant-noisy-labeled samples intensively through class-wise penalty labels. By averaging prediction confidences for the each observed label, we obtai…
The fair-ranking problem, which asks to rank a given set of items to maximize utility subject to group fairness constraints, has received attention in the fairness, information retrieval, and machine learning literature. Recent works, however, observe that errors in socially-salient (including protected) attributes of items can significantly undermine fairness guarantees of existing fair-ranking algorithms and raise the problem of mitigating the effect of such errors. We study the fair-ranking problem under a model where socially-salient attributes of items are randomly and independently pertu…
This manuscript presents the following: (1) an improved version of the Binary Simultaneous Perturbation Stochastic Approximation (SPSA) Method for feature selection in machine learning (Aksakalli and Malekipirbazari, Pattern Recognition Letters, Vol. 75, 2016) based on non-monotone iteration gains computed via the Barzilai and Borwein (BB) method, (2) its adaptation for feature ranking, and (3) comparison against popular methods on public benchmark datasets. The improved method, which we call SPSA-FSR, dramatically reduces the number of iterations required for convergence without impacting sol…
Non-convex Regularizations for Feature Selection in Ranking With Sparse SVM
Institut de Recherche en Informatique de Toulouse（图卢兹计算机科学研究所）；CNRS（法国国家科学研究中心）；Université de Toulouse（图卢兹大学）；Nomao SA（Nomao公司）；Laboratoire Lagrange（拉格朗日实验室）；Université de Nice-Sophia Antipolis（尼斯-索菲亚·安提波利斯大学）；INSA de Rouen（鲁昂国立应用科学学院）；Institut de Mathématiques de Toulouse（图卢兹数学研究所）；Institut Universitaire de Formation de Maîtres（大学教师培训学院）
Online Multi-Label Classification under Noisy and Changing Label Distribution
School of Computer Science and Information Engineering, Hefei University of Technology（合肥工业大学计算机与信息工程学院）；School of Computing, National University of Singapore（新加坡国立大学计算学院）
The Science4cast Competition consists of predicting new links in a semantic network, with each node representing a concept and each edge representing a link proposed by a paper relating two concepts. This network contains information from 1994-2017, with a discretization of days (which represents the publication date of the underlying papers). Team Hash Brown's final submission, \emph{ee5a}, achieved a score of 0.92738 on the test set. Our team's score ranks \emph{second place}, 0.01 below the winner's score. This paper details our model, its intuition, and the performance of its variations in…
Accelerating Benchmarking of Functional Connectivity Modeling via Structure-aware Core-set Selection
College of Computer and Information Science, Southwest University（西南大学计算机与信息科学学院）；Chongqing Key Laboratory of Brain-Inspired Cognitive Computing and Educational Rehabilitation for Children with Special Needs, Chongqing Normal University（重庆脑启发认知计算与特殊需要儿童教育康复重点实验室，重庆师范大学）；College of Computer and Information Science, Chongqing Normal University（重庆师范大学计算机与信息科学学院）
George Mason University（乔治梅森大学）；Los Alamos National Laboratory（洛斯阿拉莫斯国家实验室）
Identifying one or more biologically-active/native decoys from millions of non-native decoys is one of the major challenges in computational structural biology. The extreme lack of balance in positive and negative samples (native and non-native decoys) in a decoy set makes the problem even more complicated. Consensus methods show varied success in handling the challenge of decoy selection despite some issues associated with clustering large decoy sets and decoy sets that do not show much structural similarity. Recent investigations into energy landscape-based decoy selection approaches show pr…
Deep neural networks (DNNs) suffer from noisy-labeled data because of the risk of overfitting. To avoid the risk, in this paper, we propose a novel DNN training method with sample selection based on adaptive k-set selection, which selects k (< n) clean sample candidates from the whole n noisy training samples at each epoch. It has a strong advantage of guaranteeing the performance of the selection theoretically. Roughly speaking, a regret, which is defined by the difference between the actual selection and the best selection, of the proposed method is theoretically bounded, even though the bes…
DEVCOM Army Research Laboratory（美国陆军研究实验室）；University of Maryland（马里兰大学）
In today's technology environment, information is abundant, dynamic, and heterogeneous in nature. Automated filtering and prioritization of information is based on the distinction between whether the information adds substantial value toward one's goal or not. Contextual multi-armed bandit has been widely used for learning to filter contents and prioritize according to user interest or relevance. Learn-to-Rank technique optimizes the relevance ranking on items, allowing the contents to be selected accordingly. We propose a novel approach to top-K rankings under the contextual multi-armed bandi…
Politecnico di Milano（米兰理工大学）；ContentWise；Università degli Studi di Padova（帕多瓦大学）
Feature selection is a common step in many ranking, classification, or prediction tasks and serves many purposes. By removing redundant or noisy features, the accuracy of ranking or classification can be improved and the computational cost of the subsequent learning steps can be reduced. However, feature selection can be itself a computationally expensive process. While for decades confined to theoretical algorithmic papers, quantum computing is now becoming a viable tool to tackle realistic problems, in particular special-purpose solvers based on the Quantum Annealing paradigm. This paper aim…
Communication University of China（中国传媒大学）；Guangzhou First People’s Hospital（广州市第一人民医院）；South China University of Technology（华南理工大学）；Xidian University（西安电子科技大学）；Stanford University（斯坦福大学）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）
Feature selection is important in data representation and intelligent diagnosis. Elastic net is one of the most widely used feature selectors. However, the features selected are dependant on the training data, and their weights dedicated for regularized regression are irrelevant to their importance if used for feature ranking, that degrades the model interpretability and extension. In this study, an intuitive idea is put at the end of multiple times of data splitting and elastic net based feature selection. It concerns the frequency of selected features and uses the frequency as an indicator o…
Series Expansion of Probability of Correct Selection for Improved Finite Budget Allocation in Ranking and Selection
PKU-Wuhan Institute for Artificial Intelligence（北大-武汉人工智能研究院）；Guanghua School of Management, Peking University（北京大学光华管理学院）；Xiangjiang Laboratory（湘江实验室）；Inria（法国国家信息与自动化研究所）；Univ Rennes（雷恩大学）；CNRS（法国国家科学研究中心）；IRISA（法国信息与随机系统研究所）
Drug Selection via Joint Push and Learning to Rank
Georgia Institute of Technology（佐治亚理工学院）；Twentieth Century Fox（二十世纪福克斯公司）；Starfleet Academy（星际舰队学院）；Indiana University - Purdue University Indianapolis（印第安纳大学-普渡大学印第安纳波利斯分校）
Tokyo Institute of Technology（东京工业大学）；CyberAgent, Inc（CyAgent公司）
We study the model selection problem in conditional average treatment effect (CATE) prediction. Unlike previous works on this topic, we focus on preserving the rank order of the performance of candidate CATE predictors to enable accurate and stable model selection. To this end, we analyze the model performance ranking problem and formulate guidelines to obtain a better evaluation metric. We then propose a novel metric that can identify the ranking of the performance of CATE predictors with high confidence. Empirical evaluations demonstrate that our metric outperforms existing metrics in both m…
Ranking over Regression for Bayesian Optimization and Molecule Selection
University of Toronto（多伦多大学）；Vector Institute for Artificial Intelligence（向量人工智能研究所）；CIFAR Lebovic Fellow（CIFAR 莱博维奇 Fellow）；Canada（加拿大）；Chair, Director of Acceleration Consortium（150 职位，加速联盟主任）
Nanyang Technological University（南洋理工大学）；City University of Hong Kong（香港城市大学）；University of Tennessee-Knoxville（田纳西大学诺克斯维尔分校）
Ranking and selection (R&S) is a popular model for studying discrete-event dynamic systems. It aims to select the best design (the design with the largest mean performance) from a finite set, where the mean of each design is unknown and has to be learned by samples. Great research efforts have been devoted to this problem in the literature for developing procedures with superior empirical performance and showing their optimality. In these efforts, myopic procedures were popular. They select the best design using a 'naive' mechanism of iteratively and myopically improving an approximation of th…
Texas A&M University（德州农工大学）；University of Virginia（弗吉尼亚大学）
We analyze the performance of the Borda counting algorithm in a non-parametric model. The algorithm needs to utilize probabilistic rankings of the items within $m$-sized subsets to accurately determine which items are the overall top-$k$ items in a total of $n$ items. The Borda counting algorithm simply counts the cumulative scores for each item from these partial ranking observations. This generalizes a previous work of a similar nature by Shah et al. using probabilistic pairwise comparison data. The performance of the Borda counting algorithm critically depends on the associated score separa…
Goethe University Frankfurt（法兰克福歌德大学）；Free University Berlin（柏林自由大学）；Singapore University of Technology and Design（新加坡科技设计大学）
In this paper, we tackle an open research question in transfer learning, which is selecting a model initialization to achieve high performance on a new task, given several pre-trained models. We propose a new highly efficient and accurate approach based on duality diagram similarity (DDS) between deep neural networks (DNNs). DDS is a generic framework to represent and compare data of different feature dimensions. We validate our approach on the Taskonomy dataset by measuring the correspondence between actual transfer learning performance rankings on 17 taskonomy tasks and predicted rankings. C…
Optimizing Feature Selection for Binary Classification with Noisy Labels: A Genetic Algorithm Approach
A. I. Virtanen Institute for Molecular Sciences, University of Eastern Finland（A.I. Virtanen分子科学研究所，东芬兰大学）；Department of Computer Science, Aalto University（计算机科学系，艾尔托大学）；Institute of Biomedicine, University of Eastern Finland（生物医学研究所，东芬兰大学）
Transformer-based rankers have shown state-of-the-art performance. However, their self-attention operation is mostly unable to process long sequences. One of the common approaches to train these rankers is to heuristically select some segments of each document, such as the first segment, as training data. However, these segments may not contain the query-related parts of documents. To address this problem, we propose query-driven segment selection from long documents to build training data. The segment selector provides relevant samples with more accurate labels and non-relevant samples which …
Learning with Imbalanced Noisy Data by Preventing Bias in Sample Selection
School of Computer Science and Engineering, Nanjing University of Science and Technology（南京理工大学计算机科学与工程学院）；Terminus Group（特步集团）；School of Electronic and Information Engineering, Tongji University（同济大学电子与信息工程学院）
Knowledge graph based simple question answering (KBSQA) is a major area of research within question answering. Although only dealing with simple questions, i.e., questions that can be answered through a single knowledge base (KB) fact, this task is neither simple nor close to being solved. Targeting on the two main steps, subgraph selection and fact selection, the research community has developed sophisticated approaches. However, the importance of subgraph ranking and leveraging the subject--relation dependency of a KB fact have not been sufficiently explored. Motivated by this, we present a …
