We consider the problem of dynamically selecting among competing strategies using observable predictors. To mitigate estimation noise and excessive switching, we regularise the estimated policy with a switching cost, which may exceed actual transaction costs when these are too small to provide sufficient regularisation. Since switching costs make current choices affect future decisions, we formulate the problem using dynamic programming. The optimal policy is estimated using a reinforcement learning algorithm based on iterated regressions, making the procedure simple to implement. We apply the method to monthly rotation among momentum, value, profitability, and defensive U.S. equity factors over a sample of about 60 years. Using a parsimonious set of state variables, the estimated policy outperforms both the myopic dynamic policy and the best fixed factor in hindsight in out-of-sample evaluation. Under weak conditions, we show theoretically that the decision rule estimated from empirical data is asymptotically consistent for the unknown optimal decision rule. Simulation results illustrate the finite-sample relevance of these asymptotic results.
DeepMind（深度思维）；UC Berkeley（加州大学伯克利分校）；University of Massachusetts（马萨诸塞大学）
We consider model selection in stochastic bandit and reinforcement learning problems. Given a set of base learning algorithms, an effective model selection strategy adapts to the best learning algorithm in an online fashion. We show that by estimating the regret of each algorithm and playing the algorithms such that all empirical regrets are ensured to be of the same order, the overall regret balancing strategy achieves a regret that is close to the regret of the optimal base algorithm. Our strategy requires an upper bound on the optimal base regret as input, and the performance of the strateg…
The transition from defined benefit to defined contribution pension plans shifts the responsibility for saving toward retirement from governments and institutions to the individuals. Determining optimal saving and investment strategy for individuals is paramount for stable financial stance and for avoiding poverty during work-life and retirement, and it is a particularly challenging task in a world where form of employment and income trajectory experienced by different occupation groups are highly diversified. We introduce a model in which agents learn optimal portfolio allocation and saving s…
We consider off-policy evaluation in the contextual bandit setting for the purpose of obtaining a robust off-policy selection strategy, where the selection strategy is evaluated based on the value of the chosen policy in a set of proposal (target) policies. We propose a new method to compute a lower bound on the value of an arbitrary target policy given some logged data in contextual bandits for a desired coverage. The lower bound is built around the so-called Self-normalized Importance Weighting (SN) estimator. It combines the use of a semi-empirical Efron-Stein tail inequality to control the…
RADS: Reinforcement Learning-Based Sample Selection Improves Transfer Learning in Low-resource and Imbalanced Clinical Settings
School of Computing Technologies, RMIT University（皇家墨尔本理工大学计算技术学院）；School of Computing and Information Systems, The University of Melbourne（墨尔本大学计算与信息学院）；National Centre for Infections in Cancer, Melbourne（癌症感染国家研究中心，墨尔本）；Department of Infectious Disease, Peter MacCallum Cancer Centre（彼得·麦克卡伦癌症中心感染病科）；Sir Peter MacCallum Department of Oncology, The University of Melbourne（彼得·麦克卡伦肿瘤学系，墨尔本大学）
Beyond Cross-Validation: Adaptive Parameter Selection for Kernel-Based Gradient Descents
Center for Intelligent Decision-Making and Machine Learning, School of Management, Xi’an Jiaotong University, Xi’an, China（智能决策与机器学习中心，管理学院，西安交通大学，西安，中国）；Department of Mathematics, The University of Hong Kong, Hongkong, China（数学系，香港大学，香港，中国）
Mercedes-Benz AG（梅赛德斯-奔驰股份公司）；Institute of Automotive Technology Dresden（德累斯顿汽车技术研究所）
The test bench time needed for model-based calibration can be reduced with active learning methods for test design. This paper presents an improved strategy for active output selection. This is the task of learning multiple models in the same input dimensions and suits the needs of calibration tasks. Compared to an existing strategy, we take into account the noise estimate, which is inherent to Gaussian processes. The method is validated on three different toy examples. The performance compared to the existing best strategy is the same or better in each example. In a best case scenario, the ne…
Northeast Normal University（东北师范大学）；University of Central Florida（中佛罗里达大学）
Feature selection aims to select a subset of features to optimize the performances of downstream predictive tasks. Recently, multi-agent reinforced feature selection (MARFS) has been introduced to automate feature selection, by creating agents for each feature to select or deselect corresponding features. Although MARFS enjoys the automation of the selection process, MARFS suffers from not just the data complexity in terms of contents and dimensionality, but also the exponentially-increasing computational costs with regard to the number of agents. The raised concern leads to a new research que…
Client-wise data heterogeneity is one of the major issues that hinder effective training in federated learning (FL). Since the data distribution on each client may vary dramatically, the client selection strategy can significantly influence the convergence rate of the FL process. Active client selection strategies are popularly proposed in recent studies. However, they neglect the loss correlations between the clients and achieve only marginal improvement compared to the uniform selection strategy. In this work, we propose FedCor -- an FL framework built on a correlation-based client selection…
Machine learning has opened up new tools for financial fraud detection. Using a sample of annotated transactions, a machine learning classification algorithm learns to detect frauds. With growing credit card transaction volumes and rising fraud percentages there is growing interest in finding appropriate machine learning classifiers for detection. However, fraud data sets are diverse and exhibit inconsistent characteristics. As a result, a model effective on a given data set is not guaranteed to perform on another. Further, the possibility of temporal drift in data patterns and characteristics…
University of Central Florida（中佛罗里达大学）；DAMO Academy, Alibaba Group（阿里巴巴达摩院）；University of California, Merced（加州大学默塞德分校）；University of Florida（佛罗里达大学）
In this paper, we propose a single-agent Monte Carlo based reinforced feature selection (MCRFS) method, as well as two efficiency improvement strategies, i.e., early stopping (ES) strategy and reward-level interactive (RI) strategy. Feature selection is one of the most important technologies in data prepossessing, aiming to find the optimal feature subset for a given downstream machine learning task. Enormous research has been done to improve its effectiveness and efficiency. Recently, the multi-agent reinforced feature selection (MARFS) has achieved great success in improving the performance …
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）
Automating machine learning has achieved remarkable technological developments in recent years, and building an automated machine learning pipeline is now an essential task. The model ensemble is the technique of combining multiple models to get a better and more robust model. However, existing automated machine learning tends to be simplistic in handling the model ensemble, where the ensemble strategy is fixed, such as stacked generalization. There have been many techniques on different ensemble methods, especially ensemble selection, and the fixed ensemble strategy limits the upper limit of …
This work proposes a novel portfolio management technique, the Meta Portfolio Method (MPM), inspired by the successes of meta approaches in the field of bioinformatics and elsewhere. The MPM uses XGBoost to learn how to switch between two risk-based portfolio allocation strategies, the Hierarchical Risk Parity (HRP) and more classical Naïve Risk Parity (NRP). It is demonstrated that the MPM is able to successfully take advantage of the best characteristics of each strategy (the NRP's fast growth during market uptrends, and the HRP's protection against drawdowns during market turmoil). As a res…
Dept. Computing Science & Alberta Machine Intelligence Institute (Amii), University of Alberta（计算科学系及阿尔伯塔人工智能研究所（Amii），阿尔伯塔大学）；Canada CIFAR AI Chair, Amii（加拿大CIFAR人工智能主席，Amii）
For sparse matrices up to size $8 \times 8$, we determine optimal choices for pivot selection in Gaussian elimination. It turns out that they are slightly better than the pivots chosen by a popular pivot selection strategy, so there is some room for improvement. We then create a pivot selection strategy using machine learning and find that it indeed leads to a small improvement compared to the classical strategy.
AQUILA: Communication Efficient Federated Learning with Adaptive Quantization in Device Selection Strategy
Tsinghua University（清华大学）；Tsinghua Shenzhen International Graduate School（清华大学深圳国际研究生院）；Tsinghua-Berkeley Shenzhen Institute（清华-伯克利深圳学院）；Institute for AI Industry Research (AIR), Tsinghua University（清华大学人工智能产业研究院（AIR））；Shanghai AI Lab（上海人工智能实验室）；George Washington University（乔治华盛顿大学）；Ryerson University（瑞尔森大学）
Lebanese American University（黎巴嫩美国大学）；Polytechnique Montréal（蒙特利尔理工学院）；Khalifa University（哈利法大学）；Zayed University（扎耶德大学）；New York University（纽约大学）；Mohammad Bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Federated Learning (FL) is a novel distributed privacy-preserving learning paradigm, which enables the collaboration among several participants (e.g., Internet of Things devices) for the training of machine learning models. However, selecting the participants that would contribute to this collaborative training is highly challenging. Adopting a random selection strategy would entail substantial problems due to the heterogeneity in terms of data quality, and computational and communication resources across the participants. Although several approaches have been proposed in the literature to ove…
Active learning shows promise to decrease test bench time for model-based drivability calibration. This paper presents a new strategy for active output selection, which suits the needs of calibration tasks. The strategy is actively learning multiple outputs in the same input space. It chooses the output model with the highest cross-validation error as leading. The presented method is applied to three different toy examples with noise in a real world range and to a benchmark dataset. The results are analyzed and compared to other existing strategies. In a best case scenario, the presented strat…
Emulating Full Participation: An Effective and Fair Client Selection Strategy for Federated Learning
The Chinese University of Hong Kong（香港中文大学）；Zhejiang Lab（之江实验室）；School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-sen University（中山大学深圳校区网络空间安全学院）；Tsinghua University（清华大学）
Boosting Backdoor Attack with A Learnable Poisoning Sample Selection Strategy
The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Shenzhen Research Institute of Big Data（深圳大数据研究院）；JD Explore Academy（京东探索研究院）；Tencent AI Lab（腾讯人工智能实验室）
