To appear as a chapter in the Handbook of Statistical Methods for Precision Medicine edited by Tianxi Cai, Bibhas Chakraborty, Eric Laber, Erica Moodie, and Mark van der Laan
《牛津英语词典》将精准医疗定义为“旨在优化特定患者群体的效率或治疗益处的医疗护理，特别是通过使用遗传或分子分析。“这并不是一个全新的想法：古代的医生已经认识到，医疗需要考虑患者特征的个体差异。然而，现代精准医学运动是由一系列事件共同促成的：遗传学和药理学等领域的科学进步，移动设备和可穿戴传感器的技术进步，以及计算和数据科学的方法进步。本章介绍的是强盗算法：这是一个与精准医疗特别相关的数据科学领域。基于Bellman、Robbins、Lai等人的开创性工作，强盗算法已经在现代数据科学中占据了核心地位（Latterian and Szepesvari，2020）。Bandit算法可以用于任何需要做出治疗决策以优化某些健康结果的情况。由于精准医学专注于使用患者特征来指导治疗，因此上下文强盗算法特别有用，因为它们被设计为考虑这些信息。之前已经对Bandit算法在移动医疗和数字表型等精准医疗领域的作用进行了综述（Tewari和Murphy，2017; Rabbi等人，2019年）。自这些评论发表以来，强盗算法继续在移动健康中找到用途，并且在强盗算法的研究中出现了几个新的主题。本章是为统计学、机器学习和运筹学等领域的定量研究人员编写的，他们可能有兴趣了解更多关于移动医疗中使用的bandit算法的算法和数学细节。
英文摘要：
The Oxford English Dictionary defines precision medicine as "medical care designed to optimize efficiency or therapeutic benefit for particular groups of patients, especially by using genetic or molecular profiling." It is not an entirely new idea: physicians from ancient times have recognized that medical treatment needs to consider individual variations in patient characteristics. However, the modern precision medicine movement has been enabled by a confluence of events: scientific advances in fields such as genetics and pharmacology, technological advances in mobile devices and wearable sensors, and methodological advances in computing and data sciences.
This chapter is about bandit algorithms: an area of data science of special relevance to precision medicine. With their roots in the seminal work of Bellman, Robbins, Lai and others, bandit algorithms have come to occupy a central place in modern data science ( Lattimore and Szepesvari, 2020). Bandit algorithms can be used in any situation where treatment decisions need to be made to optimize some health outcome. Since precision medicine focuses on the use of patient characteristics to guide treatment, contextual bandit algorithms are especially useful since they are designed to take such information into account. The role of bandit algorithms in areas of precision medicine such as mobile health and digital phenotyping has been reviewed before (Tewari and Murphy, 2017; Rabbi et al., 2019). Since these reviews were published, bandit algorithms have continued to find uses in mobile health and several new topics have emerged in the research on bandit algorithms. This chapter is written for quantitative researchers in fields such as statistics, machine learning, and operations research who might be interested in knowing more about the algorithmic and mathematical details of bandit algorithms that have been used in mobile health.
A Biologically Plausible Benchmark for Contextual Bandit Algorithms in Precision Oncology Using in vitro Data
Harvard Medical School（哈佛医学院）；German Cancer Research Center（德国癌症研究中心）；Massachusetts Institute of Technology（麻省理工学院）；University of California, San Francisco（加州大学旧金山分校）；Harvard T.H. Chan School of Public Health（哈佛陈曾熙公共卫生学院）；Google Research（谷歌研究院）
An optimal algorithm for the Thresholding Bandit Problem
University of Potsdam（波茨坦大学）
我们研究了一个特定的\textit{combinatorial pure exploration stochastic bandit problem}，学习者的目的是找到一组武器，其手段是高于给定的阈值，达到给定的精度，和\textit{for a fixed time horizon}。我们提出了一个无参数算法的基础上，原来的启发式，并证明它是最佳的这个问题，通过推导匹配的上，下界。据我们所知，这是第一个非平凡的纯探索设置与\textit{fixed budget}的最佳策略构建。
Improving drug sensitivity predictions in precision medicine through active expert knowledge elicitation
Helsinki Institute for Information Technology HIIT（赫尔辛基信息技术研究所（HIIT））；Aalto University（阿尔托大学）；Institute for Molecular Medicine Finland (FIMM)（芬兰分子医学研究所（FIMM））；University of Helsinki（赫尔辛基大学）
Contextual Bandits for Resource-Constrained Devices using Probabilistic Learning
Sapienza University of Rome（罗马大学）；Örebro University（奥雷布罗大学）；Örebro University Linköping University（奥雷布罗大学-林奈大学）；Örebro University Research Institutes of Sweden（瑞典奥雷布罗大学研究机构）
Batched Single-Index Global Multi-Armed Bandits with Covariates
Department of Mathematics, Applied Mathematics and Statistics, Case Western Reserve University（数学系、应用数学与统计学系，凯斯西储大学）；Department of Statistics, Pennsylvania State University（统计系，宾夕法尼亚州立大学）
LIBRA: Language Model Informed Bandit Recourse Algorithm for Personalized Treatment Planning
McCombs School of Business, University of Texas at Austin（德克萨斯大学奥斯汀分校麦克斯韦商学院）；Naveen Jindal School of Management, University of Texas at Dallas（德克萨斯大学达拉斯分校奈文·金达管理学院）；Mays Business School, Texas A&M University（德克萨斯农工大学梅斯商学院）；Wharton School, University of Pennsylvania（宾夕法尼亚大学沃顿商学院）
SPARKLE: A Nonparametric Approach for Online Decision-Making with High-Dimensional Covariates
Department of Industrial Systems Engineering and Management, National University of Singapore（新加坡国立大学工业系统工程与管理系）；Division of Emerging Interdisciplinary Areas, The Hong Kong University of Science and Technology（香港科学与技术大学新兴跨学科领域分校）；Department of Industrial Engineering and Decision Analytics, The Hong Kong University of Science and Technology（香港科学与技术大学工业工程与决策分析系）
Faster Activity and Data Detection in Massive Random Access: A Multi-armed Bandit Approach
ShanghaiTech University（上海科技大学）；The Hong Kong Polytechnic University（香港理工大学）；Tsinghua University（清华大学）；Beijing National Research Center for Information Science and Technology（北京信息科学与技术国家研究中心）
Precision medicine as a control problem: Using simulation and deep reinforcement learning to discover adaptive, personalized multi-cytokine therapy for sepsis
Lawrence Livermore National Laboratory（劳伦斯利弗莫尔国家实验室）；University of Chicago（芝加哥大学）；LLNL（劳伦斯利弗莫尔国家实验室）
Technical Background for "A Precision Medicine Approach to Develop and Internally Validate Optimal Exercise and Weight Loss Treatments for Overweight and Obese Adults with Knee Osteoarthritis"
Department of Biostatistics, University of North Carolina（北卡罗来纳大学生物统计学系）；Division of Rheumatology, Allergy and Immunology and the Thurston Arthritis Research Center, University of North Carolina（北卡罗来纳大学风湿病学、过敏与免疫学系及瑟斯顿关节炎研究中心）；Department of Biostatistics and Data Science, Wake Forest School of Medicine（维克森林医学院生物统计学与数据科学系）；Department of Health and Exercise Science, Wake Forest University（维克森林大学健康与运动科学系）
我们在“为超重和肥胖成人膝关节骨关节炎患者开发和内部优化最佳运动和减肥治疗的精准医学方法”（Jiang et al. 2020）中为膝关节骨关节炎临床分析中开发的方法提供了额外的统计背景。Jiang et al. 2020提出了一种管道，用于使用精确医学模型学习最佳治疗规则，并使用Z检验将其与零阶模型进行比较。模型性能基于价值函数，这是一个预测每个决策规则未来回报的标量。折叠刀（即，采用留一交叉验证法估计IDEA中多个结果的价值函数及其方差。IDEA是一项随机临床试验，研究了三种干预措施（运动（E），饮食减肥（D）和D+E）对超重和肥胖的膝关节骨关节炎参与者的影响。在本报告中，我们将通过额外的统计背景来扩展讨论和论证。我们详细阐述了精确医学的背景，价值函数的刀切估计及其估计方差的推导，刀切估计的一致性，以及额外的模拟结果，反映了更多的刀切估计的性能。我们建议阅读Jiang et al. 2020，以了解膝关节骨关节炎最佳ITR的临床应用和解释以及对管道的整体理解，并建议使用本文了解基础统计学推导和方法。
Deconfounded Warm-Start Thompson Sampling with Applications to Precision Medicine
Daniels School of Business, Purdue University（普渡大学 Daniels 商学院）；Mays Business School, Texas A&M University（德克萨斯农工大学 Mays 商学院）；McCombs School of Business, University of Texas at Austin（德克萨斯大学奥斯汀分校 McCombs 商学院）
Large Language Models-Enabled Digital Twins for Precision Medicine in Rare Gynecological Tumors
Technical University of Munich (TUM), School of Medicine and Health, Klinikum rechts der Isar, TUM University Hospital（慕尼黑技术大学（TUM）、医学院与健康学院、右岸医院、TUM大学医院）；Center for Personalized Medicine (ZPM)（个性化医学中心）；German Cancer Consortium (DKTK)（德国癌症联合会（DKTK））；EUropean Reference Network for RAre CANcers (EURACAN) Initiative（罕见癌症欧洲参考网络（EURACAN）计划）；University of Mainz Medical Center（法兰克福大学医学中心）；Google Cloud（谷歌云）；Institute of Artificial Intelligence in Medicine and Healthcare（医学与健康人工智能研究所）
