There has been a current trend in reinforcement learning for healthcare literature, where in order to prepare clinical datasets, researchers will carry forward the last results of the non-administered test known as the last-observation-carried-forward (LOCF) value to fill in gaps, assuming that it is still an accurate indicator of the patient's current state. These values are carried forward without maintaining information about exactly how these values were imputed, leading to ambiguity. Our approach models this problem using OpenAI Gym's Mountain Car and aims to address when to observe the patient's physiological state and partly how to intervene, as we have assumed we can only act after following an observation. So far, we have found that for a last-observation-carried-forward implementation of the state space, augmenting the state with counters for each state variable tracking the time since last observation was made, improves the predictive performance of an agent, supporting the notion of "informative missingness", and using a neural network based Dynamics Model to predict the most probable next state value of non-observed state variables instead of carrying forward the last observed value through LOCF further improves the agent's performance, leading to faster convergence and reduced variance.
On the Convergence of Reinforcement Learning with Monte Carlo Exploring Starts
University of Waterloo（滑铁卢大学）
一种基本的基于模拟的强化学习算法是蒙特卡洛探索状态（Monte Carlo Exploring States，MCES）方法，也称为乐观策略迭代，其中值函数由模拟返回近似，并且在每次迭代时选择贪婪策略。该算法在一般情况下的收敛性一直是一个悬而未决的问题。在本文中，我们研究了该算法的收敛性的情况下，与未贴现成本，也被称为随机最短路问题。这些结果补充了已有的部分结果，从而有助于进一步解决这一问题。作为一个附带的结果，我们还提供了一个版本的上鞅收敛定理通常用于随机逼近的证明。
Locally Pareto-Optimal Interpretations for Black-Box Machine Learning Models
University of California at Berkeley（加州大学伯克利分校）；Indian Institute of Technology Bombay（印度班加罗尔理工学院）；Chalmers University of Technology and University of Gothenburg（查尔姆斯理工大学和哥德堡大学）
Exploring Applications of State Space Models and Advanced Training Techniques in Sequential Recommendations: A Comparative Study on Efficiency and Performance
Exploring Winograd Convolution for Cost-effective Neural Network Fault Tolerance
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Beijing Institute of Control Engineering（北京控制工程研究所）；Institute of High Performance Computing, A*STAR（新加坡科技研究局高性能计算研究所）
Exploring the Optimal Choice for Generative Processes in Diffusion Models: Ordinary vs Stochastic Differential Equations
Institute of Natural Sciences & School of Mathematical Sciences, Shanghai Jiao Tong University（上海交通大学自然科学研究院与数学科学学院）；University of Science and Technology of China（中国科学技术大学）；Suzhou Institute of Advanced Research, University of Science and Technology of China（中国科学技术大学苏州高等研究院）；City University of Hong Kong（香港城市大学）
EventChat: Implementation and user-centric evaluation of a large language model-driven conversational recommender system for exploring leisure events in an SME context
Swiss Federal Institute of Technology, Zurich（苏黎世联邦理工学院）
Exploring the Utility of MALDI-TOF Mass Spectrometry and Antimicrobial Resistance in Hospital Outbreak Detection
Carnegie Mellon University（卡内基梅隆大学）；University of Pittsburgh（匹兹堡大学）；University of Pittsburgh School of Medicine（匹兹堡大学医学院）；School of Public Health, University of Pittsburgh（匹兹堡大学公共卫生学院）
