Model-free and reinforcement learning-based adaptive filtering methods are gaining traction for denoising in dynamic, non-stationary environments such as wireless signal channels, biomedical monitoring, and sensor networks. Traditional filters such as LMS, RLS, Wiener, and Kalman are often limited by assumptions of stationarity, the need for exact noise statistics, or fragile parameter tuning. This paper proposes an adaptive filtering framework using Proximal Policy Optimization (PPO), guided by a composite reward that balances SNR improvement, MSE reduction, and residual smoothness. We frame adaptive filtering as a Markov decision process and train a PPO agent to adjust filter coefficients directly in response to changing noise. Experiments on synthetic nonstationary signals with diverse noise types show that the PPO agent generalizes beyond its training distribution. Moreover, real-world analysis is made and evaluated on ECG recordings from the MIT-BIH Noise Stress Test Database corrupted by baseline wander, electrode motion, and muscle artifacts. The learned PPO policy achieves real-time inference and slightly outperforms strong classical baselines on ECG denoising. These results demonstrate the viability of policy-gradient reinforcement learning as a computationally efficient and flexible tool for adaptive filtering in nonlinear, time-varying dynamical systems.
Segmental Advantage Estimation: Enhancing PPO for Long-Context LLM Training
LLM Department, Tencent（腾讯大语言模型部门）；University of Chinese Academy of Sciences（中国科学院大学）；Intelligent Software Research Center, Institute of Software, CAS（软件研究所智能软件研究中心）；State Key Lab of Processors, Institute of Computing Technology, CAS（计算技术研究所处理器国家重点实验室）
United States Air Force Research Laboratory（美国空军研究实验室）；United States Air Force Artificial Intelligence Accelerator（美国空军人工智能加速器）
Traditional frequency based projection filters, or projection operators (PO), separate signal and noise through a series of transformations which remove frequencies where noise is present. However, this technique relies on a priori knowledge of what frequencies contain signal and noise and that these frequencies do not overlap, which is difficult to achieve in practice. To address these issues, we introduce a PO-neural network hybrid model, the Pseudo Projection Operator (PPO), which leverages a neural network to perform frequency selection. We compare the filtering capabilities of a PPO, PO, …
We present the PowerGridworld software package to provide users with a lightweight, modular, and customizable framework for creating power-systems-focused, multi-agent Gym environments that readily integrate with existing training frameworks for reinforcement learning (RL). Although many frameworks exist for training multi-agent RL (MARL) policies, none can rapidly prototype and develop the environments themselves, especially in the context of heterogeneous (composite, multi-device) power systems where power flow solutions are required to define grid-level variables and costs. PowerGridworld i…
Reinforcement Learning for Synchronised Flow Control in a Dual-Gate Resin Infusion System
Instituto Universitario de Investigación en Tecnología Centrada en el Ser Humano, HUMAN-tech, Universitat Politècnica de València（人类中心技术大学研究院，HUMAN-tech，瓦伦西亚理工大学）；Department of Civil and Mechanical Engineering, Technical University of Denmark（土木与机械工程系，技术大学 of Denmark）；Artikode Intelligence S.L
EFRame: Deeper Reasoning via Exploration-Filter-Replay Reinforcement Learning Framework
Zhongguancun Academy（中关村学院）；College of Software, Nankai University（南开大学软件学院）；School of Computer Science, Shanghai Jiao Tong University（上海交通大学计算机科学学院）；Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院）；Department of Electronic Engineering, Tsinghua University（清华大学电子工程系）
Act Only When It Pays: Efficient Reinforcement Learning for LLM Reasoning via Selective Rollouts
Carnegie Mellon University（卡内基梅隆大学）；Lawrence Livermore National Laboratory（劳伦斯利弗莫尔国家实验室）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Meta AI
Voice activity detection (VAD) is an important pre-processing step for speech technology applications. The task consists of deriving segment boundaries of audio signals which contain voicing information. In recent years, it has been shown that voice source and vocal tract system information can be extracted using zero-frequency filtering (ZFF) without making any explicit model assumptions about the speech signal. This paper investigates the potential of zero-frequency filtering for jointly modeling voice source and vocal tract system information, and proposes two approaches for VAD. The first …
Joint radar-communication (JRC) waveform can be used for simultaneous radar detection and communication in the same frequency band. However, radar detection processing requires the prior knowledge of the waveform including the embedded information for matched filtering. To remove this requirement, we propose a unimodular JRC waveform based on composite modulation where the internal modulation embeds information by mapping the bit sequence to different orthogonal signals, and the external modulation performs phase modulation on the internal waveform to satisfy the demand of detection. By adjust…
The task of heart rate estimation using photoplethysmographic (PPG) signal is challenging due to the presence of various motion artifacts in the recorded signals. In this paper, a fast algorithm for heart rate estimation based on modified SPEctral subtraction scheme utilizing Composite Motion Artifacts Reference generation (SPECMAR) is proposed using two-channel PPG and three-axis accelerometer signals. First, the preliminary noise reduction is obtained by filtering unwanted frequency components from the recorded signals. Next, a composite motion artifacts reference generation method is develo…
