人类反馈强化学习（Reinforcement Learning from Human Feedback, RLHF）在大语言模型（large language models, LLMs）中应用广泛，而传统强化学习（Reinforcement Learning, RL）往往存在不足。当前的自动驾驶方法通常要么在机器学习（含RL）中引入人类反馈，要么借助LLMs，多数反馈用于引导车辆智能体的学习过程（例如控制车辆）。RLHF通常应用于微调阶段，需要直接的人类“偏好”，而这类偏好很少用于自动驾驶模型的优化。本研究创新性地结合RLHF与LLMs以提升自动驾驶安全性。从零开始用人类引导训练模型效率很低，我们的框架以预训练的自动驾驶车辆智能体模型为基础，部署多个人类控制的智能体（如车辆、行人）来模拟真实道路环境，自动驾驶车辆模型不受人类直接控制。我们整合物理与生理两类反馈对模型进行微调，并利用LLMs优化该过程。这种多智能体交互环境可确保模型在实际应用前实现安全、真实的交互。最后，我们将使用位于新泽西州和纽约市的真实测试平台收集的数据验证所提模型。
英文摘要：
Reinforcement Learning from Human Feedback (RLHF) is popular in large language models (LLMs), whereas traditional Reinforcement Learning (RL) often falls short. Current autonomous driving methods typically utilize either human feedback in machine learning, including RL, or LLMs. Most feedback guides the car agent's learning process (e.g., controlling the car). RLHF is usually applied in the fine-tuning step, requiring direct human "preferences," which are not commonly used in optimizing autonomous driving models. In this research, we innovatively combine RLHF and LLMs to enhance autonomous driving safety. Training a model with human guidance from scratch is inefficient. Our framework starts with a pre-trained autonomous car agent model and implements multiple human-controlled agents, such as cars and pedestrians, to simulate real-life road environments. The autonomous car model is not directly controlled by humans. We integrate both physical and physiological feedback to fine-tune the model, optimizing this process using LLMs. This multi-agent interactive environment ensures safe, realistic interactions before real-world application. Finally, we will validate our model using data gathered from real-life testbeds located in New Jersey and New York City.
Fighting Numerical Hallucinations via Data-centric Compilation for Online Financial QA
Shenzhen Technology University（深圳科技大学）；FiT, Tencent（腾讯金融科技部）；South China University of Technology（华南理工大学）；The Hong Kong University of Science and Technology（香港科学与技术大学）；Huazhong University of Science and Technology（华中科技大学）；McGill University（麦吉尔大学）
GSRM: Generative Speech Reward Model for Speech RLHF
Meta Superintelligence Labs（Meta超智能实验室）；Massachusetts Institute of Technology（麻省理工学院）；Arizona State University（亚利桑那州立大学）；University of Southern California（南加州大学）
Geometric-Guided Few-Shot Dental Landmark Detection with Human-Centric Foundation Model
Department of Electronic and Computer Engineering, The Hong Kong University of Science and Technology, Hong Kong SAR, China（电子与计算机工程系，香港科学与技术大学）；Division of Paediatric Dentistry and Orthodontics, Faculty of Dentistry, The University of Hong Kong, Hong Kong SAR, China（牙科学院儿童牙科与正畸学系，香港大学）
