arXivDaily arXiv每日学术速递 周一至周五更新

高校专区

Stanford University(斯坦福大学)

至 收录 2269
2607.18106 2026-07-21 cs.RO 新提交

Importance Sampling and PCA for Finding Failures in Commercial Autonomous Vehicles

用于发现商用自动驾驶车辆故障的重要性采样和主成分分析

Hailey Warner, Duncan Eddy, Shreya Parjan, Caroline Cahilly, Harrison Delecki, Matthias Kleinstauber, Chaitanya Shinde, Jerry Lopez, Mykel J. Kochenderfer

机构 * Department of Aeronautics and Astronautics, Stanford University(斯坦福大学航空航天系) Torc Robotics(Torc机器人公司)

AI总结 研究商用自动驾驶车辆故障发现问题,应用自适应压力测试和基于扩散的故障采样算法找到模拟碰撞,引入基于主成分分析的统计分析将故障分类并恢复噪声轨迹,为故障发现到系统诊断提供路径。

Comments IEEE ICVES 2026 (Submitted)

详情
AI中文摘要

目前,用于发现自主系统中罕见故障的方法几乎仅在具有简单学术驾驶堆栈的模拟中得到验证,尚不清楚它们是否能推广到商用系统中更强大的规划器。我们通过将两种罕见事件发现算法应用于商用自动驾驶卡车堆栈来填补这一空白。自适应压力测试(AST)使用强化学习搜索最可能导致模拟碰撞的噪声轨迹,而基于扩散的故障采样(DiFS)训练去噪扩散模型以采样各种故障。我们表明这两种算法都能发现传统蒙特卡罗模拟无法发现的合并和切入操作中的模拟碰撞。为使这些故障具有可操作性,我们引入基于主成分分析(PCA)的统计分析,将故障分类为常见模式并识别对结果影响最大的时间步长。我们对主成分进行聚类并反转PCA变换以恢复广义噪声轨迹,并表明这些轨迹在相同和相似场景中重现故障。这提供了一条从故障发现到感知级缺陷系统诊断的路径。

英文摘要

Methods for discovering rare failures in autonomous systems have so far been demonstrated almost exclusively in simulations with simple, academic driving stacks, leaving open whether they generalize to the more robust planners used in commercial systems. We address this gap by applying two rare-event discovery algorithms to a commercial autonomous trucking stack. Adaptive stress testing (AST) uses reinforcement learning to search for the most likely noise trajectories leading to a simulated collision, while diffusion-based failure sampling (DiFS) trains a denoising diffusion model to sample a diverse set of failures. We show that both algorithms find simulated collisions during merge and cut-in maneuvers where traditional Monte Carlo simulation does not. To make these failures actionable, we introduce a statistical analysis based on principal component analysis (PCA) that classifies failures into common modes and identifies the timesteps that most influence the outcome. We cluster the principal components and invert the PCA transform to recover generalized noise trajectories, and show that these trajectories reproduce failures in identical and similar scenarios. This provides a path from failure discovery to systematic diagnosis of perception-level flaws.

URL PDF HTML 收藏
2607.17914 2026-07-21 cs.MA cs.LG cs.RO 新提交

Value-Aware Prediction for Robust Multi-Agent Coordination Under Communication Loss

通信丢失情况下用于鲁棒多智能体协调的价值感知预测

Kemal Devrim Kafadar, Eren Özaltun, Mahmud Efnan Şanlı, Feyza Orak, Emirhan Gazi, Kubilay Kağan Kömürcü, Nazım Kemal Üre

机构 * Istanbul Technical University(伊斯坦布尔技术大学) University of Stuttgart(斯图加特大学) Istanbul Technical University Artificial Intelligence and Data Science Application and Research Center(伊斯坦布尔技术大学人工智能与数据科学应用与研究中心) Stanford University(斯坦福大学)

AI总结 研究通信丢失时多智能体协调问题,提出价值感知MARO方法,利用优势估计动态加权预测器损失函数,在多智能体粒子环境实验中,该方法在通信可靠性下降时能维持性能,提升平均回报并降低性能方差。

Comments Accepted to 2026 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

详情
AI中文摘要

鲁棒多智能体协调严重依赖智能体间通信,而在实际部署中常受物理和环境约束干扰。为在通信间歇性故障时维持运行,智能体可采用内部预测模型估计缺失的共享状态信息。但用标准重建目标训练的预测器对所有转移一视同仁。本文提出通信丢包下多智能体观察共享(MARO)的价值感知扩展方法,即价值感知MARO。通过利用底层演员-评论家架构的优势估计动态加权预测器的损失函数,使预测器学习过程与策略演变明确耦合。在多智能体粒子环境的多个任务上评估该框架,实验结果表明在通信可靠性下降时,尤其是低于40%时,该方法能维持性能。在高损耗场景下,与标准未加权基线相比,平均回报提高超20%,性能方差平均降低64.7%。

英文摘要

Robust multi-agent coordination relies heavily on inter-agent communication, which is frequently disrupted by physical and environmental constraints in real-world deployments. To maintain operation during these intermittent communication failures, agents can employ internal prediction models to estimate missing shared state information. However, predictors trained with standard reconstruction objectives treat all transitions equally. In a Reinforcement Learning context, this forces the model to waste capacity learning stochastic exploration noise and the outdated dynamics of suboptimal policies. In this paper, we propose a value-aware extension of Multi-Agent Observation Sharing under Communication Dropout (MARO) to patch communication gaps; we refer to this method as Value-Aware MARO. By dynamically weighting the predictor's loss function using advantage estimates derived from the underlying actor-critic architecture, our objective explicitly couples the predictor's learning process to the policy's evolution. This formulation focuses the model's capacity on the intentional, high-return dynamics actively reinforced by the agents. We evaluate our framework on several tasks within the Multi-Agent Particle Environment under varying communication reliability levels. Experimental results demonstrate that our approach maintains performance under declining communication reliability, particularly below 40%. While our method performs comparably in tasks where the baseline already maintains high coordination, our value-aware weighting effectively prevents the performance collapse observed in the standard predictor during high-attrition scenarios. In these environments, our method achieves an average improvement in mean returns of more than 20% and reduces performance variance by a mean of 64.7% compared to the standard unweighted baseline.

URL PDF HTML 收藏
2607.17585 2026-07-21 cs.CV 新提交

Pixel-Space Diffusion Transformers

像素空间扩散变换器

Renye Yan, Jikang Cheng, You Wu, Ling Liang, Wei Peng, Athanasios V. Vasilakos, Qingyu Zhao, Yu Zhang, Ehsan Adeli, Kilian M. Pohl, Guoying Zhao

机构 * Peking University(北京大学) Nanjing University(南京大学) Stanford University(斯坦福大学) Cornell University(康奈尔大学) University of Oulu(奥卢大学)

AI总结 本文探讨像素空间扩散变换器,针对潜在扩散模型的局限,研究直接对原始像素建模的像素空间扩散方法,介绍其在高维建模中的挑战与多模态建模优势,从多方面回顾pDiTs,总结方法、识别挑战并展望未来方向。

详情
AI中文摘要

潜在扩散模型(LDMs)通过在VAE压缩的潜在空间中去噪来实现高效的高分辨率图像合成。然而,固定视觉tokenizer会丢弃精细纹理和结构细节,且单独的表示和扩散训练会导致重建与生成目标不匹配。这些限制使人们重新关注像素空间扩散,它直接对原始像素建模,消除VAE瓶颈并支持端到端优化。这虽更符合高保真生成需求,但在高维建模中带来挑战。像素空间建模也为统一多模态系统提供了基础。本文从模型架构、连续生成机制和统一多模态建模角度回顾像素空间扩散变换器(pDiTs),总结方法、识别挑战并探讨未来方向。

英文摘要

Latent diffusion models (LDMs) enable efficient high-resolution image synthesis by denoising in a VAE-compressed latent space. However, fixed visual tokenizers can discard fine textures and structural details, while separate representation and diffusion training creates a mismatch between reconstruction and generation objectives. These limitations have renewed interest in pixel-space diffusion, which models raw pixels directly, removes the VAE bottleneck, and supports end-to-end optimization. This formulation better matches the demands of high-fidelity generation but introduces challenges in high-dimensional modeling, including noise scheduling, loss weighting, token efficiency, and scalable architecture design. Pixel-space modeling also offers a promising basis for unified multimodal systems: raw pixels, text, and task conditions can be represented in a shared token space and jointly processed by a single Transformer, narrowing the gap between visual understanding and generation. This paper reviews Pixel-Space Diffusion Transformers (pDiTs) from the perspectives of model architecture, continuous generative mechanisms, and unified multimodal modeling. We summarize representative methods, identify key technical challenges, and discuss future directions toward high-fidelity, end-to-end vision foundation models that integrate generation and understanding.

URL PDF HTML 收藏
2607.17551 2026-07-21 cs.CV cs.AI 新提交

Hierarchy-Aware and Anatomy-Guided Learning for Lung Ultrasound Video Classification

用于肺部超声视频分类的层次感知与解剖引导学习

Alya Almsouti, Lotfi Mecharbat, Noha Aboukhater, Yousef Alabrach, Siddiq Anwar, Andre Kumar, Ibrahim Almakky, Mohammad Yaqub

机构 * Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学) SEHA(赛哈) Sheikh Khalifa Medical City(谢赫·哈利法医疗城) Khalifa University(哈里发大学) Stanford University(斯坦福大学)

AI总结 研究针对肺部超声视频分类的挑战,提出深度学习框架,采用层次感知训练与解剖引导学习,结合临床结构化目标与解剖监督,经实验验证该方法能提升病理分离、实现高宏F1,且具迁移适应性,是稳健可解释的视频分析实用方法。

详情
AI中文摘要

肺部超声(LUS)是评估因心力衰竭或肾功能受损而有风险患者肺水肿的床边工具。然而,由于斑点噪声、成像伪影和依赖操作员的采集变异性,自动LUS分析仍然具有挑战性。在这项工作中,我们提出了一个用于多类LUS视频分类的深度学习框架,该框架探索了两个组成部分:层次感知训练和解剖引导学习。从一个强大的基线开始,我们引入分层训练策略,然后引入胸膜线掩码监督,以引导模型关注与解剖相关的区域。我们使用来自219名患者的1886个视频的开放获取数据集研究四个临床相关类别——健康、B线、实变和伴有实变的混合B线,并通过患者级五折交叉验证进行评估。结果表明,层次感知训练相对于平面分类改善了病理分离,而掩码引导的注意力监督实现了65.7%的最高平均宏F1,并产生了更局部化的注意力模式。在外部COVID-BLUeS数据集上的迁移实验进一步显示了具有竞争力和参数效率的适应性,同时保留了以胸膜为重点的注意力行为。这些发现表明,将临床结构化目标与解剖引导监督相结合是一种用于稳健、可解释的LUS视频分析的实用方法。代码和模型实现可在该https URL上获得。

英文摘要

Lung ultrasound (LUS) is a bedside tool for assessing pulmonary edema in patients at risk due to heart failure or impaired kidney function. However, automated LUS analysis remains challenging because of speckle noise, imaging artifacts, and operator-dependent acquisition variability. In this work, we present a deep learning framework for multi-class LUS video classification that explores two components: hierarchy-aware training, and anatomy-guided learning. Starting from a strong baseline, we introduce hierarchical training strategies and then introduce pleural line mask supervision to guide model attention toward anatomically relevant regions. We study four clinically relevant classes--healthy, B-lines, consolidations, and mixed B-lines with consolidations--using an open-access dataset of 1,886 videos from 219 patients, evaluated with patient-level five-fold cross-validation. Results show that hierarchy-aware training improves pathological separation relative to flat classification, while mask-guided attention supervision achieves the highest mean macro-F1 of 65.7\% and produces more localized attention patterns. Transfer experiments on the external COVID-BLUeS dataset further show competitive and parameter-efficient adaptation while preserving pleural-focused attention behavior. These findings suggest that combining clinically structured objectives with anatomy-guided supervision is a practical approach to robust, interpretable LUS video analysis. Code and model implementations are available at https://github.com/Alya-Almsouti/LUS-video-classification.

URL PDF HTML 收藏
2607.17290 2026-07-21 cs.LG cs.AI cs.LO 新提交

Lookahead Branching for Neural Network Verification

神经网络验证中的前瞻分支

Liam Davis, Duo Zhou, Huan Zhang, Guy Katz, Clark Barrett, Haoze Wu

机构 * Amherst College(阿默斯特学院) University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校) Hebrew University of Jerusalem(耶路撒冷希伯来大学) Stanford University(斯坦福大学)

AI总结 研究神经网络验证中前瞻分支策略,提出通用集成方法,以FSB为例说明,还阐述其能生成加速验证的引理,通过在两个验证器中实例化,实现验证时间加速及解决实例数增加。

Comments Accepted to IJCAI 2026. Lookahead branching is part of the Marabou and $α$-$β$-CROWN verifiers

详情
AI中文摘要

在这项工作中,我们研究了前瞻分支策略在神经网络验证中的作用。我们提出了一种将前瞻集成到任何分支定界验证器中的通用方法,并展示了当前最先进的分支启发式算法之一FSB如何可被视为前瞻分支策略的一种特殊实例。我们还描述了除提高分支决策质量外,前瞻如何能生成加速验证的附加引理。我们在两个基于分支定界的代表性验证器(Marabou和α-β-CROWN)中实例化该方法,证明前瞻能使验证时间持续加速,且解决实例数量最多增加57%。代码可在指定网址获取。

英文摘要

In this work, we investigate the effect of lookahead branching strategies in neural network verification. We present a general recipe to integrate lookahead into any branch-and-bound verifier and demonstrate how one of the current state-of-the-art branching heuristics, FSB, can be viewed as a special instantiation of the lookahead branching strategy. We also describe how, in addition to improving the quality of branching decisions, lookahead can generate additional lemmas that accelerate verification. We instantiate the method in two representative branch-and-bound-based verifiers (Marabou and $α$-$β$-CROWN), and demonstrate that lookahead leads to consistent speedups in verification time and up to $57\%$ more solved instances. Code is available at https://github.com/ai-ar-research/lookahead-branching.

URL PDF HTML 收藏
2607.17213 2026-07-21 cs.RO 新提交

Retriever: Composing Closed-Loop Asynchronous Robot Programs

Retriever:组合闭环异步机器人程序

Linfeng Zhao, Haojie Huang, Jiayuan Mao, Weiyu Liu, Mykel Kochenderfer, Lawson L. S. Wong

机构 * Stanford University(斯坦福大学) MIT(麻省理工学院) Northeastern University(东北大学)

AI总结 研究构建长期运行机器人智能体的闭环管道问题,提出Retriever,它涵盖异步决策模型等整个堆栈,将智能体表示为有状态因果流函数图,编译到支持多后端的运行时,可系统调试和确定性重放,通过案例研究等进行评估。

Comments Project website: http://retriever.systems; Package open-source website: http://openretriever.org

详情
AI中文摘要

构建长期运行的机器人智能体需要组合闭环管道,其组件运行在不同时钟且延迟可变。当前系统常采用临时并发和发布/订阅约定,导致时间和输入消费语义隐含,行为依赖调度且难重现、调试和复用。现有解决方案多只解决部分问题。本文提出Retriever,涵盖整个堆栈,包括异步决策模型、编程模型、运行时和示例闭环智能体管道。它将智能体表示为在显式运行时钟上执行的有状态因果流函数图,通过连续时间流上的异步环境-智能体循环形式化此观点,表明有限内存因果策略可由这些算子组合表示。Retriever将这些图编译到支持多个后端的运行时,实现跨运行环境的系统调试和从记录的异步数据进行确定性重放。我们通过实际机器人案例研究以及对运行时开销和确定性重放行为的控制研究对Retriever进行评估。

英文摘要

Building long-horizon robot agents requires composing closed-loop pipelines -- perception, belief update, planning, and control -- whose components run at different clocks and with variable latency. Today, these systems are often assembled with ad-hoc concurrency and pub/sub conventions that make timing and input-consumption semantics implicit, yielding schedule-dependent behavior that is hard to reproduce, debug, and reuse. Current solutions typically solve parts of this problem at either the algorithmic or the systems layer, but not both. In this work, we propose Retriever, which spans the entire stack: an asynchronous decision model, a programming model, a runtime, and an example closed-loop agent pipeline. Retriever represents an agent as a graph of stateful causal stream functions executed on explicit run clocks. We formalize this view via an asynchronous environment-agent loop over continuous-time streams and show that finite-memory causal policies can be represented by compositions of these operators. Retriever compiles these graphs into a runtime that supports multiple backends, enabling systematic debugging across running environments and deterministic replay from logged asynchronous data. We evaluate Retriever through a real-robot case study together with controlled studies of runtime overhead and deterministic replay behavior.

URL PDF HTML 收藏
2607.16922 2026-07-21 cs.CV 新提交

Pedestrian Archetypes Extension -- More Pedestrian Models for Autonomous Vehicle Safety Testing

行人原型扩展——用于自动驾驶车辆安全测试的更多行人模型

Taorui Huang, Namita Gaidhani, Ritvik Bansal, S M Jubaer, Regina Lim, Rhett Zhao, Gavin Rafael Selin, Sunnie Deng Gao, Hasnain N Syed

机构 * Stanford University(斯坦福大学) University of California San Diego(加利福尼亚大学圣地亚哥分校) University of Washington(华盛顿大学)

AI总结 研究在行人原型基础上,通过注释YouTube行车记录仪视频,识别出7种新的行人原型,介绍其行为,阐述与旧原型差异并提供视频证据,为自动驾驶车辆安全测试提供更多行人模型。

Comments Extended version of Pedestrian Archetypes paper (published in IEEE IV 2025)

详情
AI中文摘要

在我们之前的工作《行人原型》中,我们将行人原型定义为唯一标识特定类型行人的行为集合。第一篇论文提出了12种行人原型,包括漫步者、醉酒者、分心者、闪行者、优柔寡断者、盲人、群体、乱穿马路者、老年人、儿童、多变者和停车行人。引入这些原型是为了超越单一行为标签,提供一种更自然的方式来描述危险行人在现实交通场景中实际逐步的行为方式。然而,在对YouTube行车记录仪视频进行进一步注释后,我们识别出7种额外的行人原型,它们与先前提出的原型存在明显的行为差异。这些新原型捕捉到了原始分类法无法完全解释的行人行为模式。在本预印本中,我们介绍每种新原型,定义其基本和可选行为,解释其与先前提出的原型的不同之处,并提供显示该原型实际应用的视频帧证据。

英文摘要

In our prior work, Pedestrian Archetypes, we defined pedestrian archetypes as collections of behaviors that uniquely identify a specific type of pedestrian. The first paper proposed 12 pedestrian archetypes, including the Wanderer, Drunk, Distracted, Flash, Indecisive, Blind, Flock, Jaywalker, Elderly, Kid, Eventful, and Parked Pedestrian. These archetypes were introduced to move beyond single behavior labels and provide a more natural way to describe how dangerous pedestrians actually behave progressively in real-world traffic scenarios. However, upon further annotation of YouTube dash-cam videos, we identified 7 additional pedestrian archetypes with observable and significant behavioral differences from the previously proposed ones. These new archetypes capture pedestrian behavior patterns that could not be fully explained by the original taxonomy. In this pre-print, we introduce each new archetype, define its essential and optional behaviors, explain how it differs from previously proposed archetypes, and provide video-frame evidence showing the archetype in action.

URL PDF HTML 收藏
2607.16412 2026-07-21 cs.AI 新提交

Interactive Task Alignment as a POMDP

作为部分可观测马尔可夫决策过程的交互式任务对齐

Andy Dai, Zexue He, Zhenyu Zhang, Alex Pentland, Jiaxin Pei

机构 * Stanford University(斯坦福大学)

AI总结 研究语言模型在面对模糊用户任务时的任务对齐问题,引入基于部分可观测马尔可夫决策过程的框架,通过用户研究验证,发现模型任务对齐困难,训练能改善但仍落后于人类,揭示其缺乏关键交互能力。

详情
AI中文摘要

当前语言模型基准主要评估在完全指定任务上的执行情况。然而,实际用户任务往往模糊不清,用户目标不完整、具探索性甚至不一致,需要助手先确定预期任务再执行。我们将此问题作为任务对齐来研究,引入一个将指定任务转换为未完全指定交互的通用框架,形式化为部分可观测马尔可夫决策过程,模型须从部分且不断演变的用户意图中推断潜在任务。通过用户研究事后验证用户模拟器。结果表明,虽任务指定后模型表现良好,但在任务对齐方面仍有困难,平均仅22 - 32%能恢复用户意图任务,人类在相同设置下可达48%。训练后能改善任务对齐,但模型在通过交互解决不确定性方面仍落后于人类。

英文摘要

Current benchmarks for language models primarily evaluate execution on fully specified tasks. However, real user tasks are often ambiguous. Users arrive with incomplete, exploratory, or even inconsistent goals, requiring the assistant to first determine the intended task before carrying it out. We study this problem as task alignment: the ability to align with a user on their intended task. We introduce a general framework for converting specified tasks into underspecified interactions, formalized as a POMDP in which the model must infer a latent task from partial and evolving user intent. We validate our user simulator post hoc with a human user study. Across shopping, coding, and professional work settings, we find that while models often perform well once the task is specified, models still struggle with task alignment: current models act prematurely, interact ineffectively, and fail to resolve ambiguous requests. Models on average recover the user's intended task only 22-32% of the time under ambiguity. In a human study in the same setting, humans reach 48%, outperforming all evaluated models. We show that post-training with supervised fine-tuning and reinforcement learning improves task alignment, but models still lag behind humans in resolving uncertainty through interaction. Together, our results suggest that current models still lack key interaction abilities required for reliable agency.

URL PDF HTML 收藏
2607.16256 2026-07-21 cs.LG cs.AI cs.IR cs.NE 新提交

Discovery by Dreaming: Cross-Domain Recombination in Artificial Memory

通过做梦进行发现:人工记忆中的跨域重组

Oliver Zahn, James Evans, David Eagleman

机构 * University of Chicago(芝加哥大学) Santa Fe Institute(圣塔菲研究所) Stanford University(斯坦福大学)

AI总结 研究探讨人工记忆中跨域重组,通过在两个系统实现重组重播机制,发现跨域巩固创造价值,域内排练不然,符号系统跨域连接提升显著,神经系统在特定子任务有提高,验证了相关预测,表明巩固是为发现而非记忆。

Comments 38 pages, 13 figures, 7 tables

详情
AI中文摘要

梦将从未相遇的人、地点和时间拼接在一起。神经科学表明这种重组并非噪音,而是驱动洞察力和创造性发现的一种功能。这重新定义了记忆巩固:其可衡量的价值在于跨尚未同时发生的经历重组知识,而非仅仅防止遗忘。我们通过分离重组重播机制并在两个架构不相关的系统中实现它来直接测试这一点:一个LoRA微调管道(DREAMS)和一个重播结构化知识对象的符号引擎(SAPIENCE)。两个系统都得出相同结论:跨域巩固创造价值,而域内排练则不然。符号系统有85.7%的新跨域连接,比基线提高了21个百分点。神经系统整体提高了5.64个百分点,在明确需要跨域转移的子任务上提高了14.5个百分点。这种效应是权重的真实属性,而非提示工件。我们根据50000篇真实论文中的记录发现验证了这一预测,并陈述了一个可证伪的海马体记录预测以区分重组和排练。最终,这一原则是底物通用的,能大规模追踪真实发现。阅读文献教会模型回忆所见内容,但产生发现需要一个单独的离线阶段来跨域重组知识——做梦的计算模拟。巩固不是为了记忆,而是为了发现。

英文摘要

Dreams splice together people, places, and times that never met. Neuroscience suggests this recombination is not noise, but a function driving insight and creative discovery. This reframes memory consolidation: rather than merely defending against forgetting, its measurable value lies in recombining knowledge across experiences that have not yet co-occurred. We test this directly by isolating the recombinatory-replay mechanism and implementing it in two architecturally unrelated systems: a LoRA fine-tuning pipeline (DREAMS) and a symbolic engine replaying structured knowledge objects (SAPIENCE). Both systems converge on the same finding: cross-domain consolidation creates value, while within-domain rehearsal does not. The symbolic arm surfaces novel cross-domain connections at 85.7%, a +21 percentage point (pp) gain over baseline. The neural arm improves overall by +5.64 pp, but on subtasks explicitly requiring cross-domain transfer (like unseen math reasoning on GSM8K), gains reach +14.5 pp. This effect is a genuine property of the weights--not a prompt artifact--as prepending the same material in-context to a 671B-parameter model actually reverses the gain. We validate this prediction against documented discoveries across 50,000 real papers and state a falsifiable hippocampal-recording prediction to distinguish recombination from rehearsal. Ultimately, this principle is substrate-general, tracking real discovery at scale. Reading the literature teaches a model to recall what it has seen, but producing discovery requires a separate offline phase that recombines knowledge across domains--the computational analog of dreaming. Consolidation is not for remembering, but for discovering.

URL PDF HTML 收藏
2607.16241 2026-07-21 cs.LG cs.AI 新提交

KernelBench-Verified: Do LLM-Generated Kernels Actually Beat PyTorch?

KernelBench验证:大语言模型生成的内核真的能超越PyTorch吗?

Yunxiang Zhang, Ping Yu, Jianyu Wang, Max, Fan, Julian Reed, Azalia Mirhoseini, Will Su

机构 * Meta(元(公司)) FAIR at Meta SuperIntelligence Lab(元超智能实验室公平团队) Stanford University(斯坦福大学)

AI总结 研究大语言模型生成的内核是否真能超越PyTorch,指出前沿模型存在奖励黑客行为。引入KernelBench-Verified扩展评估框架及内存效率指标,实验发现最佳模型加速比低,无模型始终超PyTorch,部分模型增加GPU内存峰值使用,强调持续调整评估协议的必要性。

详情
AI中文摘要

近期的大语言模型能生成自定义CUDA内核,在如KernelBench等基准测试中看似超越PyTorch。但前沿模型常通过奖励黑客行为人为提高报告性能。本文指出评估框架需与模型能力共同发展。一方面要准确测量真实加速比,考虑TF32启用的基线计时机制;另一方面关注算法正确性,模型常硬编码特定张量值的绕过方式。为此引入KernelBench-Verified扩展评估框架及内存效率指标。实验发现,在验证单轮评估中,最佳模型的几何平均加速比远低于标准评估协议下的结果,且无模型始终优于PyTorch,部分模型还增加了GPU内存峰值使用。研究表明随着大语言模型内核生成能力提升,持续调整稳健评估协议很有必要。

英文摘要

Recent large language models (LLMs) can generate custom CUDA kernels that appear to outperform PyTorch on benchmarks such as KernelBench. Building upon this foundational framework, we demonstrate that frontier models frequently engage in reward hacking to artificially inflate reported performance. In this work, we identify two areas where evaluation frameworks must co-evolve with model capabilities. First, to accurately measure true speedup, we examine the baseline timing mechanism, noting that enabling Tensor Core acceleration with TF32 provides a more realistic estimation of execution on modern GPUs. Second, concerning algorithmic correctness, models often exploit the narrow test distribution by hardcoding bypasses for specific tensor values. By skipping required computations, these kernels artificially accelerate execution rather than implementing actual CUDA kernels. We introduce KernelBench-Verified, an extended evaluation framework that incorporates a TF32-enabled baseline and a four-distribution hidden test suite. We additionally introduce memory efficiency metrics that capture the often-overlooked speed-memory tradeoff in kernel optimization. Under verified single-turn evaluation with seven frontier LLMs, we find that the best-performing model (GPT-5.5) achieves a 0.88x geometric mean speedup, significantly lower than the 1.43x speedup observed under the standard evaluation protocol. No model consistently outperforms PyTorch when evaluated against realistic baselines. On the memory front, 28% of GPU kernels generated by the best model increase peak GPU memory usage. Our findings demonstrate the necessity of continually adapting robust evaluation protocols as LLM kernel generation capabilities advance.

URL PDF HTML 收藏
2607.17595 2026-07-21 cs.LG cs.SY eess.SY math.OC 新提交

Concentration and Mean-Square Bounds for Contractive Stochastic Approximation: A Unified Elementary Approach

压缩随机逼近的集中性和均方界:一种统一的基本方法

Siddharth Chandak

机构 * Department of Electrical Engineering, Stanford University(电气工程系,斯坦福大学)

AI总结 研究乘性噪声模型下任意范数压缩映射的随机逼近,提出统一基本分析方法,通过平均噪声序列等得到范数误差的李雅普诺夫漂移不等式,进而得出均方和集中性界,还讨论了证明技术的可推广性。

Comments Submitted to Stochastic Processes and their Applications

详情
AI中文摘要

我们在乘性噪声模型下,为具有任意范数压缩映射的随机逼近(SA)建立了均方和集中性界。该噪声模型中噪声可能与迭代的范数仿射缩放,且迭代可能无界。这些设置出现在强化学习中。早期工作通过广义莫罗包络构造光滑李雅普诺夫函数来处理任意范数,通过多阶段自举论证处理乘性噪声和无界迭代。我们提出一种统一的基本分析方法来得到这两个界。利用平均噪声序列和相应辅助迭代,直接得到范数误差的一步李雅普诺夫漂移不等式。对于均方界,将漂移不等式与归纳论证结合表明迭代在期望上保持有界。对于集中性界,在一系列控制迭代的“好”事件上进行概率归纳,从而应用标准的阿祖马 - 霍夫丁界。我们的方法通过允许步长对数依赖于置信水平,得到了乘性噪声下SA的首个次高斯尾极大(全程)集中性界。此外,我们还讨论了这些证明技术对其他噪声模型和迭代算法的可推广性。

英文摘要

We establish mean-square and concentration bounds for stochastic approximation (SA) with arbitrary norm contractive mappings, under a multiplicative noise model where the noise may scale affinely with the norm of the iterates, and the iterates are potentially unbounded. These settings arise in reinforcement learning, where operators are often contractive in the $\ell_\infty$ norm and the noise scales with the iterates. To address the arbitrary norm, earlier works replace the non-smooth squared norm with a smooth Lyapunov function constructed via the generalized Moreau envelope. For concentration analysis, these works handle multiplicative noise and unbounded iterates through a multi-stage bootstrapping argument that starts from a time-varying worst-case bound and iteratively refines it. We instead present a unified and elementary analysis that yields both bounds. Using an averaged noise sequence and corresponding auxiliary iterates, we obtain a one-step Lyapunov drift inequality for the normed error directly, without smoothing the norm or constructing an envelope. For the mean-square bound, we combine this drift inequality with an induction argument showing that the iterates remain bounded in expectation. For the concentration bound, we develop a probabilistic induction over a sequence of "good" events on which the iterates are controlled, allowing the standard Azuma-Hoeffding bound to be applied. Our approach yields the first sub-Gaussian tailed maximal (all-time) concentration bound for SA under multiplicative noise, by allowing the stepsize to depend logarithmically on the confidence level. Beyond the specific setting considered here, we discuss the generalizability of these proof techniques to other noise models and iterative algorithms.

URL PDF HTML 收藏
2607.16570 2026-07-21 cond-mat.mtrl-sci cs.LG 新提交

Mapping Order in Semicrystalline Polymers using Machine Learning of Nanobeam Electron Diffraction

利用纳米束电子衍射的机器学习绘制半结晶聚合物中的取向

Nicholas Marchese, Arthur R. C. McCray, Yael Tsarfati, Karen Bustillo, Adam Marks, Alberto Salleo, Colin Ophus

机构 * Department of Materials Science and Engineering, Stanford University(材料科学与工程系,斯坦福大学) SLAC National Accelerator Laboratory(SLAC国家加速器实验室) National Center for Electron Microscopy, Molecular Foundry, Lawrence Berkeley National Laboratory(电子显微镜国家中心、分子发现所、伯克利国家实验室)

AI总结 研究利用机器学习模型从合成数据中检测聚合物衍射峰及其强度,相比传统相关峰检测算法,该模型速度更快且性能更优,为4DSTEM实验近实时可视化开辟可能,有助于理解有机混合离子电子导体结构与性能关系。

Comments Submitted to ACS Macromolecules. Main text is 13 pages and 5 figures, 23 pages and 15 figures with supporting information

详情
AI中文摘要

有机混合离子电子导体(OMIECs)是一类很有前途的聚合物材料,可用于从神经形态计算到节能电子学和生物电子学等各种应用。尽管其具有高度可调性,但结构特征与诸如电荷载流子迁移率等关键性能特性之间的关系却知之甚少。透射电子显微镜(TEM)中的扫描纳米衍射是阐明这种结构-性能关系的有力探针,但会产生大量嘈杂的数据集,难以解释,因为聚合物反射呈现出几种不同的形态。为了解决这种复杂性,我们训练了一个机器学习(ML)模型,从合成数据中检测这些聚合物衍射峰及其强度。与分析纳米束4D扫描透射电子显微镜(4DSTEM)数据的传统相关峰检测算法相比,我们表明ML模型速度明显更快,并且在几乎所有情况下都优于相关算法,为4DSTEM实验的近实时可视化开辟了可能性。

英文摘要

Organic mixed ionic electronic conductors (OMIECs) are a promising class of polymer materials for applications spanning neuromorphic computation to energy efficient electronics and bioelectronics. Despite being highly tunable, the relationship between structural features and key performance properties such as charge carrier mobility is poorly understood. Scanning nanodiffraction in the transmission electron microscope (TEM) is a powerful probe for elucidating this structure-property relationship, but produces large, noisy datasets that are difficult to interpret because polymer reflections exhibit several distinct morphologies. To address the complexity, we trained a machine learning (ML) model to detect these polymer diffraction peaks and their intensities from synthetic data. Compared to correlative peak detection algorithms, the conventional method for analyzing nanobeam 4D scanning transmission electron microscopy (4DSTEM) data, we show that the ML model is significantly faster and outperforms correlative algorithms in almost all cases, opening up the possibility of near-live visualization of 4DSTEM experiments.

URL PDF HTML 收藏
2607.08986 2026-07-21 cs.AI cs.LO math-ph math.AP math.MP 版本更新

A Formalization of the Mean-Field Derivation of the Vlasov Equation

弗拉索夫方程平均场推导的形式化:作为策略游戏的人工智能辅助精益形式化

Joseph K. Miller

机构 * Stanford University(斯坦福大学) Massachusetts Institute of Technology(麻省理工学院)

AI总结 该研究以数学家指导AI在Lean 4中形式化研究成果为案例,将其构建为形式化游戏。通过此方式对非线性弗拉索夫方程适定性完整形式化,展示了开发过程及成果,还介绍了最优传输机制的分离情况及开发时间等,为形式化研究提供了新方法。

Comments 26 pages, 4 figures. Lean 4 development, blueprint site, and agent logs: https://github.com/Hydrodynamical/Vlasov_Meanfield_Formalization

详情
AI中文摘要

我们通过让数学家指导人工智能系统,在Lean 4证明助手里将一项研究成果形式化,并将此活动构建为一个形式化游戏。目标是把一个LaTeX文档转化为Lean。当开发代码能编译、无“抱歉”语句且机器检查表明目标定理仅基于Lean的基础公理时游戏获胜。重用是第二项检查,通过我们引入的定义判断:开发成果能否产生一个更广泛库可吸收的自包含通用数学层。案例研究是通过多布鲁申平均场路径对非线性弗拉索夫方程适定性进行完整、无公理的形式化,包括存在性、唯一性、稳定性估计和平均场极限,以及短窗口叠加原理(弱解是拉格朗日的)。人类负责指导而非编写证明,人工智能执行。形式化证明了每个书面陈述;书面陈述是否为预期定理仍由数学家判断。构建过程中出现的最优传输机制(特别是瓦瑟斯坦-1度量的性质和康托罗维奇-鲁宾斯坦对偶定理)分离成一个仅针对Mathlib编译的自包含层:约占开发内容六分之一(299个声明中的49个),位于一个无反向依赖的22个声明接口之后。主要定理运行约一周,完整开发约一个月。我们将定量结果作为一场游戏的观察报告,而非普遍规律。游戏规则未指定特定系统,所以方法框架旨在比任何一次运行的工具更持久。

英文摘要

We formalize a research result in the Lean 4 proof assistant by having a mathematician direct an AI system, and frame the activity as a formalization game. The objective is to turn a LaTeX document into Lean. The game is won when the development compiles, contains no sorry, and a machine check shows the target theorems rest on Lean's foundational axioms alone. Reuse is a second check, by a definition we introduce: whether the development yields a self-contained layer of general mathematics the wider library could absorb. The case study is a complete, axiom-clean formalization of well-posedness for the nonlinear Vlasov equation via Dobrushin's mean-field route -- existence, uniqueness, the stability estimate and mean-field limit, and a short-window superposition principle (weak solutions are Lagrangian). The human's role was to direct, not to write proofs: to scope the definitions, steer the decompositions, and triage the library's gaps; the AI agent executed. The formalization certifies the proof of each statement as written; whether the written statement is the intended theorem stays the mathematician's judgment. The optimal-transport machinery that fell out of the build (in particular, properties of the Wasserstein-1 metric and the Kantorovich-Rubinstein duality theorem) separates into a self-contained layer that compiles against Mathlib alone: about a sixth of the development (49 of 299 declarations), behind a 22-declaration interface with no reverse dependency. The headline theorems ran in about a week, the full development in about a month. We report the quantitative claims as observations of one game, not as general laws. The game's rules name no particular system, so the methodological framing is meant to outlast the tools of any one run.

URL PDF HTML 收藏
2606.04460 2026-07-21 cs.CR cs.AI cs.LG 版本更新

CyberGym-E2E: Scalable Real-World Benchmark for AI Agents' End-to-End Cybersecurity Capabilities

CyberGym-E2E:面向AI代理端到端网络安全能力的可扩展真实世界基准

Tianneng Shi, Robin Rheem, Dongwei Jiang, Mona Wang, Francisco De La Riega, Zhun Wang, Jingzhi Jiang, Alexander Cheung, Sean Tai, Jonah Cha, Jianhong Tu, Gabriel Han, Chenguang Wang, Jingxuan He, Wenbo Guo, Dawn Song

机构 * Stanford University(斯坦福大学) UC Berkeley(加州大学伯克利分校)

AI总结 提出CyberGym-E2E,一个大规模、真实的端到端网络安全基准,通过自动化流水线将开源漏洞数据转化为评估环境,全面评估AI代理在漏洞发现、PoC生成和补丁生成全生命周期中的能力。

Comments ICML 2026

详情
AI中文摘要

人工智能有潜力通过使系统能够自主检测、分析和修复软件漏洞来改变网络安全。然而,现有对AI系统的网络安全评估在规模或范围上有限,未能捕捉真实世界软件漏洞发现和修复的端到端生命周期。为了解决这一差距,我们提出了CyberGym-E2E,一个大规模、真实的端到端网络安全基准,全面评估AI代理在漏洞发现、PoC生成和补丁生成整个生命周期中的能力。CyberGym-E2E全面且可扩展,因为我们构建了一个自动化的、代理增强的流水线,用于将开源漏洞数据转化为真实的评估环境。目前,该基准包含139个不同开源项目中的920个真实世界漏洞。

英文摘要

AI has the potential to transform cybersecurity by enabling systems that can autonomously detect, analyze, and remediate software vulnerabilities. However, existing cybersecurity evaluations of AI systems are limited in scale or scope, and fail to capture the end-to-end lifecycle of real-world software vulnerability discovery and remediation. To address this gap, we propose CyberGym-E2E, a large-scale and realistic end-to-end cybersecurity benchmark that comprehensively evaluates AI agents' abilities across the full lifecycle of vulnerability discovery, PoC generation, and patch generation. CyberGym-E2E is comprehensive and scalable, as we build an automated, agent-enhanced pipeline for transforming open-source vulnerability data into realistic evaluation environments. Currently, the benchmark consists of 920 real-world vulnerabilities across 139 different open-source projects.

URL PDF HTML 收藏
2605.22681 2026-07-21 cs.AI 版本更新

Scientific reasoning does not reliably translate into scientific forecasting in frontier AI

用人工智能预测科学进步

Sean Wu, Pan Lu, Yupeng Chen, Jonathan Bragg, Yutaro Yamada, Peter Clark, David Clifton, Philip Torr, James Zou, Junchi Yu

机构 * University of Oxford(牛津大学) Stanford University(斯坦福大学) Allen Institute for AI(人工智能研究所) Sakana AI

AI总结 本文研究了人工智能在预测科学进步中的能力,提出了一种基于时间的评估框架,并介绍了CUSP基准,通过可行性评估、机制推理、生成性解决方案设计和时间预测来评估AI系统的科学预测能力,发现当前前沿模型在不同领域存在系统性限制,且预测结果受事件发生时间影响较大,表明AI在科学预测中仍存在不足。

Comments 62 pages, 14 figures, 25 tables

详情
AI中文摘要

人工智能(AI)日益融入科学发现,但其能否预测科学进步仍不明确。为研究此问题,我们引入了一个基于时间的评估框架,用于在受控知识约束下预测科学进步。我们提出了CUSP(截止条件下的未见科学进步),一个多学科和事件级别的基准,通过可行性评估、机制推理、生成性解决方案设计和时间预测来评估AI系统在科学预测中的表现。在4760个科学事件中,我们观察到当前前沿模型在不同领域存在系统性和领域依赖性的限制。虽然模型可以识别出竞争候选研究方向的可能性,但它们无法可靠地预测科学进步是否会被实现,并系统性地低估了其发生时间。性能在不同领域中高度异质,AI的进步时间比生物学、化学和物理学的进步更可预测。性能在事件发生时间在训练截止前或后时基本不受影响,表明这些限制不能仅由训练数据中的知识暴露来解释。在受控信息访问下,额外的预截止知识会提高性能,但无法缩小与全信息设置之间的差距,这种差距在高引用进步中更加明显。模型还表现出系统性的过度自信和强烈的响应偏差,表明不确定性估计不可靠。综合来看,当前AI系统在预测科学进步方面仍显不足。获取先前知识并未转化为可靠的预测,性能更受益于事后信息而非前瞻性预测。

英文摘要

AI systems are increasingly used to support forward-looking scientific judgment, but it remains unclear whether they can form reliable expectations about future scientific advances. Here we show that strong scientific reasoning does not reliably translate into accurate forecasting of future scientific advances. To study this question, we introduce CUSP, a temporally grounded evaluation suite for event-level scientific forecasting across eight scientific disciplines. Across six frontier AI models, we observe a striking asymmetry in forecasting performance together with systematic error patterns. Models often identify plausible mechanisms underlying future scientific advances, yet perform near chance on feasibility assessment, generate solution strategies that only weakly align with realized advances, and systematically predict scientific advances later than they become publicly observable. Providing additional pre-cutoff scientific knowledge improves performance but does not eliminate these forecasting limitations. These findings suggest that current AI systems possess substantial retrospective scientific competence but limited forward-looking predictive capability. Scientific forecasting should therefore be evaluated as a complementary dimension of AI scientific capability when deploying AI systems for research prioritization and scientific decision-making.

URL PDF HTML 收藏
2505.12546 2026-07-21 cs.CL cs.CY cs.LG

Extracting memorized pieces of (copyrighted) books from open-weight language models

从开放式语言模型中提取记忆中的(受版权保护)书籍

A. Feder Cooper, Mark A. Lemley, Allison Casasola, Ahmed Ahmed, Aaron Gokaslan, Amy B. Cyphert, Christopher De Sa, Daniel E. Ho, Percy Liang

机构 * Stanford University(斯坦福大学) Yale University(耶鲁大学) Cornell University(康奈尔大学) West Virginia University(西弗吉尼亚大学)

AI总结 研究通过测量书籍在语言模型中的记忆程度,发现大多数模型不完整记忆书籍,但部分模型如Llama 3.1 70B能完整记忆某些书籍,对版权案件有重要影响。

Comments COLM 2026

详情
AI中文摘要

版权诉讼中,原告和被告常对立争论大型语言模型(LLMs)是否记忆受保护的书籍内容。本文开发了一种测量书籍记忆程度的技术,应用于200本书和14个开放式语言模型,通过3000多项实验显示记忆程度因模型和书籍而异。研究发现大多数LLM不完整记忆书籍,但某些模型如Llama 3.1 70B能完整记忆某些书籍,如《哈利波特与魔法石》,可利用书籍开头几词作为提示词确定性提取整本书。结果对版权案件有重要影响,但不明确偏向任何一方。

英文摘要

Plaintiffs and defendants in copyright lawsuits over generative AI often make sweeping, opposing claims about the extent to which large language models (LLMs) memorize protected expression from books in their training data. We show that these polarized positions dramatically oversimplify the relationship between memorization and copyright. To do so, we develop a technique to measure memorization of books, which we apply to 200 books and 14 open-weight LLMs. Through over 3000 experiments, we show that memorization varies both by model and book. With respect to our specific extraction methodology, we find that most LLMs do not memorize most books -- either in whole or in part; however, there are notable exceptions. For instance, Llama 3.1 70B entirely memorizes some books, like Harry Potter and the Sorcerer's Stone; memorization is so extensive that one can deterministically extract the whole book almost verbatim using the book's first few words as an initial prompt. We discuss why our results have significant implications for copyright cases, though not ones that unambiguously favor either side.

URL PDF HTML 收藏
2603.21693 2026-07-21 cs.AI 版本更新

Deterministic Hallucination Detection in Medical VQA via Confidence-Evidence Bayesian Gain

医学VQA中通过置信度-证据贝叶斯增益实现确定性幻觉检测

Mohammad Asadi, Tahoura Nedaee, Jack W. O'Sullivan, Euan Ashley, Ehsan Adeli

机构 * Department of Electrical Engineering, Stanford University, CA, USA(电气工程系,斯坦福大学) Department of Biology, Stanford University, CA, USA(生物学系,斯坦福大学) Division of Cardiology, Department of Medicine, Stanford University, CA, USA(心脏病学部,医学系,斯坦福大学) Department of Biomedical Data Science, Stanford University, CA, USA(生物医学数据科学系,斯坦福大学) Department of Computer Science, Stanford University, CA, USA(计算机科学系,斯坦福大学) Department of Psychiatry and Behavioral Sciences, Stanford University, CA, USA(精神病学与行为科学系,斯坦福大学)

AI总结 本文提出CEBaG方法,利用模型自身log概率中的不一致置信度和弱视觉证据敏感性,实现无需随机采样和外部模型的确定性幻觉检测,在医疗MLLM和VQA基准测试中取得最佳AUC表现。

详情
AI中文摘要

多模态大语言模型(MLLMs)在医疗视觉问答(VQA)中展现出强大潜力,但仍然容易产生幻觉,即生成与输入图像矛盾的响应,这在临床环境中存在严重风险。当前幻觉检测方法如语义熵(SE)和视觉增强语义熵(VASE)需要每个样本进行10到20次随机生成并结合外部自然语言推理模型进行语义聚类,这使它们计算成本高且难以部署。我们观察到幻觉响应在模型自身log概率中表现出独特的特征:在token层面的不一致置信度和对视觉证据的弱敏感性。基于此观察,我们提出置信度-证据贝叶斯增益(CEBaG),一种无需随机采样、外部模型或任务特定超参数的确定性幻觉检测方法。CEBaG结合了两个互补信号:token层面的预测方差,捕捉响应token间的不一致置信度,以及证据幅度,衡量图像相对于纯文本推理的token预测变化程度。在四个医疗MLLM和三个VQA基准测试(16个实验设置)中评估,CEBaG在16个设置中的13个中取得最高AUC,并在平均上比VASE提高8个AUC点,同时完全确定性和自包含。代码将在接受后提供。

英文摘要

Multimodal large language models (MLLMs) have shown strong potential for medical Visual Question Answering (VQA), yet they remain prone to hallucinations, defined as generating responses that contradict the input image, posing serious risks in clinical settings. Current hallucination detection methods, such as Semantic Entropy (SE) and Vision-Amplified Semantic Entropy (VASE), require 10 to 20 stochastic generations per sample together with an external natural language inference model for semantic clustering, making them computationally expensive and difficult to deploy in practice. We observe that hallucinated responses exhibit a distinctive signature directly in the model's own log-probabilities: inconsistent token-level confidence and weak sensitivity to visual evidence. Based on this observation, we propose Confidence-Evidence Bayesian Gain (CEBaG), a deterministic hallucination detection method that requires no stochastic sampling, no external models, and no task-specific hyperparameters. CEBaG combines two complementary signals: token-level predictive variance, which captures inconsistent confidence across response tokens, and evidence magnitude, which measures how much the image shifts per-token predictions relative to text-only inference. Evaluated across four medical MLLMs and three VQA benchmarks (16 experimental settings), CEBaG achieves the highest AUC in 13 of 16 settings and improves over VASE by 8 AUC points on average, while being fully deterministic and self-contained. https://github.com/masadi-99/CEBaG

URL PDF HTML 收藏
2603.16842 2026-07-21 cs.LG cond-mat.dis-nn cond-mat.stat-mech cs.SY eess.SY physics.bio-ph 版本更新

Stochastic Resetting Accelerates Reinforcement Learning Beyond Random Search

随机重置加速强化学习中的策略收敛

Jello Zhou, David J. Schwab, Vudtiwat Ngampruetikorn

机构 * Biophysics Program, Stanford University(斯坦福大学生物物理项目) School of Physics, University of Sydney(悉尼大学物理学院) National Institute for Theory and Mathematics in Biology, Northwestern University(生物理论与数学国家研究所,西北大学) The University of Chicago(芝加哥大学) Princeton-CUNY Center for the Physics of Biological Function, The Graduate Center, CUNY(普林斯顿-纽约大学生物物理功能中心,纽约大学研究生中心)

AI总结 研究探讨随机重置与强化学习的交互,发现其能加速策略收敛,通过截断长轨迹提升价值传播,为强化学习提供新的优化机制。

Comments 10 pages, 11 figures

详情
AI中文摘要

随机重置,即动态过程间歇性返回固定参考状态,已成为优化首次通过性质的强大机制。现有理论主要处理静态非学习过程。本文探讨随机重置与强化学习的交互,其中底层动态通过经验适应。在表格网格环境中,发现重置即使不减少纯扩散代理的搜索时间,也能加速策略收敛,表明一种超越经典首次通过优化的新机制。在基于神经网络的价值近似连续控制任务中,显示随机重置在探索困难且奖励稀疏时能提升深度强化学习。不同于时间折扣,重置保留最优策略并通过截断长、无信息轨迹加速收敛,增强价值传播。研究结果确立随机重置为简单可调的加速学习机制,将统计力学的典型现象转化为强化学习的优化原理。

英文摘要

Stochastic resetting -- intermittently returning a process to a fixed reference state -- has emerged as an effective mechanism for optimizing first-passage properties. Existing theory largely treats processes that search but do not learn: the searcher follows fixed dynamics, accumulating no knowledge between resets. Here we ask how stochastic resetting interacts with reinforcement learning, where the underlying dynamics adapt through experience. In tabular grid environments, we find that resetting can accelerate learning even when it does not reduce the search time of a diffusive agent. Our results reveal a distinct additional mechanism through which resetting speeds the propagation of reward information. We show that deterministic, sharp resetting accelerates learning more than the stochastic protocol but over a narrower range of reset rates. In a continuous-state task with neural-network-based value approximation, we demonstrate that resetting speeds up learning when exploration is hard and rewards are sparse. We argue further that, in the tabular tasks, resetting accelerates learning without altering the solution the agent ultimately reaches, unlike other techniques such as temporal discounting, which biases the optimal behavior. Our results establish stochastic resetting as a simple, tunable mechanism for accelerating learning by shaping how experience accumulates, extending a canonical phenomenon of statistical mechanics to adaptive systems.

URL PDF HTML 收藏
2506.06570 2026-07-21 cs.RO 版本更新

Unsupervised Discovery of Failure Taxonomies from Deployment Logs

无监督发现部署日志中的故障分类

Aryaman Gupta, Yusuf Umut Ciftci, Somil Bansal

机构 * Stanford University(斯坦福大学) University of Southern California(南加州大学)

AI总结 本文提出无监督发现部署日志中的故障分类方法,通过视觉-语言推理和语义聚类,提升机器人系统鲁棒性和故障监控能力。

详情
AI中文摘要

随着机器人系统越来越深入地融入现实世界环境,从自动驾驶汽车到家庭助手,它们不可避免地会遇到多样且无结构的场景,导致故障。虽然这些故障带来了安全性和可靠性挑战,但它们也提供了丰富的感知数据以提高系统鲁棒性。然而,手动分析大规模故障数据集是不切实际的,也无法扩展。在本文中,我们提出了从大量原始故障日志中无监督发现故障分类的问题,旨在直接从感知轨迹中获得语义连贯且可操作的故障模式。我们的方法首先通过视觉-语言推理从多模态输入中推断出结构化的故障解释,然后在由此产生的语义推理空间中进行聚类,从而发现重复的故障模式,而不是孤立的事件级描述。我们评估了我们的方法在机器人操作、室内导航和自动驾驶领域,并证明所发现的分类是连贯的、可解释的且在实践中是有用的。特别是,我们展示了结构化的故障分类指导了针对离线策略优化的数据收集,并增强了运行时故障监控系统。

英文摘要

As robotic systems become increasingly integrated into real-world environments, ranging from autonomous vehicles to household assistants, they inevitably encounter diverse and unstructured scenarios that lead to failures. While such failures pose safety and reliability challenges, they also provide rich perceptual data for improving system robustness. However, manually analyzing large-scale failure datasets is impractical and does not scale. In this work, we introduce the problem of unsupervised discovery of failure taxonomies from large volumes of raw failure logs, aiming to obtain semantically coherent and actionable failure modes directly from perceptual trajectories. Our approach first infers structured failure explanations from multimodal inputs using vision language reasoning, then clusters them in the resulting semantic reasoning space, discovering recurring failure modes rather than isolated episode-level descriptions. We evaluate our method across robotic manipulation, indoor navigation, and autonomous driving domains, demonstrating that the discovered taxonomies are consistent, interpretable, and useful in practice. In particular, we show that structured failure taxonomies guide targeted data collection for offline policy refinement and enhance runtime failure monitoring systems. Website: https://mllm-failure-clustering.github.io/

URL PDF HTML 收藏
2511.18436 2026-07-21 cs.CV 版本更新

When Generative Replay Meets Evolving Deepfakes: Dual Confusion-Aware Regularization for Incremental Face Forgery Detection

当生成式重放遇上不断演变的深度伪造:用于增量式人脸伪造检测的双混淆感知正则化

Hao Shen, Jikang Cheng, Renye Yan, Zhongyuan Wang, Wei Peng, Baojin Huang

机构 * Huazhong Agricultural University(华中农业大学) Peking University(北京大学) Wuhan University(武汉大学) Stanford University(斯坦福大学)

AI总结 研究人脸生成技术发展下的增量式深度伪造检测问题,提出双混淆感知正则化策略双CARE,通过引入域感知混淆分数量化域混淆,对重放生成器和检测器进行双调制,有效利用生成式重放改进检测效果。

详情
AI中文摘要

人脸生成技术的快速发展带来了越来越多的伪造方法,使得增量式深度伪造检测对于维持强大的检测器至关重要。虽然生成式重放为缓解灾难性遗忘且无需存储历史数据提供了一个有前景的解决方案,但其有效性受到生成样本与真实数据之间的“域混淆”阻碍。我们观察到重放样本分为两类:当重放生成器与新引入的伪造模型非常相似时,生成的真实样本与伪造域重叠并变得“域有风险”;当生成器差异很大时,生成的样本保持更清晰的域分离,可被视为“域安全”。为应对这一挑战,我们提出了一种双混淆感知正则化策略,称为双CARE。引入了一个域感知混淆分数(DC分数)来量化域混淆,并对重放生成器和增量检测器的优化进行“双调制”。在DC分数的指导下,更新重放生成器以更好地逼近先前任务的分布,而检测器采用不同的监督策略:对域安全样本直接监督,对域有风险的样本使用相对分离损失进行调节以平衡监督和混淆。大量实验表明,双CARE有效地利用了生成式重放,并在不断演变的伪造场景下改进了增量式深度伪造检测。

英文摘要

The rapid advancement of face generation techniques has introduced an increasing variety of forgery methods, making incremental deepfake detection essential for maintaining robust detectors. While generative replay provides a promising solution to alleviate catastrophic forgetting without storing historical data, its effectiveness is hindered by \textbf{domain confusion} between generated samples and real data. We observe that replay samples fall into two categories: when the replay generator closely resembles the newly introduced forgery model, generated real samples overlap with the fake domain and become \textbf{domain-risky}; when the generator differs significantly, generated samples maintain clearer domain separation and can be treated as \textbf{domain-safe}. To address this challenge, we propose a Dual \textbf{C}onfusion-\textbf{A}ware \textbf{RE}gularization strategy, termed \textbf{Dual-CARE}. A Domain-aware Confusion Score (DC Score) is introduced to quantify domain confusion and \textbf{dual-modulate} the optimization of both replay generators and the incremental detector. Guided by DC Score, replay generators are updated to better approximate previous-task distributions, while the detector adopts different supervision strategies: domain-safe samples are directly supervised, whereas domain-risky samples are regulated using a Relative Separation Loss to balance supervision and confusion. Extensive experiments demonstrate that Dual-CARE effectively exploits generative replay and improves incremental deepfake detection under evolving forgery scenarios.

URL PDF HTML 收藏
2404.11459 2026-07-21 cs.CL cs.CV 版本更新

Octopus v3: Technical Report for On-device Sub-billion Multimodal AI Agent

章鱼v3:用于设备端低于十亿参数的多模态人工智能代理的技术报告

Wei Chen, Zhiyuan Li

机构 * Stanford University(斯坦福大学)

AI总结 研究针对多模态人工智能代理将图像数据转化为可行动结果的挑战,引入含功能令牌概念的多模态模型,优化至小于10亿参数,能处理中英文,可在多种边缘设备高效运行。

详情
AI中文摘要

多模态人工智能代理的特点是能够处理各种类型的数据并从中学习,以指导其行动。尽管像GPT-4V这样结合视觉数据的大语言模型有进展,但将基于图像的数据有效转化为人工智能代理的可行动结果仍具挑战。本文引入了一个为人工智能代理应用专门设计功能令牌概念的多模态模型。为与边缘设备兼容,模型被优化到小于10亿参数的紧凑规模。像GPT-4一样,该模型能处理中英文。实验表明此模型能在包括树莓派等多种边缘设备上高效运行。

英文摘要

A multimodal AI agent is characterized by its ability to process and learn from various types of data, including natural language, visual, and audio inputs, to inform its actions. Despite advancements in large language models that incorporate visual data, such as GPT-4V, effectively translating image-based data into actionable outcomes for AI agents continues to be challenging. In this paper, we introduce a multimodal model that incorporates the concept of functional token specifically designed for AI agent applications. To ensure compatibility with edge devices, our model is optimized to a compact size of less than 1B parameters. Like GPT-4, our model can process both English and Chinese. We demonstrate that this model is capable of operating efficiently on a wide range of edge devices, including as constrained as a Raspberry Pi.

URL PDF HTML 收藏
2404.01744 2026-07-21 cs.CL

Octopus v2: On-device language model for super agent

章鱼v2:用于超级代理的设备端语言模型

Wei Chen, Zhiyuan Li

机构 * Stanford University(斯坦福大学)

AI总结 章鱼v2通过设备端20亿参数模型,在准确性和延迟上超越GPT-4,同时将上下文长度减少95%,提升边缘设备应用性能。

详情
AI中文摘要

语言模型在各种软件应用中已显示出有效性,特别是在与自动工作流相关的任务中。这些模型具备调用函数的关键能力,这对于创建AI代理至关重要。尽管在云环境中大规模语言模型表现出色,但它们通常与隐私和成本问题相关。当前用于函数调用的设备端模型面临延迟和准确性的问题。我们的研究提出了一种新方法,使具有20亿参数的设备端模型在准确性和延迟方面超越GPT-4,并将上下文长度减少95%。与具有基于RAG的函数调用机制的Llama-7B相比,我们的方法将延迟降低了35倍。该方法将延迟降低到适合在生产环境中部署在各种边缘设备上的水平,符合实际应用的性能要求。

英文摘要

Language models have shown effectiveness in a variety of software applications, particularly in tasks related to automatic workflow. These models possess the crucial ability to call functions, which is essential in creating AI agents. Despite the high performance of large-scale language models in cloud environments, they are often associated with concerns over privacy and cost. Current on-device models for function calling face issues with latency and accuracy. Our research presents a new method that empowers an on-device model with 2 billion parameters to surpass the performance of GPT-4 in both accuracy and latency, and decrease the context length by 95\%. When compared to Llama-7B with a RAG-based function calling mechanism, our method enhances latency by 35-fold. This method reduces the latency to levels deemed suitable for deployment across a variety of edge devices in production environments, aligning with the performance requisites for real-world applications.

URL PDF HTML 收藏
2404.01549 2026-07-21 cs.CL cs.SE 版本更新

Octopus: On-device language model for function calling of software APIs

章鱼:用于软件API函数调用的设备端语言模型

Wei Chen, Zhiyuan Li, Mingyuan Ma

机构 * Stanford University(斯坦福大学) Harvard University(哈佛大学)

AI总结 研究利用设备端大语言模型调用软件API,通过编译数据集微调不同参数模型,提升其API交互能力,提出条件掩码技术和新基准,经微调的Octopus模型在API调用上性能超GPT-4,推动自动化软件开发和API集成。

详情
AI中文摘要

在快速发展的人工智能领域,大语言模型(LLMs)因其先进的文本处理和生成能力发挥着关键作用。本研究引入一种在调用软件API时利用设备端LLMs的新策略。精心编译源自软件API文档的数据集,对2B、3B和7B参数的LLMs进行微调,以提高其在软件API交互方面的能力,专注提升模型对API结构和语法的理解,增强API函数调用准确性。还提出条件掩码技术确保输出格式正确并降低错误率,同时保持推理速度。提出新基准评估LLMs在API交互中的有效性。经微调的Octopus模型在软件API调用方面性能优于GPT-4,推动了自动化软件开发和API集成。

英文摘要

In the rapidly evolving domain of artificial intelligence, Large Language Models (LLMs) play a crucial role due to their advanced text processing and generation abilities. This study introduces a new strategy aimed at harnessing on-device LLMs in invoking software APIs. We meticulously compile a dataset derived from software API documentation and apply fine-tuning to LLMs with capacities of 2B, 3B and 7B parameters, specifically to enhance their proficiency in software API interactions. Our approach concentrates on refining the models' grasp of API structures and syntax, significantly enhancing the accuracy of API function calls. Additionally, we propose \textit{conditional masking} techniques to ensure outputs in the desired formats and reduce error rates while maintaining inference speeds. We also propose a novel benchmark designed to evaluate the effectiveness of LLMs in API interactions, establishing a foundation for subsequent research. Octopus, the fine-tuned model, is proved to have better performance than GPT-4 for the software APIs calling. This research aims to advance automated software development and API integration, representing substantial progress in aligning LLM capabilities with the demands of practical software engineering applications.

URL PDF HTML 收藏
2403.06025 2026-07-21 cs.CV cs.AI 版本更新

CarbonNet: How Computer Vision Plays a Role in Climate Change? Application: Learning Geomechanics from Subsurface Geometry of CCS to Mitigate Global Warming

CarbonNet:计算机视觉如何在气候变化中发挥作用?应用:从碳捕获与封存的地下几何结构学习地质力学以缓解全球变暖

Wei Chen, Yunan Li, Yuan Tian

机构 * Stanford University(斯坦福大学)

AI总结 研究利用计算机视觉从碳捕获与封存的地下几何图像预测地表位移,以应对相关挑战。实现多种模型用于静态和瞬态力学问题,实验表明ResNetUNet在静态问题中表现出色,LSTM在瞬态问题中与Transformer性能相当,为CCS项目决策提供支持。

详情
AI中文摘要

我们引入一种新方法,利用计算机视觉从碳捕获与封存(CCS)的地下几何图像预测地表位移。CCS是碳中和社会的关键组成部分,但存在模型规模大计算成本高、预训练模型泛化受复杂物理限制等挑战。我们直接从地下几何图像训练模型应对这些挑战,目标是理解碳注入导致的地表位移响应并为CCS项目决策提供信息。我们实现了用于静态力学问题(图像预测问题)的多种模型(CNN、ResNet和ResNetUNet),以及用于瞬态力学场景(视频预测问题)的LSTM和Transformer。结果表明ResNetUNet在静态力学问题中因架构优势优于其他模型,LSTM在瞬态问题中与Transformer性能相当。本文详细介绍数据集,接着描述方法部分的模型,结果与讨论阐述关键学习、观察和结论,最后展望未来工作。

英文摘要

We introduce a new approach using computer vision to predict the land surface displacement from subsurface geometry images for Carbon Capture and Sequestration (CCS). CCS has been proved to be a key component for a carbon neutral society. However, scientists see there are challenges along the way including the high computational cost due to the large model scale and limitations to generalize a pre-trained model with complex physics. We tackle those challenges by training models directly from the subsurface geometry images. The goal is to understand the respons of land surface displacement due to carbon injection and utilize our trained models to inform decision making in CCS projects. We implement multiple models (CNN, ResNet, and ResNetUNet) for static mechanics problem, which is a image prediction problem. Next, we use the LSTM and transformer for transient mechanics scenario, which is a video prediction problem. It shows ResNetUNet outperforms the others thanks to its architecture in static mechanics problem, and LSTM shows comparable performance to transformer in transient problem. This report proceeds by outlining our dataset in detail followed by model descriptions in method section. Result and discussion state the key learning, observations, and conclusion with future work rounds out the paper.

URL PDF HTML 收藏
2607.15631 2026-07-20 q-bio.NC cs.CV 新提交

STSBench: A Large-Scale Dataset for Modeling Neuronal Activity in the Dorsal Stream of Primate Visual Cortex

STSBench:用于灵长类动物视觉皮层背侧流神经元活动建模的大规模数据集

Ethan B. Trepka, Ruobing Xia, Shude Zhu, Sharif Saleki, Danielle Abreu Lopes, Stephen J. Niño Cital, Konstantin F. Willeke, Mindy Kim, Tirin Moore

机构 * Neuroscience Interdepartmental Program, Stanford University(斯坦福大学神经科学跨学科项目) Department of Neurobiology, Stanford University(斯坦福大学神经生物学系) Howard Hughes Medical Institute, Stanford University(斯坦福大学霍华德·霍夫曼医学研究所)

AI总结 针对灵长类动物视觉系统背侧流建模缺乏大规模数据集的问题,提出STSBench数据集,它包含超2000个神经元记录,比现有数据集增加近50倍,可用于背侧流神经元反应编码模型基准测试及视觉输入重建。

Comments 21 pages, 10 figures, Advances in Neural Information Processing Systems 38 (NeurIPS 2025) Datasets and Benchmarks Track

Journal ref Advances in Neural Information Processing Systems 38 (2025)

详情
AI中文摘要

灵长类动物视觉系统通常分为两个流——负责物体识别的腹侧流和负责编码空间关系与运动的背侧流。近期研究表明,在物体识别任务上预训练的卷积神经网络能有效预测腹侧流中的神经元反应。但由于缺乏涵盖背侧流区域的大规模数据集,背侧流的类似模型仍未充分发展。为填补这一空白,我们展示了STSBench,这是一个来自颞上沟(STS)超过2000个神经元的大规模单神经元记录数据集,比现有背侧流数据集增加了近50倍,是在恒河猴观看数千个独特自然视频时收集的。我们表明,我们的数据集可用于对背侧流神经元反应的编码模型进行基准测试,并从神经活动中重建视觉输入。

英文摘要

The primate visual system is typically divided into two streams - the ventral stream, responsible for object recognition, and the dorsal stream, responsible for encoding spatial relations and motion. Recent studies have shown that convolutional neural networks (CNNs) pretrained on object recognition tasks are remarkably effective at predicting neuronal responses in the ventral stream, shedding light on the neural mechanisms underlying object recognition. However, similar models of the dorsal stream remain underdeveloped due to the lack of large scale datasets encompassing dorsal stream areas. To address this gap, we present STSBench, a dataset of large-scale, single neuron recordings from over 2,000 neurons in the superior temporal sulcus (STS), a nearly 50-fold increase over existing dorsal stream datasets, collected while Rhesus macaques viewed thousands of unique, natural videos. We show that our dataset can be used for benchmarking encoding models of dorsal stream neuronal responses and reconstructing visual input from neural activity.

URL PDF HTML 收藏
2607.16187 2026-07-20 cs.RO 新提交

Handroid: Bridging Dexterous Hand and Humanoid

Handroid:连接灵巧手与人形机器人

Ruogu Li, Chenyang Ma, Sikai Li, Zhenyu Wei, Yunchao Yao, Haochen Shi, C. Karen Liu, Shuran Song, Mingyu Ding

机构 * University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校) Stanford University(斯坦福大学)

AI总结 研究旨在开发兼具灵巧手与拟人机器人功能的平台,提出Handroid,它能在单一可重构平台集成两种能力,有统一控制学习框架,经多种任务验证,成为推进形态可重构机器人技术及跨实体学习的紧凑可复制平台。

Comments Project website: https://handroid.org

详情
AI中文摘要

灵巧手和人形机器人通常是作为不同的实体开发的:前者能在物体尺度上进行丰富的接触式操作,后者能在以人类为中心的环境中实现移动和全身交互。我们介绍了Handroid,一个桌面级的双实体机器人,它在单个可重构平台上集成了这两种能力。Handroid重用一个27自由度的机电体,既可以作为灵巧手,也可以作为桌面人形机器人,高0.33米,重2.05千克。在灵巧手模式下,20个自由度形成一个与人手运动结构紧密匹配的拟人化手。在人形模式下,相同的关节模块重新配置成一个有头、手臂和腿的人形机器人,包括一个用于移动和全身运动的12自由度下肢结构。Handroid还提供了一个统一的控制和学习框架,支持手部遥操作、灵巧抓取、手中操作、人形机器人移动、步态生成和交互式运动创作。我们通过实际的灵巧操作、基于强化学习的移动、关键帧运动部署以及一个涉及实体重新配置、移动、对接和灵巧抓取放置的长期任务来验证该平台。这些结果使Handroid成为一个紧凑且可复制的平台,用于推进形态可重构机器人技术和跨实体机器人学习。

英文摘要

Dexterous hands and humanoid robots are typically developed as distinct embodiments: the former enable contact-rich manipulation at the object scale, whereas the latter provide mobility and whole-body interaction in human-centered environments. We introduce \textbf{Handroid}, a desktop-scale dual-embodiment robot that integrates both capabilities within a single reconfigurable platform. Handroid reuses one 27-DoF electromechanical body as either a dexterous hand or a desktop humanoid, measuring 0.33 m in height and 2.05 kg in weight. In the dexterous hand embodiment, 20 DoFs form an anthropomorphic hand closely matching the kinematic structure of the human hand. In the humanoid embodiment, the same articulated modules are reconfigured into a humanoid with a head, arms, and legs, including a 12-DoF lower-limb structure for locomotion and whole-body motion. Handroid further provides a unified control and learning framework supporting hand teleoperation, dexterous grasping, in-hand manipulation, humanoid locomotion, gait generation, and interactive motion authoring. We validate the platform through real-world dexterous manipulation, reinforcement-learning-based locomotion, keyframe motion deployment, and a long-horizon task involving embodiment reconfiguration, locomotion, docking, and dexterous pick-and-place. These results position Handroid as a compact and reproducible platform for advancing morphology-reconfigurable robotics and cross-embodiment robot learning.

URL PDF HTML 收藏
2606.16073 2026-07-20 cs.LG stat.ML 版本更新

Stop the Sampler! Classifier-Based Adaptive Stopping for Sampling Kernels

停止采样器!基于分类器的采样核自适应停止

Kirill Korolev, Nikita Morozov, Stepan Pavlenko, Esmeralda S. Whitammer, Sergey Samsonov

机构 * Stanford University(斯坦福大学)

AI总结 提出将MCMC轨迹终止作为可学习组件,利用非循环生成流网络训练状态依赖分类器,在保证详细平衡条件下自适应停止采样,显著缩短轨迹长度并改善模式覆盖与混合。

Comments ICML 2026 SPIGM Workshop

详情
AI中文摘要

从复杂、未归一化的概率密度中采样是贝叶斯推断和概率建模中的基本挑战。虽然马尔可夫链蒙特卡罗(MCMC)方法提供了渐近保证,但由于固定或手动调整的轨迹长度,它们常常遭受慢混合和高计算成本。在这项工作中,我们提出了一种新颖的框架,将轨迹终止视为采样动力学的可学习组件。通过将MCMC置于非循环生成流网络(GFlowNets)的理论中,我们训练状态依赖的神经分类器来决定轨迹何时到达高密度区域并应终止。我们通过详细平衡条件从理论上建立了最优分类器与目标密度之间的联系,并引入了一种多级训练方案以促进复杂几何中的探索。在各种基准密度上的实验结果表明,与标准MCMC基线相比,我们的方法显著减少了平均轨迹长度,同时改善了模式覆盖和混合。

英文摘要

Sampling from complex, unnormalized probability densities is a fundamental challenge in Bayesian inference and probabilistic modeling. While Markov chain Monte Carlo (MCMC) methods provide asymptotic guarantees, they often suffer from slow mixing and high computational costs due to fixed or manually tuned trajectory lengths. In this work, we propose a novel framework that treats trajectory termination as a learnable component of the sampling dynamics. By framing MCMC within the theory of non-acyclic generative flow networks (GFlowNets), we train state-dependent neural classifiers to decide when a trajectory has reached a high-density region and should terminate. We theoretically establish the connection between optimal classifiers and the target density via detailed balance conditions and introduce a multilevel training scheme to facilitate exploration in complex geometries. Experimental results across various benchmark densities demonstrate that our approach significantly reduces average trajectory lengths while improving mode coverage and mixing compared to standard MCMC baselines.

URL PDF HTML 收藏
2406.01514 2026-07-20 cs.CL cs.AI cs.CR 版本更新

Decoupled Alignment for Robust Plug-and-Play Adaptation

用于鲁棒即插即用适应的解耦对齐

Haozheng Luo, Jiahao Yu, Wenxin Zhang, Jialong Li, Chenghao Qiu, Yimin Wang, Eric Hanchen Jiang, Jerry Yao-Chieh Hu, Yan Chen, Binghui Wang, Xinyu Xing, Han Liu

机构 * Northwestern University(西北大学) New York University Abu Dhabi(纽约大学阿布扎克分校) Stanford University(斯坦福大学) Texas A&M University(德克萨斯农工大学) University of California, Los Angeles(加州大学洛杉矶分校) Illinois Institute of Technology(伊利诺伊理工学院)

AI总结 研究提出无需训练的大语言模型对齐方法,利用知识蒸馏提取对齐信号,经模型融合实现即插即用的对齐校正,采用增量调试识别关键知识组件,在有害问题数据集上显著提升防御成功率,且不损性能。

Comments Revised to correct the Acknowledgments section. Previous versions inadvertently included acknowledgments of NSF and NIH awards that did not support this work. Those funding acknowledgments have been removed. The technical content, results, and conclusions are unchanged

详情
AI中文摘要

我们引入一种无需训练的安全增强方法,用于对齐大语言模型,无需监督微调或从人类反馈进行强化学习。主要思想是提供一种鲁棒的即插即用方法,防止模型适应下游任务时的影子对齐。具体利用知识蒸馏从对齐良好的大语言模型中提取对齐信号,并通过模型融合注入到影子对齐模型中,实现即插即用的对齐校正。采用增量调试识别有效蒸馏所需知识的关键组件。在有害问题数据集上,该方法显著提高平均防御成功率约14.42%,在17个受影响的大语言模型上高达51.39%,且不影响性能。代码可通过链接获取。

英文摘要

We introduce a training-free safety enhancement method for aligning large language models (LLMs) without the need for supervised fine-tuning or reinforcement learning from human feedback. Our main idea is to provide a robust plug-and-play approach to prevent shadow alignment when models are adapted to downstream tasks. Specifically, we leverage knowledge distillation to extract alignment signals from well-aligned LLMs and inject them into shadow-aligned models via model fusion, enabling plug-and-play alignment correction. In our methodology, we employ delta debugging to identify the critical components of knowledge necessary for effective distillation. On the harmful question dataset, our method significantly enhances the average defense success rate by approximately 14.42%, reaching as high as 51.39% across 17 influenced LLMs, without compromising performance. Our code is available at https://github.com/NWULIST/DAPA.

URL PDF HTML 收藏
2205.04599 2026-07-20 cs.LG cs.AI 版本更新

Perception-Aligned AI Outputs: End-to-End Visual Prediction for Uncertainty Communication in Clinical Decision-Making

感知对齐的人工智能输出:临床决策中用于不确定性通信的端到端视觉预测

Mohammad Eslami, Solale Tabarestani, Saber Kazeminasab, Ehsan Adeli, Glyn Elwyn, Tobias Elze, Mengyu Wang, Nazlee Zebardast, Lucia Sobrin, Nassir Navab, Daniel Shu Wei Ting, Malek Adjouadi

机构 * Harvard Ophthalmology AI Lab(哈佛眼科人工智能实验室) Schepens Eye Research Institute of Massachusetts Eye and Ear(马萨诸塞眼耳医院施佩恩眼科研究所) Harvard Medical School(哈佛医学院) Center for Advanced Technology and Education(先进教育技术中心) Florida International University(佛罗里达国际大学) Dartmouth Institute for Health Policy and Clinical Practice(达特茅斯健康政策与临床实践研究所) Dartmouth College(达特茅斯学院) Computer Aided Medical Procedures(医学辅助程序) Technical University of Munich(慕尼黑技术大学) Singapore Eye Research Institute(新加坡眼科研究所) Singapore National Eye Centre(新加坡国家眼科中心) Department of Ophthalmology, Byers Eye Institute, Stanford University(眼科部门,比尔斯眼科研究所,斯坦福大学)

AI总结 研究针对医疗保健中可解释人工智能的问题,提出以人为本的机器学习可视化学习框架VL4ML,通过直观视觉表示传达模型预测与不确定性,经多临床任务验证及评估,结果显示其能有效支持临床决策,具有广泛可及性。

详情
AI中文摘要

可解释人工智能(XAI)对医疗保健领域中可靠的人工智能至关重要,但现有许多方法依赖于临床医生和患者难以理解的技术解释。我们引入了机器学习可视化学习(VL4ML),这是一个以人为本的可解释性框架,通过直观的视觉表示而非数值或事后解释来传达模型预测和不确定性。通过在颜色、图案和空间结构中编码诊断信息,VL4ML使用户无需了解模型内部或统计专业知识就能解释预测。我们在包括分类、回归、纵向预测和多模态分析等多个临床任务中展示了该框架。通过一项涉及158名参与者(39.2%为临床专业人员)的以人为本的研究和专家可解释性评估对其有效性进行了评估。超过79%的参与者在评估维度上对视觉解释给予了积极评价,84.0%的人认为它们比数字输出更令人难忘,76.9%的人报告决策更快。超过82%的人在没有事先统计训练的情况下成功感知到视觉表示中嵌入的不确定性。临床医生和非临床医生之间以及男性和女性参与者之间没有观察到显著差异,表明其具有广泛的可及性。这些结果表明,VL4ML通过提供直观、普遍可解释的视觉解释来补充现有的XAI和不确定性量化方法,支持透明和可靠的临床决策。

英文摘要

Explainable Artificial Intelligence (XAI) is essential for trustworthy AI in healthcare, yet many existing methods rely on technical explanations that are difficult for clinicians and patients to interpret. We introduce Visualized Learning for Machine Learning (VL4ML), a human-centered explainability framework that communicates model predictions and uncertainty through intuitive visual representations rather than numerical or post-hoc explanations. By encoding diagnostic information in colors, patterns, and spatial structures, VL4ML enables users to interpret predictions without requiring knowledge of model internals or statistical expertise. We demonstrate the framework across multiple clinical tasks, including classification, regression, longitudinal prediction, and multimodal analysis. Its effectiveness was evaluated through a human-centered study involving 158 participants (39.2% clinical professionals) and an expert interpretability assessment. More than 79% of participants positively rated the visual explanations across evaluation dimensions, 84.0% found them more memorable than numeric outputs, and 76.9% reported faster decision-making. Over 82% successfully perceived uncertainty embedded in the visual representations without prior statistical training. No significant differences were observed between clinicians and non-clinicians or between male and female participants, indicating broad accessibility. These results suggest that VL4ML complements existing XAI and uncertainty quantification methods by providing intuitive, universally interpretable visual explanations that support transparent and trustworthy clinical decision-making.

URL PDF HTML 收藏
2607.14163 2026-07-17 q-bio.QM cs.CV cs.LG 新提交

A vision foundation model for single-cell biology via spatial gene cartography

通过空间基因制图构建单细胞生物学的视觉基础模型

Ridvan Yesiloglu, Sakib Mostafa, James Zou, Ash Alizadeh, Jiajun Wu, Lei Xing, Ehsan Adeli, Md Tauhidul Islam

机构 * Stanford University(斯坦福大学) Stanford University School of Medicine(斯坦福大学医学院)

AI总结 研究提出scVision视觉基础模型,利用最优传输将基因布局成图像,通过掩码图像建模预训练视觉Transformer。该模型在零样本评估中表现出色,能无监督恢复基因程序,在多研究整合中效果好,还证明基因布局位置携带信号,重塑单细胞表示学习为视觉问题。

Comments 54 pages, 34 figures, 10 tables, including supplementary information. Project page: https://islamlab.org/scvision

详情
AI中文摘要

大多数单细胞基础模型改编自语言模型,将每个细胞表示为基因令牌序列,这会丢弃基因间关系及其表达量。本文提出scVision,一种将每个细胞渲染为连续图像的视觉基础模型。利用最优传输将基因置于单个共享的全组织布局上的固定位置,使共表达基因成为空间邻居。通过对7200万个细胞进行掩码图像建模预训练视觉Transformer,在六项独立研究的零样本评估中,scVision是最准确的细胞类型注释器,能无监督恢复基因程序,在多研究整合中表现出色,证明有生物学意义的位置携带信号,并将单细胞表示学习重塑为视觉问题。

英文摘要

Most single-cell foundation models are adapted from language models, representing each cell as a sequence of gene tokens. This discards the relationships among genes and often the magnitude of their expression. We present scVision, a vision foundation model that instead renders each cell as a continuous image. Using optimal transport, it places genes at fixed positions on a single shared, pan-tissue layout so that co-expressed genes become spatial neighbours, turning a transcriptome into an image in which gene programs appear as local texture. We pretrain a vision transformer by masked image modelling on 72 million human cells and use the frozen encoder with no fine-tuning. In zero-shot evaluations on six independent, held-out studies, scVision is the most accurate cell-type annotator and recovers gene programs without supervision, ahead of existing foundation models and classical baselines; on multi-study integration it matches the strongest token-based model while conserving the most biological structure, without ever seeing a batch label. Permuting the gene layout with the network fixed sharply lowers accuracy, more than removing the vision transformer itself, showing that biologically meaningful position, not the network, carries the signal. By preserving expression magnitude and gene relationships, scVision reframes single-cell representation learning as a vision problem, connecting it to the mature methods of computer vision.

URL PDF HTML 收藏