arXivDaily arXiv每日学术速递 周一至周五更新
重置
全部学科分类 3868
2606.09646 2026-06-09 cs.CV cs.AI cs.LG 新提交

Do Video Foundation Models Understand Intuitive Physics? A Layerwise Probing Analysis

视频基础模型是否理解直觉物理?逐层探测分析

Samuele Punzo, Niccolò Caselli, Ippokratis Pantelidis, Francesco Massafra, Salvatore Lo Sardo, Mohammadreza Salehi

发表机构 * University of Amsterdam(阿姆斯特丹大学)

AI总结 通过冻结特征探测,研究预训练视频基础模型在直觉物理信息上的编码能力,发现V-JEPA表现最佳,物理信息在中后期层最易获取,且时序破坏显著降低性能。

详情
AI中文摘要

我们研究预训练视频基础模型是否在其冻结表示中编码直觉物理信息,以及该信息如何随模型家族、层和探测类型变化。通过在IntPhys2和Minimal Video Pairs (MVP)上进行冻结特征探测,我们比较了预测联合嵌入模型(V-JEPA)、掩码重建模型(VideoMAE)和基于扩散的视频生成器(LTX-Video)。V-JEPA在基准测试中取得最强整体结果,尤其是在建模时序动态的探测器中,而VideoMAE仍具竞争力,LTX-Video恢复较弱但非平凡的信号。逐层分析表明,物理相关信息在早期层最弱,在中后期深度最易获取;时序控制表明,打乱帧顺序显著降低性能,尤其是在MVP上。综合来看,这些结果表明直觉物理知识在预训练视频表示中可靠地出现,但其可获取性强烈依赖于预训练范式、表示深度和读出机制。

英文摘要

We study whether pretrained video foundation models encode intuitive-physics information in their frozen representations, and how this information varies across model families, layers, and probe types. Using frozen-feature probing on IntPhys2 and Minimal Video Pairs (MVP), we compare predictive joint-embedding models (V-JEPA), masked reconstruction models (VideoMAE), and a diffusion-based video generator (LTX-Video). V-JEPA achieves the strongest overall results across benchmarks, especially with probes that model temporal dynamics, while VideoMAE remains competitive and LTX-Video recovers weaker but non-trivial signal. Layerwise analyses show that physics-relevant information is weakest in early layers and becomes most accessible at intermediate-to-late depth, and temporal controls show that disrupting frame order substantially reduces performance, especially on MVP. Together, these results suggest that intuitive-physics knowledge emerges reliably in pretrained video representations, but its accessibility depends strongly on pretraining paradigm, representational depth, and readout mechanism.

2606.09645 2026-06-09 cs.RO cs.PL cs.SE 新提交

Modeling Components and Connections in Cyber-Physical Systems

信息物理系统中的组件与连接建模

Kate Sanborn, Tanuj Kenchannavar, Vakul Nath, Jonathan Sprinkle

发表机构 * Vanderbilt University(范德堡大学)

AI总结 提出基于WebGME的模型集成工具ROSLaunchVisual,通过图形界面可视化ROS启动文件中的节点、发布者、订阅者和参数,提升开发效率和系统理解。

详情
AI中文摘要

信息物理系统的基于文本的配置文件很好地展示了组件模块的层次结构,但往往隐藏了模块之间连接和接口的细节。对这些配置文件采用基于模型的视觉方法可以更好地捕获这些信息。机器人操作系统(ROS)启动文件的XML结构可以通过建模方法得到改进。本文介绍了ROSLaunchVisual,一个基于WebGME构建的模型集成环境,用于设计、可视化和管理ROS启动文件。该工具通过允许开发者使用图形界面创建和修改启动文件来提高抽象层次,该界面将节点、发布者、订阅者和参数表示为互连组件。该工具提供动态系统分析,可用于新启动文件和现有启动文件的静态开发和分析。ROSLaunchVisual集成了元模型驱动验证、启动文件的自动导入/导出以及可视化通信映射等功能。插件通过更新库、检查语义错误和管理重映射进一步增强功能。通过使启动文件创建更直观且不易出错,ROSLaunchVisual提高了开发效率和系统理解,特别是在协作或大规模机器人项目中。

英文摘要

Text based configuration files for cyber-physical systems show the hierarchy of component modules well but often hide the details of connections and interfaces between modules. A model-based visual approach to these configuration files can better capture this information. The XML structure of Robot Operating System (ROS) launch files can be improved using a modeling approach. This paper presents ROSLaunchVisual, a model-integrated environment built on WebGME for designing, visualizing, and managing ROS launch files. The tool raises the level of abstraction by allowing developers to create and modify launch files using a graphical interface that represents nodes, publishers, subscribers, and arguments as interconnected components. The tool provides a dynamic system analysis that can then be used in the static development and analysis of new and existing launch files. ROSLaunchVisual incorporates features such as metamodel-driven validation, automatic import/export of launch files, and visual communication mapping. Plugins further enhance functionality by updating libraries, checking for semantic errors, and managing remaps. By making launch file creation more intuitive and less error-prone, ROSLaunchVisual improves development efficiency and system understanding, especially in collaborative or large-scale robotics projects.

2606.09644 2026-06-09 cs.CL cs.CV 新提交

Where Does the Answer Come From? Benchmarking View-Level Visual Evidence Identification in Multi-View MLLMs for Autonomous Driving

答案从何而来?面向自动驾驶的多视角MLLMs中视角级视觉证据识别基准

Yimu Wang, Yee Man Choi, Barry Zhang, Mozhgan Nasr Azadani, Sean Sedwards, Krzysztof Czarnecki

发表机构 * University of Waterloo(滑铁卢大学)

AI总结 针对多视角自动驾驶场景,提出一个基准测试,评估多模态大模型在视觉问答中识别支持性相机视角的能力,包含122个冲突中心问题对,并区分视角选择与答案正确性。

详情
AI中文摘要

多模态大语言模型(MLLMs)在视觉推理基准测试中取得了强劲结果,但仅凭答案准确性并不能表明模型是否依赖了正确的视觉证据。这一差距在用于自动驾驶的多视角驾驶场景中尤为重要,因为模型可能产生看似合理的答案,却将其归因于错误的相机视角。我们引入了一个多视角视觉问答基准,用于评估证据来源识别:给定六个同步的NuScenes视角和一个问题,模型必须识别支持性的相机视角并回答问题。该基准包含来自73个场景的122个冲突中心问答对,涵盖因果关系、反事实推理和意图预测。视角标签由自动冲突挖掘流程提出,并由标注者手动验证。我们评估了三种设置:相机视角选择、给定黄金视角的Oracle问答,以及模型在一次前向中同时选择视角并回答的联合预测。答案以多项选择和自由形式两种格式进行评估,使用精确匹配处理结构化预测,并使用LLM评判器处理自由形式回答。通过明确分离视觉来源识别与答案正确性,该基准揭示了仅凭答案评估无法发现的接地失败案例。

英文摘要

Multimodal large language models (MLLMs) achieve strong results on visual reasoning benchmarks, but answer accuracy alone does not indicate whether a model relied on the correct visual evidence. This gap is particularly important in multi-view driving scenes used for autonomous driving, where a model can produce a plausible answer while grounding it in the wrong camera view. We introduce a multi-view visual question answering benchmark for evaluating evidence-source identification: given six synchronized NuScenes views and a question, the model must identify the supporting camera view and answer the question. The benchmark contains 122 conflict-centric question-answer pairs from 73 scenes, spanning causality, counterfactual reasoning, and intent prediction. View labels are proposed by an automatic conflict-mining pipeline and manually verified by annotators. We evaluate three settings: camera-view selection, oracle QA given the golden view, and joint prediction in which the model selects a view and answers in one pass. Answers are evaluated in both multiple-choice and free-form formats, using exact match for structured predictions and an LLM judge for free-form responses. By explicitly separating visual-source identification from answer correctness, the benchmark exposes grounding failures that answer-only evaluation misses.

2606.09641 2026-06-09 cs.CV 新提交

MAVIS: Multi-Agent Video Retrieval via Structured Video Understanding

MAVIS: 通过结构化视频理解实现多智能体视频检索

Jie Zhang, Qilang Ye, Hao Zhou, Haochen Liang, Fei Luo

发表机构 * School of Computing and Information Technology, Great Bay University(大湾区大学计算机与信息技术学院) College of Computer Science, Nankai University(南开大学计算机学院) Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院) Graduate School of Information Science and Technology, The University of Tokyo(东京大学信息科学与技术研究生院)

AI总结 提出多智能体框架MAVIS,通过结构化语义库解析视频,利用逻辑感知辩论机制协作推理,无需全库扫描和微调即可实现高效视频检索。

详情
AI中文摘要

视频检索的主流范式依赖于基于嵌入的全库扫描,这种方法存在固有的计算低效以及信息密集视频与稀疏文本查询之间的语义不对称问题。为弥合这一差距,我们引入了\textbf{MAVIS},一种新颖的多智能体框架,将检索重新构想为协作推理而非暴力搜索。MAVIS首先通过将原始视频解析为\textbf{结构化语义库}来弥合粒度不匹配,从而实现显式的属性级索引。在检索过程中,规划器将复杂的用户意图分解为原子子任务,分派专门的智能体独立提名候选。关键的是,MAVIS采用带有严格否决协议的\textbf{逻辑感知辩论}机制,智能体协作修剪逻辑不匹配,以识别紧凑的“有争议”候选集进行细粒度验证。这种智能体工作流有效避免了全库遍历的低效。在MSR-VTT、MSVD和ActivityNet上的大量实验表明,MAVIS在无需任务特定微调的情况下实现了有竞争力的性能,为传统的双编码器方法提供了可扩展且可解释的替代方案。

英文摘要

The dominant paradigm in video retrieval relies on embedding-based full-corpus scanning, which suffers from inherent computational inefficiency and the semantic asymmetry between information-dense videos and sparse textual queries. To bridge this gap, we introduce \textbf{MAVIS}, a novel multi-agent framework that rethinks retrieval as cooperative reasoning rather than brute-force search. MAVIS first bridges the granularity mismatch by parsing raw videos into a \textbf{Structured Semantic Library}, enabling explicit attribute-level indexing. During retrieval, a planner decomposes complex user intents into atomic sub-tasks, dispatching specialized agents to independently nominate candidates. Crucially, MAVIS employs a \textbf{Logic-aware Debate} mechanism with a strict veto protocol, where agents collaboratively prune logical mismatches to identify a compact set of ``controversial'' candidates for fine-grained verification. This agentic workflow effectively bypasses the inefficiency of full-library traversal. Extensive experiments on MSR-VTT, MSVD, and ActivityNet demonstrate that MAVIS achieves competitive performance without task-specific fine-tuning, offering a scalable and interpretable alternative to traditional dual-encoder approaches.

2606.09640 2026-06-09 cs.RO 新提交

Physics-Aware Sparse Learning and Selective Online Adaptation for Euler-Lagrange Robot Dynamics

面向欧拉-拉格朗日机器人动力学的物理感知稀疏学习与选择性在线自适应

Rishabh Dev Yadav, Samaksh Ujjawal, Sihao Sun, Spandan Roy, Wei Pan

发表机构 * The University of Manchester(曼彻斯特大学) International Institute of Information Technology Hyderabad(海得拉巴国际信息技术学院) Delft University of Technology(代尔夫特理工大学) Newcastle University(纽卡斯尔大学)

AI总结 提出一种保结构残差学习框架,将模型误差分解为惯性修正、科里奥利项和广义力残差,通过物理约束学习机械部分,并用稀疏历史依赖潜变量模型和贝叶斯线性回归在线自适应扰动敏感部分,提升多机器人平台动力学预测与轨迹跟踪性能。

详情
AI中文摘要

精确的动力学模型对于基于模型的机器人控制至关重要,然而名义上的欧拉-拉格朗日模型在存在负载变化、未建模耦合、摩擦、空气动力学效应和变化操作条件时往往变得不准确。大多数基于学习的校正方法通过引入单个加性残差来提高预测精度,但未能保留欧拉-拉格朗日系统的内部机械结构。这导致模型不保留对称性、正定性或惯性与速度相关项之间的耦合,当嵌入基于模型的控制器时,可能导致物理上不一致的预测和降低的可靠性。我们提出了一种保结构残差学习框架,将模型不匹配分解为惯性修正、相应的诱导科里奥利项和广义力残差。机械部分在物理约束下学习,而扰动敏感部分通过稀疏历史依赖潜变量交互模型表示,并使用贝叶斯线性回归在线自适应。这种分离保留了关键的机械结构,同时将自适应限制在最受变化条件影响的动力学部分。在多个机器人平台(包括移动机器人、空中机器人和机械臂系统)上的实验表明,所提出的方法在耦合和时变动力学下改善了动力学预测和轨迹跟踪。这些结果凸显了将结构化残差建模、紧凑潜变量交互选择和选择性在线自适应相结合对于实际基于模型控制的价值。

英文摘要

Accurate dynamics models are essential for model-based robotic control, yet nominal Euler--Lagrange models often become inaccurate in the presence of payload variation, unmodeled coupling, friction, aerodynamic effects, and changing operating conditions. Most learning-based correction methods improve prediction accuracy by introducing a single additive residual, but do not preserve the internal mechanical structure of Euler--Lagrange systems. This leads to models that do not preserve symmetry, positive-definiteness, or the coupling between inertia and velocity-dependent terms, which can result in physically inconsistent predictions and reduced reliability when embedded in model-based controllers. We propose a structure-preserving residual learning framework that decomposes model mismatch into an inertia correction, the corresponding induced Coriolis term, and a generalized-force residual. The mechanical component is learned under physical constraints, while the disturbance-sensitive component is represented through a sparse history-dependent latent interaction model and adapted online using Bayesian linear regression. This separation preserves key mechanical structure while restricting adaptation to the part of the dynamics most affected by changing conditions. Experiments across multiple robotic platforms, including mobile, aerial, and manipulator systems, show that the proposed method improves dynamics prediction and trajectory tracking under coupled and time-varying dynamics. These results highlight the value of combining structured residual modeling, compact latent interaction selection, and selective online adaptation for real-world model-based control.

2606.09638 2026-06-09 cs.LG cs.SC math-ph math.MP physics.comp-ph stat.AP 新提交

Data-driven discovery of governing differential equations across physical systems

跨物理系统的控制微分方程数据驱动发现

Siyu Lou, Hao Xu, Wenguan Wang, Lu Lu, Hao Sun, Yang Liu, Linfeng Zhang, Dongxiao Zhang, Yuntian Chen

发表机构 * School of Computer Science, Shanghai Jiao Tong University(上海交通大学计算机科学与工程学院) Ningbo Key Laboratory of Advanced Manufacturing Simulation, Eastern Institute of Technology(东部理工学院宁波先进制造仿真重点实验室) The State Key Lab of Brain-Machine Intelligence, Zhejiang University(浙江大学脑机智能全国重点实验室) Department of Statistics and Data Science, Yale University(耶鲁大学统计与数据科学系) Department of Chemical and Environmental Engineering, Yale University(耶鲁大学化学与环境工程系) Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学高瓴人工智能学院) School of Engineering Sciences, University of Chinese Academy of Sciences(中国科学院大学工程科学学院) DP Technology

AI总结 本文提出问题导向视角,通过二维相图组织方程可发现性,并引入表示-评估-优化(REO)框架抽象发现过程,旨在从数据中推断物理定律,推动理论修正与新概念形成。

详情
AI中文摘要

微分方程在科学发现中扮演关键角色,因为它们提供了描述物理现象行为的数学框架。作为传统第一性原理的有前景替代,数据驱动微分方程发现因其直接从实验或模拟数据推断控制定律的能力而日益受到关注,尤其是在底层物理机制不明确时。然而,该领域沿着多样化的方法论方向迅速扩展,特别是随着基于AI的方法的出现,仍缺乏清晰的组织视角。在本综述中,我们提出数据驱动微分方程发现的问题导向视角。首先引入方程可发现性的二维相图,其中发现问题根据结构复杂性和系数复杂性进行组织。该相图展示了该领域如何从稀疏方程与简单系数的发现转向具有更丰富结构和更灵活参数化的更复杂控制定律。它还阐明了为什么不同的方法论家族在不同问题设置中成功或失败。然后,我们提出表示-评估-优化(REO)框架作为发现过程的基本抽象。通过识别跨算法变体持续存在的方程发现核心问题,REO将讨论从单个算法转向决定可发现性的基本原理。我们将这些视角与物理学及相邻科学的应用联系起来,并认为下一个挑战不仅仅是恢复方程,而是利用它们来修正现有理论、提炼机制并形成新的科学概念。

英文摘要

Differential equations play a critical role in scientific discovery because they provide a mathematical framework to describe the behaviour of physical phenomena. As a promising alternative to traditional first principles, data-driven differential equation discovery has attracted increasing attention for its ability to infer governing laws directly from experimental or simulated data, especially when the underlying physics is unclear. However, the field has expanded rapidly along diverse methodological directions, particularly with the emergence of AI-based approaches, and still lacks a clear organizing perspective. In this Review, we propose a problem-oriented perspective on data-driven differential equation discovery. We first introduce a two-dimensional phase diagram of equation discoverability, where discovery problems are organized according to structural complexity and coefficient complexity. This phase diagram shows how the field has moved from the discovery of sparse equations with simple coefficients toward more complex governing laws with richer structures and more flexible parameterizations. It also clarifies why different methodological families succeed or fail in different problem settings. We then present the representation-evaluation-optimization (REO) framework as a fundamental abstraction of the discovery process. By identifying the core problems of equation discovery that persist across algorithmic variations, REO shifts the discussion from individual algorithms to the fundamental principles that determine discoverability. We connect these perspectives to applications across physics and adjacent sciences, and argue that the next challenge is not merely recovering equations, but using them to revise existing theories, distil mechanisms and form new scientific concepts.

2606.09634 2026-06-09 cs.CV cs.AI 新提交

ATN3D: Density-Aware LiDAR-Radar Early 3D Object Detection Under Extreme Sparsity

ATN3D:面向极端稀疏性的密度感知激光雷达-雷达早期3D目标检测

Debojyoti Biswas, Xianbiao Hu

发表机构 * University of California, Berkeley(加州大学伯克利分校) Tsinghua University(清华大学)

AI总结 针对远距离稀疏感知下早期融合丢失信息、通道监督不均衡的问题,提出ATN3D框架,通过密度感知融合、占用门控邻域聚合、证据条件通道自注意力和距离感知损失,在VoD数据集上显著提升远距离检测性能。

详情
AI中文摘要

3D目标检测是自动驾驶车辆及更广泛智能交通系统感知的基石。远距离检测因感知证据稀疏而具有挑战性,然而这种“远距离”场景在交通中很常见。尽管在计算机视觉中>30m常被标记为远距离,但在道路上仅提供约1-2秒的感知和决策时间。在这种极端稀疏性下,出现两个核心挑战。首先,早期多模态融合倾向于丢弃稀疏性信息,并从空或错误占用的单元中注入噪声,降低远距离召回率。其次,上下文无关的统一通道监督偏向密集和近距样本,导致远处和小目标优化不足,延迟对远处目标的最早检测。我们提出“Ask The Neighbor”(ATN3D),一种专为稀疏范围条件设计的激光雷达-雷达框架。ATN3D引入:(i) 密度感知早期融合与跨模态门控,根据体素密度/稀疏性和雷达证据调节融合;(ii) 占用门控邻域聚合,使用圆形核仅从可信单元聚合;(iii) 证据条件通道自注意力,根据天气/距离自适应调整通道权重;(iv) 距离感知损失,按距离重新平衡分类和定位,使训练与距离分层评估对齐。在VoD基准的晴朗和雾天条件下,ATN3D超越强基线:晴朗天气mAP提升+3.55%,模拟浓雾下提升+8.41%;对于>30m目标,提升分别为+3.33%(晴朗)和+2.09%(浓雾)。这些结果表明在道路稀疏感知下更早、更可靠的远距离检测。

英文摘要

3D object detection is the backbone of perception for automated vehicles (AV) and broader intelligent transportation systems applications. Long-range detection is challenging because sensing evidence is sparse; yet this ``long-range'' scenario is routine in traffic. Although >30m is often labeled long-range in computer vision, on roadways it affords only approx. 1-2s for perception and decision-making. Under such extreme sparsity, two core challenges arise. First, early multimodal fusion tends to discard sparsity information and inject noise from empty or falsely occupied cells, degrading long-range recall. Second, context-agnostic uniform channel supervision favors dense and near-range samples, leaving far and small objects under-optimized, delaying the earliest detection of distant objects. We propose ``Ask The Neighbor'' (ATN3D), a LiDAR-Radar framework tailored for sparse-range conditions. ATN3D introduces (i) Density-aware early fusion with cross-modal gating that conditions fusion on per-voxel density/sparsity and Radar evidence, (ii) Occupancy-gated neighborhood aggregation with circular kernels to aggregate only from credible cells, (iii) Evidence-conditioned channel self-attention to adapt channel weights with weather/range, and (iv) a Range-aware loss that re-balances classification and localization by distance, aligning training with distance-stratified evaluation. On the VoD benchmark across clear and foggy conditions, ATN3D surpasses strong baselines: +3.55% mAP in clear weather and +8.41% mAP under simulated heavy fog; for >30m objects, gains are +3.33% (clear) and +2.09% (heavy fog). These results indicate earlier and more reliable long-range detections under sparse sensing in on-road traffic.

2606.09632 2026-06-09 cs.CL 新提交

Civil Court Simulation with Large Language Models

基于大型语言模型的民事法庭模拟

Yifan Chen, Haitao Li, Kaiyuan Zhang, Yueyue Wu, Qingyao Ai, Yiqun Liu

发表机构 * Beijing University of Posts and Telecommunications(北京邮电大学) Tsinghua University(清华大学)

AI总结 提出多智能体民事法庭模拟框架,通过五阶段审判程序、记忆模块和法规检索实现可靠判决,在责任分配和多项裁决上表现优异。

详情
AI中文摘要

法庭模拟连接了法律教育与司法实践,但基于人类的模拟成本高且难以扩展。大型语言模型(LLMs)提供了一种可扩展的替代方案,但现有的法庭模拟研究主要集中于刑事案件。民事诉讼在实践中更为常见且更难模拟,因为其诉求、责任和救济方式更加灵活。我们提出了一个面向中国民事案件的多智能体法庭模拟框架。该框架通过五阶段民事审判程序组织基于角色的交互,并集成记忆模块和法规检索以支持长过程裁判。实验表明,该框架能产生可靠的民事判决,在责任分配和多项裁决方面具有明显优势。进一步实验显示,记忆质量显著影响下游模拟质量。通过五层因素框架,我们分析了法律基础、信息条件、司法能力与角色定位、组织压力以及社会背景如何影响框架的可靠性和行为。这些结果支持了所提框架在民事法庭模拟中的有效性。数据集和代码可在 https://github.com/foggpoy/Civil-Court 获取。

英文摘要

Court simulation bridges legal education and judicial practice, yet human-based simulations are costly and difficult to scale. Large language models (LLMs) offer a scalable alternative, but existing court-simulation research mainly focuses on criminal cases. Civil litigation is more common in practice and harder to simulate because its claims, liability, and remedies are more flexible. We present a multi-agent court simulation framework for Chinese civil cases. The framework organizes role-based interaction through a five-stage civil trial procedure and integrates memory module and statute retrieval to support long-process adjudication. Experiments show that the framework produces reliable civil judgments, with clear strengths in liability allocation and multi-item adjudication. Further experiments show that memory quality substantially affects downstream simulation quality. Through a five-layer factor framework, we analyze how legal grounding, information conditions, judicial capability and role orientation, organizational pressure, and social context affect the framework's reliability and behavior. These results support the effectiveness of the proposed framework for civil court simulation. The dataset and code are available at: https://github.com/foggpoy/Civil-Court.

2606.09630 2026-06-09 cs.RO cs.AI cs.LG 新提交

ReCoVLA: VLM-Guided Reward Compilation for Failure Recovery in Vision-Language-Action Policies

ReCoVLA: VLM引导的奖励编译用于视觉-语言-动作策略的故障恢复

Haodi Hu, Chung-Ta Huang, Jing Liu, Ye Wang, Kei Suzuki, Matthew Brand, Toshiaki Koike-Akino

发表机构 * University of Southern California(南加州大学) Mitsubishi Electric Research Laboratories (MERL)(三菱电机研究实验室) Harvard University(哈佛大学)

AI总结 提出ReCoVLA框架,通过冻结预训练VLA策略,利用外部VLM推断故障模式并编译结构化奖励,训练残差恢复策略,实现零样本仿真到真实部署,在多种操作任务中提升成功率。

Comments 19 pages, 7 figures

详情
AI中文摘要

视觉-语言-动作(VLA)策略为语言条件操作提供了强大的先验知识,但在需要针对性恢复的非标称状态下仍然脆弱。我们提出ReCoVLA——一种故障条件的残差恢复框架,它保持预训练的VLA策略冻结,使用外部视觉-语言模型(VLM)推断故障模式和恢复阶段,并从任务相关组件编译结构化奖励。ReCoVLA并非使用VLM直接生成动作或奖励,而是将其作为语义奖励选择器:它预测恢复描述符和奖励掩码,用于仿真中的残差策略训练,随后将训练好的恢复策略零样本部署到真实世界。这解耦了高层故障理解与低层纠正控制,以支持不同的VLA。在短时域、长时域和接触丰富的操作任务上的实验表明,ReCoVLA在平均性能上优于测试的基线。在仿真中,我们的奖励编译器将微调$π_{0.5}$基线的平均成功率从36.7%提升到66.7%。在物理零样本仿真到真实实验中,ReCoVLA取得了最佳平均性能,成功率为61.7%。

英文摘要

Vision-language-action (VLA) policies provide strong priors for language-conditioned manipulation, but remain brittle in off-nominal states requiring targeted recovery. We propose ReCoVLA -- a failure-conditioned residual recovery framework that keeps a pretrained VLA policy frozen, uses an external vision-language model (VLM) to infer the failure mode and recovery stage, and compiles a structured reward from task-relevant components. Rather than using the VLM to generate actions or rewards directly, ReCoVLA uses it as a semantic reward selector: it predicts a recovery descriptor and reward mask for in-simulation residual-policy training, followed by zero-shot sim-to-real deployment of the trained recovery policies. This decouples high-level failure understanding from low-level corrective control to support different VLAs. Experiments across short-horizon, long-horizon, and contact-rich manipulation tasks show that ReCoVLA outperforms the tested baselines on average. In simulation, our reward compiler improves average success from 36.7% for the fine-tuned $π_{0.5}$ baseline to 66.7%. In physical zero-shot sim-to-real experiments, ReCoVLA achieves the best average performance, with 61.7% success.

2606.09623 2026-06-09 cs.LG 新提交

Constrained user-item allocation for e-commerce marketing campaigns

面向电子商务营销活动的约束用户-物品分配

Maja Lindström, Natalija Glisovic, Jan von Pichowski, Tommy Löfstedt, Martin Rosvall

发表机构 * Umeå University(于默奥大学) KTH Royal Institute of Technology(皇家理工学院) University of Würzburg(维尔茨堡大学)

AI总结 提出自动定向方法,通过约束谱双聚类、贪心局部搜索和多臂老虎机框架联合选择用户和物品构建多个不重叠营销活动,在合成数据、Amazon评论和商业数据上优于模拟退火。

详情
AI中文摘要

在开展营销活动时,零售商必须决定推广哪些产品以及针对哪些用户。这些决策本质上是耦合的:有效的活动将具有强烈相互亲和力的用户和物品匹配到预定义大小的非重叠组中。然而,现有方法假设预定义的活动结构或将物品选择与用户分配解耦,无法直接从联合交互模式中发现活动分组。因此,我们将该活动问题形式化为自动定向:联合选择用户和物品以构建多个不相交的活动。为了解决这个组合问题,我们提出了三种互补策略:(i)约束谱双聚类,以在用户-物品亲和力矩阵中找到密集区域;(ii)具有成对交换的贪心局部搜索,用于组合优化;(iii)多臂老虎机框架,通过探索逃离局部最优。我们在合成数据集、Amazon Reviews基准测试和大规模专有商业数据上评估了这些方法,并将结果与模拟退火基线进行比较。结果表明,双聚类始终获得最高的活动质量、提升度和公平性得分。虽然双聚类在较小数据集上运行高效,但在非常大的数据集上其运行时间显著增加,而基于老虎机的方法则提供了可扩展的替代方案。

英文摘要

When running marketing campaigns, retailers must decide which products to promote and which users to target. These decisions are inherently coupled: effective campaigns match users and items with strong mutual affinity into non-overlapping groups of predefined sizes. However, existing approaches assume predefined campaign structure or decouple item selection from user assignment, and cannot discover campaign groupings directly from joint interaction patterns. We therefore formalize this campaign problem as auto-targeting: jointly selecting users and items to construct multiple disjoint campaigns. To solve this combinatorial problem, we propose three complementary strategies: (i) constrained spectral biclustering to find dense regions in the user-item affinity matrix, (ii) greedy local search with pairwise swaps for combinatorial refinement, and (iii) a multi-armed bandit framework to escape local optima through exploration. We evaluate these methods on a synthetic dataset, the Amazon Reviews benchmarks, and large-scale proprietary commercial data, and compare the results to simulated annealing as a baseline. The results show that biclustering consistently achieves the highest campaign quality, lift, and fairness scores. While biclustering runs efficiently on smaller datasets, its runtime increases substantially on very large ones, where bandit-based methods instead offer a scalable alternative.

2606.09620 2026-06-09 cs.RO cs.SY eess.SY 新提交

Motion planning for hundreds of floating robots

数百个浮动机器人的运动规划

Jan Kamm, Antonio Terpin, Raffaello D'Andrea, Aswin Ramachandran

发表机构 * Institute for Dynamic Systems and Control, ETH Zürich(苏黎世联邦理工学院动态系统与控制研究所)

AI总结 针对大型浮动机器人编队的无碰撞运动规划问题,提出一种可扩展的流水线方法,通过碰撞图分解为独立子问题并行求解,在500个机器人仿真和实际演示中验证了有效性。

详情
AI中文摘要

为大型机器人编队规划无碰撞运动是困难的,因为碰撞避免引入了随团队规模快速增长且强烈的智能体间耦合。我们考虑水面上的全向浮动机器人,其编队动作由稀疏关键帧指定,交互工具必须在几秒内生成轨迹,即使过渡跨越几分钟和数千个时间步。我们提出一种可扩展的流水线,从初始化构建碰撞图,将耦合问题分解为交互簇,并独立(并行)求解这些簇,同时针对常见分解病态问题提供鲁棒性机制。我们在多达500个机器人的仿真中验证了该方法。合成的轨迹还已在两个实际演示中部署:在苏黎世湖上使用24艘Way of Water船只,以及在2025年威尼斯双年展的“时间空间存在”展览中。

英文摘要

Planning collision-free motion for large robot fleets is difficult because collision avoidance induces strong inter-agent coupling that grows rapidly with team size. We consider omnidirectional floating robots on water, where choreographies are specified by sparse keyframes and an interactive tool must generate trajectories within seconds, even when transitions span minutes and thousands of time steps. We propose a scalable pipeline that builds a collision graph from an initialization, decomposes the coupled problem into interaction clusters, and solves clusters independently (and in parallel) with robustness mechanisms for common decomposition pathologies. We validate the approach in simulations up to 500 robots. The synthesized trajectories have also been deployed in two real-world demonstrations, on Lake Zürich with a fleet of 24 Way of Water crafts and at the Time Space Existence 2025 Venice Biennale.

2606.09615 2026-06-09 cs.RO cs.CV 新提交

DexPIE: Stable Dexterous Policy Improvement from Real-World Experience

DexPIE:基于真实世界经验的稳定灵巧策略改进

Ruizhe Liao, Wenrui Chen, Liangji Zeng, Haoran Lin, Fan Yang, Kailun Yang, Yaonan Wang

发表机构 * Hunan University(湖南大学)

AI总结 提出DexPIE后训练框架,通过灵巧手适配干预系统、多阶段DAgger数据收集、相对动作空间异步推理和连续最优性指标条件化,在三个真实灵巧操作任务上成功率提升37%。

Comments Project website: https://siiuuuuuu.github.io/DexPIE

详情
AI中文摘要

灵巧操作因其高维动作空间和复杂的接触动力学,给模仿学习带来了巨大挑战。纯粹从演示中训练的策略在部署时常常遭受复合误差,并且需要大量专家数据才能达到可靠性能。为了超越演示数据的局限性,本文提出DexPIE,一个通过真实世界部署收集的经验来改进灵巧策略的后训练框架。首先,DexPIE通过灵巧手适配的干预系统和跨初始与中间任务阶段的多阶段DAgger式数据收集,实现了有效的探索覆盖,为准确的策略评估提供了可靠的监督。为了减少后训练 rollout 与演示数据之间的时间噪声,我们引入了相对动作空间中的异步推理,这能更好地将 rollout 数据与演示行为对齐,并允许评论家学习由更一致的基础策略诱导的值函数。最后,DexPIE通过对连续最优性指标进行条件化来改进策略,使策略能够以更细粒度的方式利用数据质量。在三个具有挑战性的真实世界灵巧操作任务中,DexPIE相比基于演示的参考策略实现了37%的成功率提升,优于所有基线方法,并展现出更强的鲁棒性。源代码和数据集将公开提供。

英文摘要

Dexterous manipulation presents substantial challenges for imitation learning due to its high-dimensional action space and complex contact-rich dynamics. Policies trained purely from demonstrations often suffer from compounding errors during deployment and require large amounts of expert data to achieve reliable performance. To move beyond the limitations of demonstration data, in this work, we propose DexPIE, a post-training framework for dexterous policy improvement from experience collected through real-world deployment. First, DexPIE enables effective exploration coverage through a dexterous-hand-adapted intervention system and multi-stage DAgger-style data collection across initial and intermediate task stages, providing reliable supervision for accurate policy evaluation. To reduce temporal noise between post-training rollouts and demonstration data, we introduce asynchronous inference in the relative action space, which better aligns rollout data with demonstrated behavior and allows the critic to learn a value function induced by a more consistent underlying policy. Finally, DexPIE improves the policy through conditioning on a continuous optimality indicator, allowing the policy to leverage the quality of data in a more fine-grained manner. Across three challenging real-world dexterous manipulation tasks, DexPIE achieves a 37% improvement in success rate over the demonstration-based reference policy, outperforming all baseline methods and demonstrating stronger robustness. The source code and dataset will be made publicly available.

2606.09613 2026-06-09 cs.CL cs.AI 新提交

AGENTSERVESIM: A Hardware-aware Simulator for Multi-Turn LLM Agent Serving

AGENTSERVESIM:面向多轮LLM智能体服务的硬件感知模拟器

Rakibul Hasan Rajib, Mengxin Zheng, Qian Lou

发表机构 * University of Central Florida(中佛罗里达大学)

AI总结 提出AGENTSERVESIM模拟器,通过程序编排器、工具模拟器、会话感知路由器和KV驻留模型等模块,在程序粒度上评估多轮LLM智能体服务策略,在CPU上以6%误差复现真实系统行为。

Comments Preprint

详情
AI中文摘要

多轮LLM智能体将模型调用与外部工具调用交织在一起,将服务从无状态请求处理转变为有状态程序执行。处理这些工作负载需要利用程序级上下文的调度、KV缓存管理和路由策略,包括轮次依赖、工具引入的间隙和可重用的KV状态。直接在真实系统上评估此类策略成本高昂,因为每个设计点可能需要跨到达率、模型规模、服务实例数量和内存层次结构的专用加速器时间。模拟提供了一种可扩展的替代方案,但现有的LLM服务模拟器针对无状态请求级工作负载,因此忽略了智能体服务的核心动态:多轮程序执行、跨轮缓存局部性以及工具间隙期间的KV缓存驻留。我们提出了AGENTSERVESIM,一种面向多轮LLM智能体服务的硬件感知模拟器。AGENTSERVESIM通过可组合模块在程序粒度上评估服务策略:程序编排器保留程序标识和轮次顺序,工具模拟器实现工具引入的间隙,会话感知路由器维护程序到实例的亲和性以实现缓存感知调度,KV驻留模型跟踪策略定义的跨HBM、主机DRAM/CXL和驱逐的KV放置。在真实服务部署和硬件配置上,AGENTSERVESIM在关键性能指标上的误差在6%以内,且完全在普通CPU上运行。这些结果表明,AGENTSERVESIM能够在不需在昂贵加速器上全面部署的情况下,实现受控、可重复的智能体服务策略探索。

英文摘要

Multi-turn LLM agents interleave model calls with external tool invocations, shifting serving from stateless request processing to stateful program execution. Serving these workloads requires scheduling, KV-cache management, and routing policies that use program-level context, including turn dependencies, tool-induced gaps, and reusable KV state. Evaluating such policies directly on real systems is costly, since each design point may require dedicated accelerator time across arrival rates, model scales, serving-instance counts, and memory hierarchies. Simulation offers a scalable alternative, but existing LLM serving simulators target stateless request-level workloads and therefore omit the core dynamics of agent serving: multi-turn program execution, cross-turn cache locality, and KV-cache residency during tool gaps. We present AGENTSERVESIM, a hardware-aware simulator for multi-turn LLM agent serving. AGENTSERVESIM evaluates serving policies at program granularity through composable modules: a Program Orchestrator preserves program identity and turn order, a Tool Simulator materializes tool-induced gaps, a Session-Aware Router maintains program-to-instance affinity for cache-aware dispatch, and a KV Residency Model tracks policy-defined KV placement across HBM, host DRAM/CXL, and eviction. Across real serving deployments and hardware configurations, AGENTSERVESIM reproduces real-system behavior within 6% error across key performance metrics while running entirely on commodity CPUs. These results show that AGENTSERVESIM enables controlled, repeatable exploration of agent-serving policies without requiring exhaustive deployment on costly accelerators.

2606.09610 2026-06-09 cs.RO cs.AI 新提交

Shape Formation for the Cooperative Transportation of Arbitrary Objects Using Multi-Agent Reinforcement Learning

基于多智能体强化学习的任意物体协同运输中的形状形成

Mohamed Sayed, Wolfram Burgard, Tanja Katharina Kaiser

发表机构 * University of Technology Nuremberg(纽伦堡工业大学)

AI总结 提出一种多智能体强化学习方法,使多机器人系统自主形成支撑任意形状和非均匀质量分布物体的编队,同时避免障碍物,实现可靠且泛化的协同运输。

详情
AI中文摘要

协同物体运输在众多领域(包括工业到家庭服务)中至关重要。一种流行的运输策略是将物体承载在多机器人系统之上。相应的任务通常通过将其分解为三个相互关联的子问题来解决:编队控制、协同导航和碰撞避免。现实世界物体带来的一个特殊挑战是其可能具有任意形状和非均匀质量分布,这需要机器人编队能够牢固支撑物体。在这项工作中,我们通过提出一种新颖的多智能体强化学习方法来解决运输此类现实世界物体时的模式形成控制挑战。我们的方法使多机器人系统能够自主定位在物体下方以支撑其重量,同时在编队过程中避免障碍物。我们在不同环境和不同数量机器人下的评估表明,我们的方法能够产生可靠形成平衡编队的策略,并泛化到杂乱场景以及具有复杂几何形状和非均匀质量分布的物体。

英文摘要

Cooperative object transportation is essential in numerous domains, including industrial to domestic services. A popular transportation strategy is to carry objects on top of multi-robot systems. The corresponding task is typically solved by decomposing it into three interconnected subproblems: formation control, cooperative navigation, and collision avoidance. A particular challenge posed by real-world objects is their potentially arbitrary shape and non-uniform mass distribution, necessitating robot formations that securely support the object. In this work, we address the challenge of pattern formation control for transporting such real-world objects by proposing a novel multi-agent reinforcement learning approach. Our approach enables a multi-robot system to autonomously position itself underneath an object to support its weight while avoiding obstacles during the formation process. Our evaluations with diverse environments and varying numbers of robots show that our approach leads to policies that reliably produce balanced formations and generalize to cluttered scenes and objects with complex geometry and non-uniform mass distribution.

2606.09608 2026-06-09 cs.CV 新提交

TUDSR: Twice Upsampling-Diffusion for Higher Super-Resolution

TUDSR: 用于更高超分辨率的两次上采样扩散

Zhiqiang Wu, Yitong Dong, Xian Wei

发表机构 * East China Normal University(华东师范大学) Zhejiang University(浙江大学)

AI总结 提出TUDSR框架,通过两阶段训练(R分辨率和NR分辨率)结合循环分块策略,在SD2.1基础上实现1024²和2048²高分辨率图像超分辨率,显著优于现有方法。

详情
AI中文摘要

基于扩散的生成模型在真实世界图像超分辨率(SR)中取得了显著成功。通过分块扩散技术,这些模型可以生成超出其原生支持分辨率的高分辨率图像。然而,这种高分辨率(例如2048²)输出的质量通常仍然极差,主要归因于我们考虑的两个因素:图像上采样比率(例如×8)超过模型原生支持的上采样比率(例如×4),以及模型的原生支持分辨率。在实践中,训练原生高分辨率模型需要更大的架构,这会导致显著的计算开销和GPU内存成本,使其在资源有限的设备上难以实现。因此,我们提出了TUDSR,一种用于更高超分辨率的两次上采样扩散框架。TUDSR框架主要包括两个阶段:第一阶段在R分辨率下训练,第二阶段引入基于循环分块的训练策略在NR分辨率下训练。每个阶段采用包含生成器和判别器的单步GAN架构。基于SD2.1-base,我们开发了TUDSR-S,在多个基准测试中取得了最先进的性能。大量实验进一步表明,TUDSR-S在1024²甚至2048²分辨率下生成高质量图像,显著优于现有方法。代码可在https://github.com/wuer5/TUDSR获取。

英文摘要

Diffusion-based generative models have achieved remarkable success in real-world image super-resolution (SR). With tiled diffusion techniques, these models can produce high-resolution images that exceed their native-supported resolution. However, the quality of such high-resolution (e.g $2048^2$) outputs often remains extremely poor, primarily due to two factors we consider: the image upsampling ratio (e.g $\times8$) exceeding the model's native-supported upsampling ratio (e.g $\times4$), and the model's native-supported resolution. In practice, training a native high-resolution model requires larger architectures, which incur significant computational overhead and GPU memory costs, making it hard on limited-resource equipment. Thus, we present TUDSR, a Twice Upsampling-Diffusion framework for higher SR. The TUDSR framework mainly consists of two stages: the first involves training at $R$-resolution, and the second introduces a looped chunk-based training strategy at $NR$-resolution. Each stage adapts a one-step GAN architecture comprising a generator and a discriminator. Based on SD2.1-base, we develop TUDSR-S, which achieves state-of-the-art performance across multiple benchmarks. Extensive experiments further demonstrate that TUDSR-S generates high-quality images at the resolutions of $1024^2$ and even $2048^2$, significantly outperforming existing approaches. Code is available at https://github.com/wuer5/TUDSR.

2606.09607 2026-06-09 cs.LG cs.AI 新提交

Closure-Validated Circuit Discovery in Attention Heads: Co-activation Proposes, Ablation Disposes

注意力头中的闭包验证电路发现:共激活提出,消融处置

Yongzhong Xu

发表机构 * GitHub

AI总结 通过共激活聚类提出注意力头电路假设,并用因果消融验证闭包性,发现该方法在密集模型有效但在MoE模型失效,表明共激活仅是电路提议而非确认。

Comments 22 pages, 3 figures

详情
AI中文摘要

可解释性越来越将组件组(而非单个单元)作为基本对象,并提议通过聚类共激活统计来发现它们。我们询问这种廉价信号是否真正识别出注意力头电路。将稀疏自编码器聚类方法适配到注意力头——但通过因果消融而非重构进行验证——我们聚类头,然后运行闭包测试:消融发现的社区,并将每个示例的损伤与匹配随机对照进行比较。在两个密集的1B规模模型(Pythia 1B, OLMo 1B)和两种输入分布上,社区通过了闭包测试。在混合专家模型(OLMoE-1B-7B)中,路由条件聚类恢复了一个统计上真实的信号,但该信号未能通过闭包测试——消融反而改善了损失,方向错误。将闭包测试扩展到训练过程中,注意力目标选择性和参与比率在双向与功能解耦。我们得出结论:廉价信号是电路提议,而非确认的电路;闭包是区分二者的关键。

英文摘要

Interpretability increasingly treats groups of components, not individual units, as the basic object, and proposes to find them by clustering co-activation statistics. We ask whether such a cheap signal actually identifies an attention-head circuit. Adapting a sparse-autoencoder clustering recipe to attention heads -- but validating by causal ablation rather than reconstruction -- we cluster heads and then run a closure test: ablate the discovered community and compare per-example damage to matched-random controls. Across two dense 1B-scale models (Pythia 1B, OLMo 1B) and two input distributions, the communities pass closure. In a Mixture-of-Experts model (OLMoE-1B-7B), route-conditional clustering recovers a statistically real signal that nonetheless does not survive closure -- ablation improves loss, the wrong direction. Extending closure across training, attention-target selectivity and participation ratio decouple from function in both directions. We conclude that a cheap signal is a circuit proposal, not a confirmed circuit; closure is what separates them.

2606.09605 2026-06-09 cs.AI 新提交

Next-Token Prediction Learns Generalisable Representations of Sleep Physiology

下一个词预测学习睡眠生理学的可泛化表示

Jonathan F. Carter, Lionel Tarassenko

发表机构 * Institute of Biomedical Engineering, University of Oxford(牛津大学生物医学工程研究所)

AI总结 提出Hypnos模型,通过下一个词预测目标,从多模态生理信号中学习可泛化表示,在睡眠阶段分类和房颤检测等任务上显著优于现有基础模型。

详情
AI中文摘要

基础模型提供了一种有前景的途径,将多模态生理信号压缩为人类健康的紧凑表示,在睡眠医学、心脏病学、神经病学及其他医疗领域具有广泛应用。现有模型通常采用掩码重建或对比学习目标进行训练。然而,掩码重建可能不适用于这些信号的随机性质,而对比方法依赖于正样本对定义,尽管生理信号的语义不变性尚不明确。在这项工作中,我们展示了下一个词预测是一种简单且可扩展的替代方案。我们开发了Hypnos,一个多模态睡眠基础模型,使用来自超过20,000次夜间多导睡眠图记录的八种不同传感模态(例如EEG、ECG、呼吸信号)进行训练。我们使用残差向量量化将每种模态标记化为离散标记流,然后训练一个大型自回归RQ-Transformer,以并行方式联合预测所有模态的下一个标记。训练后,Hypnos可应用于任何支持模态子集的连续传感器数据流,为下游任务生成嵌入。在一系列基准测试中,Hypnos显著优于现有基础模型。在睡眠阶段分类中,我们在保留测试集上匹配了强监督基线的性能,同时使用的标记数据减少了100倍。Hypnos甚至泛化到日间生理学,在检测房颤方面超越了专用的ECG基础模型。我们的结果表明,下一个词预测是从多模态生理信号进行表示学习的强自监督目标。

英文摘要

Foundation models offer a promising route to compress multi-modal physiological signals into compact representations of human health, with broad applications across sleep medicine, cardiology, neurology and other healthcare domains. Existing models have typically been trained with masked-reconstruction or contrastive objectives. However, masked reconstruction may be poorly suited to the stochastic nature of these signals, while contrastive approaches rely on positive-pair definitions despite the semantic invariances of physiological signals being poorly understood. In this work, we show that next-token prediction is a simple and scalable alternative. We develop Hypnos, a multi-modal sleep foundation model trained using eight different sensing modalities (e.g. EEG, ECG, respiratory signals) drawn from over 20,000 overnight polysomnography recordings. We tokenize each modality into streams of discrete tokens using residual vector quantization, then train a large auto-regressive RQ-Transformer to jointly predict the next token across all modalities in parallel. After training, Hypnos can be applied to continuous streams of sensor data from any subset of supported modalities, generating embeddings for downstream tasks. Across a range of benchmarks, Hypnos significantly outperforms existing foundation models. In sleep stage classification, we match the performance of strong supervised baselines on held-out test sets whilst using \(100\times\) less labelled data. Hypnos even generalises to daytime physiology, surpassing a dedicated ECG foundation model at detecting atrial fibrillation. Our results demonstrate that next-token prediction is a strong self-supervised objective for representation learning from multi-modal physiological signals.

2606.09603 2026-06-09 cs.CL 新提交

Automated IEP Generation from Traditional Chinese Parent-Teacher Interviews via Corpus-Grounded Feature Diffusion

基于语料库特征扩散的繁体中文家长会自动化个别化教育计划生成

Kuanlin Chen, Cheng-En Ou

发表机构 * National University of Singapore(新加坡国立大学) University of California, Berkeley(加州大学伯克利分校)

AI总结 针对繁体中文个别化教育计划(IEP)生成中数据稀缺和隐私限制问题,提出基于语料库特征扩散(CGFD)的低资源微调流程,通过种子选择、特征扩散和语法约束解码(GCD)生成高质量样本,并发现GCD在繁体中文下适得其反,无GCD路径在可靠性和速度上更优。

Comments 12 pages, 5 figures

详情
AI中文摘要

编写个别化教育计划(IEP)是一项高劳动强度、知识密集型的文档负担;英语研究表明,生成式AI可以显著减少起草时间,但由于领域数据稀缺、严格的隐私法规以及缺乏本地评估基准,繁体中文的自动化IEP生成几乎未被探索。我们提出了一种基于语料库特征扩散(CGFD)的低资源微调流程:(1)通过tau阈值和标志感知分数上限选择25个双专家高评分种子转录本;(2)从种子中提取特征画像(句子长度、结构、量化模板),并连同言语化采样风格的多样性控制注入LLM提示,以驱动扩散;(3)使用15个专家黄金种子作为扩散锚点,目标生成585个样本;获得567个有效扩散样本,形成582个样本的训练集,用于使用QLoRA微调Breeze-7B;(4)通过语法约束解码(GCD)在推理时强制执行分层SMART目标阶梯模式。在55个样本的模式压力集上的消融结果揭示了一个意外发现:在繁体中文令牌预算下,GCD适得其反——无GCD路径实现了100%的模式通过率,中位延迟降低34%,在可靠性和速度上均优于GCD。在n=10的正式保留集上,无GCD推理路径实现了BERTScore F1 = 0.779,超过了GPT-5.4(0.726)、DeepSeek-V3.2(0.703)、Gemini-3-Flash-Preview(0.703)和Llama-4-Maverick(0.700)的零样本基线,同时保持完全本地、气隙推理。该系统填补了繁体中文特殊教育NLP的空白,并在工业工程范式下提供了可扩展、保护隐私的本地推理解决方案。

英文摘要

Writing Individualized Education Programs (IEPs) is a high-labor, knowledge-intensive document burden; English-language research has demonstrated that generative AI can significantly reduce drafting time, yet automated IEP generation in Traditional Chinese remains virtually unexplored due to domain data scarcity, strict privacy regulations, and the absence of local evaluation benchmarks. We propose a low-resource fine-tuning pipeline centered on Corpus-Grounded Feature Diffusion (CGFD): (1) 25 dual-expert high-score seed transcripts are selected via a tau threshold with flag-aware score caps; (2) a FeatureProfile (sentence length, structure, quantification templates) is extracted from seeds and injected into LLM prompts alongside Verbalized-Sampling-style diversity control to drive diffusion; (3) 15 expert gold seeds are used as diffusion anchors, targeting 585 samples; 567 valid diffusion samples are obtained, yielding a 582-sample training set used to fine-tune Breeze-7B with QLoRA; (4) schema-constrained inference via Grammar-Constrained Decoding (GCD) enforces a hierarchical SMART Goal Ladder schema at inference time. Ablation results on a 55-sample schema stress set reveal an unexpected finding: GCD is counterproductive under Traditional Chinese token budgets -- the no-GCD path achieves 100% schema pass rate at 34% lower median latency, outperforming GCD on both reliability and speed. On the n=10 formal hold-out, the no-GCD inference path achieves BERTScore F1 = 0.779, exceeding GPT-5.4 (0.726), DeepSeek-V3.2 (0.703), Gemini-3-Flash-Preview (0.703), and Llama-4-Maverick (0.700) zero-shot baselines while maintaining fully local, air-gapped inference. This system addresses a gap in Traditional Chinese special-education NLP and offers a scalable, privacy-preserving local inference solution under an industrial engineering paradigm.

2606.09590 2026-06-09 cs.CL cs.CR 新提交

Clinically Grounded Privacy Evaluation of Medical LMs

临床导向的医学语言模型隐私评估

Sasha Ronaghi, Sana Tonekaboni, Lena Stempfle, Vivian Utti, Jordan Li Cahoon, Nathaniel Hendrix, Ayin Vala, Marzyeh Ghassemi, Emily Alsentzer

发表机构 * Stanford University(斯坦福大学) Massachusetts Institute of Technology(麻省理工学院) American Board of Family Medicine(家庭医学认证委员会)

AI总结 提出临床导向框架,按对抗访问等级评估医学语言模型隐私泄露,发现常规元数据可导致高比率逐字记忆和敏感诊断恢复,但部分记忆源于模板化文档。

详情
AI中文摘要

医学语言模型(LMs)可以记忆和重现受保护的健康信息,但隐私评估通常关注训练文本的恢复,而非在现实威胁模型下的泄露。我们引入了一个临床导向的框架,沿着对抗访问的分级轴评估泄露,范围从公开可推断的人口统计信息到泄露的笔记片段。在每个层级,我们测量患者特定文本的逐字记忆和敏感诊断的语义泄露。将该框架应用于一个在378k临床笔记上预训练的LM,我们发现常规就诊元数据(即姓名、出生日期、提供者、诊所、就诊日期)在患者时间线上引发高比率的逐字记忆和敏感诊断恢复(流产AUROC 0.91,HIV 0.81)。同时,精确匹配记忆可能夸大泄露:36%的记忆令牌反映了模板化文档。我们的工作强调了在纵向临床数据上训练的风险,为医学LM的上下文隐私评估提供了一个实用框架。

英文摘要

Medical language models (LMs) can memorize and reproduce protected health information, but privacy evaluations often focus on recovery of training text rather than disclosure under realistic threat models. We introduce a clinically grounded framework that evaluates leakage along a graded axis of adversarial access, ranging from publicly inferable demographics to leaked note fragments. At each tier, we measure verbatim memorization of patient-specific text and semantic leakage of sensitive diagnoses. Applying the framework to an LM pretrained on 378k clinical notes, we find that routine encounter metadata (i.e. name, date of birth, provider, practice, visit date) elicits high rates of verbatim memorization across a patient's timeline and sensitive-diagnosis recovery (AUROC 0.91 for abortion, 0.81 for HIV). At the same time, exact-match memorization can overstate disclosure: 36% of memorized tokens reflect templated documentation. Our work highlights the risks of training on longitudinal clinical data, providing a practical framework for contextual privacy evaluation of medical LMs.

2606.09585 2026-06-09 cs.AI 新提交

Optical Reasoning: Rethinking Images as an Expressive Reasoning Medium Beyond Text

光学推理:重新思考图像作为超越文本的表达性推理媒介

Yutong Bian, Dongjie Cheng, Heming Xia, Yongqi Li, Wenjie Li

发表机构 * The Hong Kong Polytechnic University(香港理工大学)

AI总结 提出光学推理概念,将图像作为独立推理媒介,通过排版和图形两种变体实现,在语言和多模态任务中匹配或超越文本推理,同时减少推理令牌。

详情
AI中文摘要

思维链(CoT)提升了大型语言模型(LLMs)的性能,并已扩展到多模态大型语言模型(MLLMs)。最近的工作进一步从基于文本的多模态推理转向交错模态推理,其中中间步骤可以同时包含文本理由和视觉证据。在这项工作中,我们提出了一个更大胆、更雄心勃勃的想法:图像能否单独作为语言和多模态任务的推理媒介?为了探索这一点,我们提出了光学推理,它将图像视为独立的推理媒介。我们通过两种变体实例化这一概念:基于排版的光学推理,优化视觉布局以实现紧凑的理由渲染;以及基于图形的光学推理,将文本和图形元素组合成结构化的视觉理由。在数学、科学和交错模态推理基准测试中,光学推理可以匹配甚至超越传统的文本推理,同时在语言任务上平均减少28.57%的推理令牌,在多模态任务上减少16%,实现文本推理1.96倍的令牌效率。这些结果表明,图像可以有效且高效地编码理由,同时为推理提供统一的视觉画布。

英文摘要

Chain-of-Thought (CoT) improves the performance of Large Language Models (LLMs) and has been extended to Multimodal Large Language Models (MLLMs). More recent work further moves from text-based multimodal reasoning toward interleaved-modal reasoning, where intermediate steps can incorporate both textual rationales and visual evidence. In this work, we propose a bolder and more ambitious idea: could images alone serve as the reasoning medium for both language and multimodal tasks? To explore this, we propose optical reasoning, which treats images as a standalone reasoning medium. We instantiate this concept with two variants: typographic-based optical reasoning, which optimizes visual layouts for compact rationale rendering, and graphical-based optical reasoning, which composes text and graphical elements into structured visual rationales. Across mathematical, scientific, and interleaved-modal reasoning benchmarks, optical reasoning can match or even exceed traditional text reasoning while reducing reasoning tokens by an average of 28.57% on language tasks and 16% on multimodal tasks, achieving 1.96 times the token efficiency of text reasoning. These results show that images can effectively and efficiently encode rationales while providing a unified visual canvas for reasoning.

2606.09582 2026-06-09 cs.LG stat.ML 新提交

On Choosing the $μ$ Parameter in Gaussian Differential Privacy

论高斯差分隐私中参数 $μ$ 的选择

Bogdan Kulynych, Antti Honkela

发表机构 * Lausanne University Hospital(拉索恩大学医院) University of Helsinki(赫尔辛基大学)

AI总结 本文通过匹配强对手成员推理攻击的最坏情况成功度,提供从纯-DP ε到GDP μ的原则性映射,并推荐 μ≈ε/5 作为保守通用转换。

详情
AI中文摘要

近期工作主张使用高斯差分隐私(GDP)来报告隐私保护机器学习中的隐私保证。我们通过匹配强对手成员推理攻击在最坏情况下的成功度,基于三个指标提供了从纯-DP ε到GDP μ的原则性映射:固定FPR下的乘法优势、固定召回率下的精确度以及标准隐私轮廓。我们在有用参数范围内列出了μ值,并推荐μ≈ε/5作为保守的通用转换。

英文摘要

Recent work argues for using Gaussian differential privacy (GDP) to report the privacy guarantees in privacy-preserving machine learning. We provide principled mappings from pure-DP $\varepsilon$ to GDP $μ$ by matching the worst-case success of a strong-adversary membership inference attack in terms of three metrics: multiplicative advantage at fixed FPR, precision at fixed recall, and the standard privacy profile. We tabulate $μ$ values across a useful range of parameters and recommend $μ\approx \varepsilon/5$ as a conservative general-purpose conversion.

2606.09578 2026-06-09 cs.AI cs.CL cs.IR 新提交

TABVERSE: Benchmarking Cross-Format Table Understanding in LLMs and VLMs

TABVERSE:大语言模型与视觉语言模型中跨格式表格理解的基准测试

Momina Ahsan, Sarfraz Ahmad, Ming Shan Hee, Roy Ka-Wei Lee, Preslav Nakov

发表机构 * Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)(穆罕默德·本·扎耶德人工智能大学) Singapore University of Technology and Design (SUTD)(新加坡科技设计大学)

AI总结 提出TABVERSE基准,通过控制表格内容、跨多种结构格式(HTML、Markdown、LaTeX)和渲染图像,系统评估LLM和VLM在问答、结构理解和结构重建任务中的表现,发现表示格式显著影响表格理解能力。

Comments 24 pages, 18 tables, 16 figures, Submitted to ARR May 2026

详情
AI中文摘要

大语言模型(LLMs)和视觉语言模型(VLMs)在表格推理任务上的评估日益增多,但表格表示的作用仍未充分探索。实践中,相同的表格内容可能以不同的结构格式出现,如HTML、Markdown和LaTeX,或作为渲染图像。然而,现有评估往往让内容、格式、布局和模态同时变化,使得难以隔离表示效应。我们引入了TABVERSE,一个受控的多模态表格基准,它在多个结构格式和渲染图像中对齐相同的表格内容,并带有问题类别和难度标签。这种设计使得在保持表格内容固定的同时,能够系统评估表示效应。我们在三个任务上评估LLMs和VLMs:问答(QA)、结构理解能力(SUC)和结构重建(SR)。我们的结果表明,表示选择显著影响表格理解。模型在结构化文本上的表现通常优于渲染图像,但这一差距的大小取决于任务、模型和格式。HTML通常是最稳健的文本格式,而行敏感的结构任务和语法可用的LaTeX重建仍然具有挑战性。这些发现表明,表格表示是可靠表格评估的关键因素。

英文摘要

Large Language Models (LLMs) and Vision-Language Models (VLMs) are increasingly evaluated on table reasoning tasks, but the role of table representation remains under-explored. In practice, the same table content may appear in different structural formats, such as HTML, Markdown, and LaTeX, or as rendered images. However, existing evaluations often let content, format, layout, and modality vary together, making it difficult to isolate representation effects. We introduce TABVERSE, a controlled multimodal table benchmark that aligns the same table content across multiple structural formats and rendered images, with question category and difficulty tags. This design enables systematic evaluation of representation effects while holding table content fixed. We evaluate LLMs and VLMs across three tasks: Question Answering (QA), Structural Understanding Capability (SUC), and Structure Reconstruction (SR). Our results show that representation choice substantially affects table understanding. Models generally perform better with structured text than with rendered images, but the size of this gap depends on the task, model, and format. HTML is often the most robust text format, while row-sensitive structural tasks and syntactically usable LaTeX reconstruction remain challenging. These findings show that table representation is a key factor in reliable table evaluation.

2606.09577 2026-06-09 cs.CL cs.LG cs.SE 新提交

Code Is More Than Text: Uncertainty Estimation for Code Generation

代码不仅仅是文本:代码生成的不确定性估计

Yuling Shi, Caiqi Zhang, Yuexian Li, Haopeng Wang, Yeheng Chen, Nigel Collier, Xiaodong Gu

发表机构 * Shanghai Jiao Tong University(上海交通大学) University of Cambridge(剑桥大学)

AI总结 针对代码生成中错误程序的可靠性问题,提出基于词法、算法和功能三个正交轴的不确定性估计方法,在五个代码LLM上将AUROC提升8.1个百分点。

详情
AI中文摘要

大型语言模型(LLMs)越来越多地被部署为代码生成器,其中静默错误的程序会带来真实的安全和可靠性风险。可靠的不确定性估计(UE)对于选择性预测、人在回路审查和下游智能体决策至关重要。然而,现有的大多数代码UE方法继承自自然语言(NL)生成,忽略了使代码独特的属性。我们认为代码在三个方面与NL不同:单个错误标记可能破坏整个程序(标记脆弱性);算法意图和具体实现可能独立不一致(意图-代码差距);程序可以被执行(可执行性)。我们将这些属性实例化为三个正交的不确定性轴:词法(Top-K标记熵)、算法(伪代码一致性)和功能(行为一致性)。在五个代码LLM上,我们的三轴集成将平均AUROC从最强NL衍生基线的0.696提高到0.776(+8.1点)。值得注意的是,在Qwen3-14B上,我们的单次Top-K标记熵匹配了最强多次基线,同时成本降低超过3倍;在各模型上,它仍然是一个有竞争力的低成本信号。这些结果表明,代码UE需要特定于代码的设计,而不是直接移植NL方法。

英文摘要

Large language models (LLMs) are increasingly deployed as code generators, where silently wrong programs pose real safety and reliability risks. Reliable uncertainty estimation (UE) is essential for selective prediction, human-in-the-loop review, and downstream agentic decisions. Yet most existing code UE methods are inherited from natural language (NL) generation and ignore properties that make code distinct. We argue that code differs from NL in three ways: a single wrong token can break an entire program (token fragility); algorithmic intent and concrete implementation can disagree independently (intent-code gap); and programs can be executed (executability). We instantiate these properties as three orthogonal uncertainty axes: lexical (Top-K token entropy), algorithmic (pseudo-code consistency), and functional (behavioral consistency). Across five code LLMs, our three-axis ensemble improves average AUROC from 0.696 for the strongest NL-derived baseline to 0.776 (+8.1 points). Notably, on Qwen3-14B, our single-pass Top-K token entropy matches the strongest multi-pass baseline while being over 3x cheaper; across models, it remains a competitive low-cost signal. These results suggest that code UE deserves code-specific design rather than direct NL ports.

2606.09572 2026-06-09 cs.RO cs.AI 新提交

CT-VAM: A Cerebello-Thalamic-Inspired Vision-Action Model for Efficient Visuomotor Control

CT-VAM: 一种小脑-丘脑启发的视觉-动作模型用于高效视觉运动控制

Jiacheng Li, Yize Guo, Jiabin Guo, Qingchen Liu, Jiahu Qin

发表机构 * University of Science and Technology of China(中国科学技术大学) AIRLab, Department of Automation(自动化系AIRLab)

AI总结 提出CT-VAM模型,通过TARS条件注意力解码器融合异构输入,以68M参数实现与大型VLA模型相当的LIBERO成功率,并降低推理延迟,支持高频控制。

详情
AI中文摘要

视觉-语言-动作模型在机器人操作中展现出强大潜力,然而原始语言主要用于指定任务意图,而非在高频低层执行过程中反复处理。受此分离的启发,我们提出了一种小脑-丘脑启发的视觉-动作模型(CT-VAM),用于高效的任务条件视觉运动控制。CT-VAM作为一个紧凑的局部执行策略,从双视角视觉观察、本体感觉和轻量级任务条件中预测动作块,从而可能实现一种实用的云-边缘范式,其中高层语义推理由大模型处理,而快速闭环控制在本地硬件上运行。为了有效融合异构输入,CT-VAM引入了TARS(丘脑动作路由流),一种流分离的条件注意力解码器,独立路由动作、视觉和任务流,防止密集的感官标记淹没紧凑的任务相关条件。仅凭68M参数,CT-VAM在LIBERO上取得了与更大规模VLA模型竞争的成功率,同时降低了推理延迟。结合用于异步块执行的流一致修补,CT-VAM支持高频控制,并在资源受限的机器人平台上展示了鲁棒的实时部署能力。

英文摘要

Vision-language-action models have shown strong promise for robot manipulation, yet raw language is primarily needed to specify task intent rather than to be repeatedly processed during high-frequency low-level execution. Motivated by this separation, we propose a cerebello-thalamic-inspired vision-action model (CT-VAM) for efficient task-conditioned visuomotor control. CT-VAM acts as a compact local execution policy that predicts action chunks from dualview visual observations, proprioception, and a lightweight task condition, potentially enabling a practical cloud-edge paradigm in which high-level semantic reasoning can be handled by large models while fast closed-loop control runs on local hardware. To fuse heterogeneous inputs effectively, CT-VAM introduces TARS (Thalamic Action Routing Stream), a stream-separated conditional attention decoder that independently routes action, visual and task streams, preventing dense sensory tokens from overwhelming compact task-relevant conditions. With only 68M parameters, CT-VAM achieves LIBERO success rates competitive with substantially larger VLA models, while reducing inference latency. Together with flow-consistent inpainting for asynchronous chunk execution, CT-VAM supports high-frequency control and demonstrates robust realworld deployment on resource-constrained robotic platforms.

2606.09569 2026-06-09 cs.RO cs.CV 新提交

Efficient Minimal Solvers for Relative Pose Estimation in Autonomous Driving Applications

自动驾驶应用中相对位姿估计的高效最小求解器

Tao Li, Liang Liu, Jianli Han, Weimin Lv

发表机构 * College of Aerospace Science and Engineering, Naval Aviation University(海军航空大学航空航天科学与工程学院)

AI总结 提出基于新平移参数化和一阶旋转近似的统一框架,设计三种最小求解器(利用IMU垂直方向、转向旋转轴方向、平面运动假设),减少点对应数量和代数复杂度,在RANSAC中加速假设生成,平衡速度与精度。

详情
AI中文摘要

随着视觉传感系统的进步,计算机视觉在自动驾驶和机器人导航中扮演着越来越重要的角色。多相机系统中的相对位姿估计对于精确的车辆定位和环境感知至关重要,要求高实时性和鲁棒性。然而,现有方法通常涉及高计算成本并严重依赖丰富的特征匹配,限制了它们在时间敏感驾驶场景中的适用性。为解决这些限制,本文引入了一个基于新颖平移参数化和一阶旋转近似的统一框架,用于高效相对位姿估计。在该框架内,我们提出了三种专门为自动驾驶车辆设计的高效最小求解器。第一个求解器集成了惯性测量单元(IMU)的垂直方向先验,第二个在转向操作期间利用旋转轴方向先验,第三个专为平面运动设计——这是结构化道路上地面车辆的现实假设。通过减少最小点对应数量和代数复杂度,我们的方法能够在基于RANSAC的流程中更快地生成假设,提高对实时系统的适用性。在合成数据集和KITTI自动驾驶基准上的大量实验表明,与现有最先进算法相比,所提出的求解器在速度和精度之间实现了有利的平衡。

英文摘要

With the advancement of visual sensing systems, computer vision is playing an increasingly important role in autonomous driving and robot navigation. Relative pose estimation in multi-camera systems is essential for accurate vehicle localization and environment perception, demanding high real-time performance and robustness. Existing methods, however, often involve high computational costs and rely heavily on abundant feature matches, limiting their applicability in time-sensitive driving scenarios. To address these limitations, this paper introduces a unified framework for efficient relative pose estimation, built upon a novel translation parameterization and first-order rotation approximation. Within this framework, we propose three efficient minimal solvers specifically designed for autonomous vehicles. The first solver integrates the vertical direction prior from Inertial Measurement Units (IMUs), the second utilizes the rotation axis direction prior during steering maneuvers, and the third is designed for planar motion - a realistic assumption for ground vehicles operating on structured roads. By reducing both the minimal number of point correspondences and the algebraic complexity, our methods enable faster hypothesis generation within RANSAC-based pipelines, improving suitability for real-time systems. Extensive experiments on synthetic datasets and the KITTI autonomous driving benchmark demonstrate that the proposed solvers achieve a favorable balance between speed and accuracy compared to existing state-of-the-art algorithms.

2606.09568 2026-06-09 cs.AI 新提交

Self-Explainability in Self-Adaptive and Self-Organising Systems: Status and Research Directions

自适应与自组织系统中的自解释性:现状与研究方向

Tom Beyer, Svea Wisy, Sven Tomforde

发表机构 * Kiel University(基尔大学)

AI总结 本文通过系统文献综述,定义自解释性(SX)并建立分类法,提出自解释性层次框架,发现多数方法仍处于概念阶段,缺乏评估标准。

Comments Under review as a regular paper at ACM Transactions on Autonomous and Adaptive Systems (TAAS)

详情
AI中文摘要

随着人工智能(AI)的进步,自适应和自组织系统的复杂性日益增加,使其越来越难以理解和信任。虽然可解释AI旨在提供对AI决策的洞察,但更高级的目标是让系统自我解释——这种能力称为自解释性(SX)。本文对SX进行了系统文献综述,分析了现有方法,包括其领域、目标和评估方法。综述提出了SX的统一定义和分类法,并引入了自解释性层次,为定位当前和未来研究提供了框架。我们的结果表明,大多数SX方法仍处于概念阶段,实际实现很少。此外,目前没有评估SX的正式或事实标准,突出了一个主要研究空白。因此,这项工作为推进复杂系统中的自解释性奠定了基础和路线图。

英文摘要

The growing complexity of self-adaptive and self-organising systems, fuelled by advances in Artificial Intelligence (AI), has made them increasingly difficult to understand and trust. While Explainable AI aims to provide insight into AI decision-making, a more advanced goal is for systems to explain themselves - an ability referred to as Self-Explainability (SX). This article presents a systematic literature review on SX, analysing existing approaches, including their domains, targets, and evaluation methods. The review develops a unified definition and taxonomy of SX and introduces Levels of Self-Explainability, providing a framework for positioning current and future research. Our results show that most SX approaches remain conceptual, with few practical implementations. Moreover, there is currently no formal or de facto standard for evaluating SX, highlighting a major research gap. This work thus establishes a foundation and roadmap for advancing Self-Explainability in complex systems.

2606.09563 2026-06-09 cs.AI cs.LG 新提交

PRISM: Recovering Instruction Sets from Language Model Activations

PRISM:从语言模型激活中恢复指令集

Gilad Gressel, Rahul Pankajakshan, Julia Diament, Efim Hudis, Krishnashree Achuthan, Yisroel Mirsky

发表机构 * Center for Cybersecurity Systems & Networks, Amrita Vishwa Vidyapeetham(阿姆里塔·维什瓦·维迪亚佩瑟姆网络安全系统与网络中心) Microsoft(微软) Ben-Gurion University of the Negev(内盖夫本-古里安大学)

AI总结 提出PRISM方法,通过激活条件解码从冻结目标模型隐藏状态中恢复活跃指令集,利用法官引导的GRPO优化,在多种场景下优于基线方法。

Comments Under Review

详情
AI中文摘要

随着LLM被部署为智能体,可靠的监控不仅需要知道它们输出了什么,还需要知道哪些指令在引导它们的行为。当模型推断出非预期的子目标、遵循上下文线索或受到提示注入和隐藏目标的影响时,这变得困难。虽然激活到语言的方法表明隐藏状态可以揭示自然语言信息,但现有方法并非设计用于恢复智能体设置中同时活跃的完整指令、约束、禁止和子目标集。我们将此问题形式化为指令集检索,并引入PRISM,一个激活条件的解释器,将冻结目标模型的隐藏状态解码为活跃指令的忠实项目符号列表。与先前的激活到语言方法不同,PRISM直接训练以恢复指令集,使用法官引导的GRPO来奖励覆盖的指令并惩罚不支持的指令。在良性、受限、提示注入和隐藏目标设置中,PRISM优于激活到语言基线,特别是在安全相关目标上。

英文摘要

As LLMs are deployed as agents, reliable monitoring requires knowing not only what they output, but which instructions are steering their behavior. This is difficult when models infer unintended subgoals, follow contextual cues, or are influenced by prompt injections and hidden objectives. While activation-to-language methods suggest that hidden states can reveal natural-language information, existing approaches are not designed to recover the full set of simultaneous instructions, constraints, prohibitions, and subgoals active in agentic settings. We formalize this problem as instruction set retrieval and introduce PRISM, an activation-conditioned interpreter that decodes hidden states from a frozen target model into a faithful bullet list of active instructions. Unlike prior activation-to-language methods, PRISM is trained to recover instruction sets directly, using judge-guided GRPO to reward covered instructions and penalize unsupported ones. Across benign, constrained, prompt-injection, and hidden-objective settings, PRISM outperforms activation-to-language baselines, especially on security-relevant objectives.

2606.09559 2026-06-09 cs.LG cs.AI cs.CR cs.RO 新提交

Safe-RULE: Safe Reinforcement UnLEarning

Safe-RULE:安全强化反学习

Shixiong Jiang, Taozheng Zhu, Fanxin Kong

发表机构 * University of Notre Dame(圣母大学)

AI总结 针对离线安全强化学习易受数据投毒攻击的问题,提出Safe-RULE框架,通过反学习移除恶意样本影响,无需从头训练或访问原始环境,实验证明能有效提升安全性。

Comments 20 pages, 3 figures

详情
AI中文摘要

离线安全强化学习(Safe RL)使得无需在线交互即可进行策略学习,适用于机器人系统等安全关键系统。然而,其对静态数据集的依赖使离线Safe RL面临数据投毒攻击,攻击者注入恶意样本以破坏安全性并诱导不安全策略行为。在这项工作中,我们提出了一种新的学习范式,称为安全强化反学习(Safe-RULE),作为一种防御框架,用于在不从头重新训练或需要访问原始训练环境的情况下移除中毒数据的影响。我们进一步将强化反学习扩展到离线Safe RL,通过在反学习过程中明确考虑任务性能和安全约束。跨基准Safe RL任务的实验表明,我们的方法能有效增强针对数据投毒攻击的安全性能。

英文摘要

Offline safe reinforcement learning (Safe RL) enables policy learning without online interactions, making it suitable for safety-critical systems such as robotics systems. However, its reliance on static datasets exposes offline Safe RL to data poisoning attacks, where adversaries inject malicious samples that compromise safety and induce unsafe policy behavior. In this work, we propose a new learning paradigm, named safe reinforcement unlearning (Safe-RULE), used as a defense framework to remove the influence of poisoned data without retraining from scratch or requiring access to the original training environment. We further extend reinforcement unlearning to offline Safe RL by explicitly accounting for both task performance and safety constraints during the unlearning process. Experiments across benchmark Safe RL tasks demonstrate that our approach effectively enhances safety performance against data poisoning attacks.

2606.09556 2026-06-09 cs.AI 新提交

AI Scientists Are Only as Good as Their Evidence: A Stratified Ablation of Proprietary Data and Reasoning Skills in Drug-Asset Valuation

AI科学家的能力取决于其证据:药物资产估值中专有数据与推理技能的分层消融研究

Yinan Wang

发表机构 * Noah AI Research(Noah AI研究)

AI总结 通过分层消融实验,发现药物资产估值中AI科学家的决策上限由专有证据集决定,而非仅依赖推理框架;加入专有数据后决策质量显著提升。

Comments Preprint; 2 figures, 5 tables

详情
AI中文摘要

AI科学家智能体通常被评估时,仿佛能力主要取决于模型质量、提示或推理框架。我们在药物资产估值中测试了一个不同的假设:对于知识密集型的科学决策,限制因素往往是智能体能够访问的证据基础。我们在一个生产级估值智能体上进行了三臂对照消融实验:A是仅使用网络的普通LLM分析师,B增加了公共结构化工具以及14维估值剧本、验证器、客观性策略和红队,C增加了专有的Noah AI语料库,包含精选的管线、试验和交易情报。在包含13个资产的分层基准测试中,B改善了校准和审计纪律:层级内准确率从0.80提高到0.89,客观性从3.16提高到3.30。但B并未消除事实上限。在能力超集核算下,A和B仅恢复了精选黄金竞争记录的0.25和0.38,而C恢复了0.96;在精选长尾子集上,C达到0.93,而A/B为0.26/0.30。原始盲审决策质量A和B相似(7.01 vs 6.96),因此我们引入了完整性感知决策效用:知情决策质量 = 决策质量 × 黄金覆盖率。在此指标上,C达到7.43,而A/B为1.76/2.57。即使一个完美的非专有数据报告,其B的覆盖率上限也仅为3.83。结果并非推理框架不重要;它们改善了校准和纪律。相反,专有证据集设定了AI科学家所能知道并因此决策的上限。

英文摘要

AI Scientist agents are often evaluated as if capability were mainly a function of model quality, prompting, or reasoning scaffolds. We test a different hypothesis in drug-asset valuation: for knowledge-intensive scientific decisions, the limiting factor is often the evidence substrate the agent can access. We run a controlled three-arm ablation on a production valuation agent: A is a plain web-only LLM analyst, B adds public structured tools plus a 14-dimension valuation playbook, verifier, objectivity policy and red-team, and C adds the proprietary Noah AI corpus of curated pipeline, trial and deal intelligence. Across a 13-asset stratified benchmark, B improves calibration and audit discipline: tier-in-range accuracy rises from 0.80 to 0.89 and objectivity from 3.16 to 3.30. But B does not remove the factual ceiling. Under capability-superset accounting, A and B recover only 0.25 and 0.38 of the curated gold competitive record, while C recovers 0.96; on the curated long-tail subset, C reaches 0.93 vs. 0.26/0.30. Raw blind-panel decision quality is similar for A and B (7.01 vs. 6.96), so we introduce completeness-aware decision utility: informed decision-quality = decision-quality x gold-coverage. On this metric, C reaches 7.43 vs. 1.76/2.57 for A/B. Even a perfect non-proprietary-data report would be capped at 3.83 by B's coverage. The result is not that reasoning scaffolds are unimportant; they improve calibration and discipline. Rather, proprietary evidence sets the upper bound of what the AI Scientist can know and therefore decide.

2606.09547 2026-06-09 cs.CV cs.LG 新提交

Streaming Interventions: Can Video Large Language Models Correct Mistakes as They Occur?

流式干预:视频大语言模型能否在错误发生时即时纠正?

Apratim Bhattacharyya, Shweta Mahajan, Sanjay Haresh, Rajeev Yasarla, Reza Pourreza, Litian Liu, Risheek Garrepalli, Roland Memisevic

发表机构 * Qualcomm AI Research(高通人工智能研究院) York University(约克大学) Vector Institute for AI(向量人工智能研究所)

AI总结 提出Ego-MC-Bench基准评估视频LLM在烹饪场景中的实时干预能力,并构建Ego-CoMist反事实合成数据集提升小模型性能。

Comments Qualcomm Interactive Cooking: Ego-MC-Bench -- available at https://huggingface.co/datasets/neuripsedtracksub/ego-mistake-corrections and Ego-CoMist -- available at https://huggingface.co/datasets/neuripsedtracksub/ego-counterfactual-mistakes

详情
AI中文摘要

学习日常技能(如烹饪一道菜)越来越依赖于教学媒体,例如在线视频。这为使用视频(和多模态)大语言模型(LLMs)作为任务指导助手打开了大门。一个潜在的任务指导助手在现实世界中成功的关键能力是,它能够在错误一出现时就主动干预以引导用户。为了评估这一关键能力,我们引入了Ego-MC-Bench(错误纠正),这是一个用于评估在现实烹饪场景中反应性、逐步任务指导的基准。大量实验表明,Ego-MC-Bench对于最先进的视频LLMs具有高度挑战性。我们认为一个关键原因是用于在此任务上微调模型的训练数据有限。尽管存在广泛的烹饪视频数据集,但现有数据集缺乏错误示例以及适当时间的干预。为了帮助解决这一数据限制,我们还引入了Ego-CoMist,这是一个反事实合成数据集,通过将非交互式烹饪视频转换为显示主动干预的监督训练示例而创建。我们表明,在Ego-CoMist上进行微调可以带来性能提升,特别是对于更适合在边缘设备上提供帮助的更小、更高效的视频LLMs。

英文摘要

Learning everyday skills, like cooking a dish, relies increasingly on instructional media such as online videos. This opens the door to the use of video (and multimodal) large language models (LLMs) as task guidance assistants. A crucial capability for the real-world success of a prospective task guidance assistant is it's ability to intervene proactively as soon as a mistake is apparent in order to guide the user. To evaluate this crucial capability, we introduce Ego-MC-Bench (Mistake Corrections), a benchmark for evaluating reactive, step-by-step task guidance in realistic cooking scenarios. Extensive experiments show that Ego-MC-Bench is highly challenging for state-of-the-art video LLMs. We argue that a key reason is the limited availability of training data for fine-tuning models on this task. Although there exists a wide range of cooking video datasets, existing datasets lack examples of mistakes along with appropriately timed interventions. To help address this data limitation, we also introduce Ego-CoMist, a counterfactual synthetic dataset created by transforming non -interactive cooking videos into supervised training examples showing proactive interventions. We show that fine-tuning on Ego-CoMist yields performance gains especially for smaller and more efficient video LLMs that are well suited for delivering assistance on edge devices.