arXivDaily arXiv每日学术速递 周一至周五更新

高校专区

Johns Hopkins University(约翰斯·霍普金斯大学)

至 收录 1032
2606.32017 2026-07-21 cs.LG cs.AI 版本更新

TRIAGE: Role-Typed Credit Assignment for Agentic Reinforcement Learning

TRIAGE:面向智能体强化学习的角色类型化信用分配

Yuanda Xu, Zhengze Zhou, Hejian Sang, Xiaomin Li, Jiaxin Zhang, Xinchen Du, Sen Na, Zhipeng Wang, Alborz Geramifard

机构 * LinkedIn Corporation(领英公司) Harvard University(哈佛大学) Johns Hopkins University(约翰霍普金斯大学)

AI总结 提出TRIAGE框架,通过角色类型化信用分配修正GRPO仅依赖结果信用的盲点,在ALFWorld等任务中提升成功率并减少交互步数。

详情
AI中文摘要

智能体强化学习需要将信用分配给环境交互动作,如搜索、点击、编辑、导航命令和对象交互。标准GRPO使用最终验证器结果作为所有动作令牌的统一优势。该结果信号有用但结构不完整:它在失败轨迹中惩罚有用的探索,并在成功轨迹中强化冗余或倒退动作。我们提出TRIAGE,一种角色类型化信用分配框架,为结果信用添加语义角色轴。结构化评判器将每个片段分类为决定性进展、有用探索、无进展基础设施或倒退,固定角色条件规则将这些标签映射到有界片段级过程奖励。这保持了验证器结果作为优化方向的来源,同时纠正了仅结果信用的两个主要盲点。我们进一步证明,角色条件信用是从角色标签本身可表达的最优片段级修正——将每片段优势残差投影到角色变量上——因此当评判器可靠时,固定角色常数减少优势估计误差,并将其与低方差策略梯度联系起来。在ALFWorld、Search-QA和WebShop上,TRIAGE在两种策略模型上均优于GRPO的成功率,并优于标量评判器推导的过程奖励和结果监督共享骨干价值基线。消融实验表明,增益来自角色类型化而非仅仅添加密集奖励:成功轨迹内倒退的可靠检测是主要贡献者,而探索信用提供一致的次要增益;在完成的ALFWorld和WebShop轨迹上,TRIAGE相对于GRPO还额外减少了10.4%和14.8%的环境交互步数。

英文摘要

Agentic reinforcement learning requires assigning credit to environment-facing actions such as searches, clicks, edits, navigation commands, and object interactions. Standard GRPO uses the final verifier outcome as a uniform advantage over all action tokens. This outcome signal is useful but structurally incomplete: it punishes useful exploration in failed rollouts and reinforces redundant or regressive actions in successful rollouts. We propose TRIAGE, a role-typed credit assignment framework that adds a semantic role axis to outcome credit. A structured judge classifies each segment as decisive progress, useful exploration, no-progress infrastructure, or regression, and a fixed role-conditioned rule maps these labels to bounded segment-level process rewards. This keeps verifier outcomes as the source of optimization direction while correcting the two main blind spots of outcome-only credit. We further show that the Bayes-optimal role-measurable correction is the L2 projection of the per-segment advantage residual onto the role variable, and that TRIAGE's fixed role constants approximate this projection, reducing advantage estimation error whenever the judge is reliable; we connect this to lower-variance policy gradients. Across ALFWorld, Search-QA, and WebShop, TRIAGE improves success rates over GRPO for two policy models and outperforms both a scalar judge-derived process reward and an outcome-supervised shared-backbone value baseline. Ablations show that the gain comes from role typing rather than merely adding dense rewards: reliable detection of regression inside successful trajectories is the dominant contributor, while exploration credit provides a consistent secondary gain; on completed ALFWorld and WebShop rollouts, TRIAGE also reduces environment-facing turns by an additional $10.4\%$ and $14.8\%$ relative to GRPO.

URL PDF HTML 收藏
2607.16095 2026-07-20 cs.RO 新提交

Let the Body Follow: Coupled Egocentric Control for Whole-Body Robot Teleoperation

让身体跟随:用于全身机器人遥操作的耦合自我中心控制

Tsung-Chi Lin, Yichen Xie, Chien-Ming Huang

机构 * New Jersey Institute of Technology(新泽西理工学院) Johns Hopkins University(约翰霍普金斯大学)

AI总结 研究全身机器人遥操作协调难题,提出耦合自我中心控制方法,通过机器人躯干和基座自动响应操作员头部及手臂动作,在用户研究中评估该方法,相比基线混合界面有诸多优势。

Comments 8 pages, 11 figures, 1 table; submitted to the 2026 IEEE-RAS International Conference on Humanoid Robots (Humanoids 2026)

详情
AI中文摘要

全身遥操作要求用户协调多个机器人组件的感知、操作、姿势和移动性。这种协调很困难,因为用户必须同时控制机器人的头部、手臂、躯干和基座,同时保持任务意识并避免运动学或环境约束。本文提出了耦合自我中心控制,这是一种身体跟随遥操作方法,其中机器人的躯干和基座会自动响应操作员的头部和手臂动作。该系统让用户专注于注视和手部控制,而非每次都需要明确的触摸板命令来调整躯干或基座。我们在一项针对TIAGo移动操纵器进行家庭护理启发任务的全身遥操作用户研究中评估了这种方法。与基线混合界面相比,耦合自我中心控制提高了物体操纵效率,减少了基于按钮的控制工作量和手臂奇异点,降低了心理需求和总体工作量,并提高了易用性、易学习性、信心以及用户对躯干和基座控制的偏好。

英文摘要

Whole-body teleoperation requires users to coordinate perception, manipulation, posture, and mobility across multiple robot components. This coordination is difficult because users must simultaneously control the robot's head, arms, torso, and base while maintaining task awareness and avoiding kinematic or environmental constraints. In this paper, we propose coupled egocentric control, a body-following teleoperation approach in which the robot's torso and base automatically respond to the operator's head and arm motions. Rather than requiring explicit touchpad commands for every torso or base adjustment, the system lets users focus on gaze and hand control: head pitch adjusts torso height, head yaw drives base rotation, end-effector height adjusts torso motion, and end-effector workspace boundaries trigger base translation. We evaluate this approach in a user study on whole-body teleoperation of a TIAGo mobile manipulator for home-care-inspired tasks. Compared with a baseline hybrid interface, coupled egocentric control improves object manipulation efficiency, reduces button-based control effort and arm singularities, lowers mental demand and overall workload, and increases ease of use, ease of learning, confidence, and user preference for torso and base control.

URL PDF HTML 收藏
2607.15190 2026-07-20 cs.AI 版本更新

Can We Trust Item Response Theory for AI Evaluation?

我们能信任项目反应理论进行人工智能评估吗?

Han Jiang, Sunbeom Kwon, Jinwen Luo, Ziang Xiao, Susu Zhang

机构 * Johns Hopkins University(约翰·霍普金斯大学) University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校) University of California, Los Angeles(加利福尼亚大学洛杉矶分校)

AI总结 研究探讨项目反应理论(IRT)用于人工智能评估的可靠性,利用六个大语言模型基准模拟响应矩阵,比较四种估计工具,评估IRT在多方面的表现,发现经典估计器在大型基准中可能不可行,可扩展估计器在特定模型集下可能产生不可靠推断。

详情
AI中文摘要

人工智能基准越来越多地利用项目级统计模型,特别是项目反应理论(IRT)来估计模型能力、对系统进行排名、选择信息丰富的示例以及诊断基准质量。然而,人工智能基准数据往往与最初开发标准IRT估计工具的人类测试数据模式不同。我们研究了这些模式不匹配如何挑战IRT建模在人工智能评估中的可靠性。通过六个广泛使用的大语言模型基准得出的项目参数和能力分布,模拟三种常见IRT模型下的响应矩阵,并比较四种估计工具。在18000个模拟条件下,系统评估IRT在模型排名、预测性能和项目特征推断方面的计算可行性、可扩展性和可靠性。结果表明,经典估计器在大型基准设置中可能变得不可行,而可扩展估计器在小或非正态分布模型集下可能产生不可靠的项目级和排名推断。本研究确定了潜在特质模型何时可靠支持或可能扭曲人工智能基准测试声明,以及需要何种样本量和诊断才能可靠使用。

英文摘要

AI benchmarks increasingly leverage item-level statistical models, particularly item response theory (IRT), to estimate model capabilities, rank systems, select informative examples, and diagnose benchmark quality. However, AI benchmark data often departs from the data regime of human testing, for which standard IRT estimation tools were originally developed: benchmarks typically involve fewer evaluated models, far more items, and capability distributions that may be skewed, clustered, or multimodal. We examine how these regime mismatches challenge the reliability of IRT modeling for AI evaluation. Using item parameters and capability distributions derived from six widely used LLM benchmarks, we simulate response matrices under three common IRT models and compare four estimation tools used in recent benchmark studies: marginal maximum likelihood, Markov chain Monte Carlo, variational inference, and a neural pseudo-Siamese estimator. Across 18,000 simulation conditions, we systematically evaluate computational feasibility, scalability, and the reliability of IRT inferences about model rankings, predicted performance, and item characteristics. Results show that classical estimators can become infeasible in large benchmark settings, whereas scalable estimators can produce unreliable item-level and ranking inferences with small or non-normally distributed model sets. This study identifies when latent trait models reliably support or risk distorting AI benchmarking claims, and what sample sizes and diagnostics are needed for trustworthy use.

URL PDF HTML 收藏
2510.11539 2026-07-20 cs.RO math.OC 版本更新

Simultaneous Calibration of Noise Covariance and Kinematics for State Estimation of Legged Robots via Bi-level Optimization

通过双层优化同时校准噪声协方差和运动学参数以实现四足机器人和双足机器人的状态估计

Denglin Cheng, Jiarong Kang, Xiaobin Xiong

机构 * Legged AI Lab(足式人工智能实验室) Johns Hopkins University(约翰霍普金斯大学) University of Wisconsin–Madison(威斯康星大学麦迪逊分校) Shanghai Innovation Institute (SII)(上海创新研究院)

AI总结 本文提出一种双层优化框架,通过联合校准协方差矩阵和运动学参数提升机器人状态估计的准确性与不确定性校准。

详情
AI中文摘要

准确的状态估计对于在动态不确定环境中运行的腿部和空中机器人至关重要。一个关键挑战在于指定过程和测量噪声协方差,这些通常未知或手动调优。本文介绍了一种双层优化框架,通过闭环估计器同时校准协方差矩阵和运动学参数。上层将噪声协方差和模型参数作为优化变量,下层执行全信息估计器。通过估计器求导可直接优化轨迹级目标,从而获得准确且一致的状态估计。我们在四足和双足机器人上验证了该方法,证明其比手动调优的基线方法在估计精度和不确定性校准上有了显著提升。本文方法将状态估计、传感器和运动学校准统一到一个原理性、数据驱动的框架中,适用于各种机器人平台。

英文摘要

Accurate state estimation is critical for legged and aerial robots operating in dynamic, uncertain environments. A key challenge lies in specifying process and measurement noise covariances, which are typically unknown or manually tuned. In this work, we introduce a bi-level optimization framework that jointly calibrates covariance matrices and kinematic parameters in an estimator-in-the-loop manner. The upper level treats noise covariances and model parameters as optimization variables, while the lower level executes a full-information estimator. Differentiating through the estimator allows direct optimization of trajectory-level objectives, resulting in accurate and consistent state estimates. We validate our approach on quadrupedal and humanoid robots, demonstrating significantly improved estimation accuracy and uncertainty calibration compared to hand-tuned baselines. Our method unifies state estimation, sensor, and kinematics calibration into a principled, data-driven framework applicable across diverse robotic platforms.

URL PDF HTML 收藏
2607.15175 2026-07-17 cs.CL 新提交

Linear representations of grammaticality in neural language models

神经语言模型中语法性的线性表示

Jane Li, Najoung Kim

机构 * Johns Hopkins University(约翰·霍普金斯大学) Boston University(波士顿大学)

AI总结 探讨神经语言模型能否基于语法性区分字符串,通过质量均值探测研究语法性是否编码在其内部表示中,结果表明语法性在多种预训练模型中有力编码,为相关争论提供新证据及评估框架。

详情
AI中文摘要

神经语言模型是否具备基于语法性区分字符串的能力在计算语言学文献中仍是一个有争议的话题。现有证据大多依赖基于概率的度量。本文超越基于概率的评估,通过质量均值探测研究语法性是否编码在神经语言模型的内部表示中,测试语法和非语法句子在表示空间中是否系统分离,还考察了表示与相关属性的独立性及跨语法现象和语言的泛化性。结果表明语法性在多种预训练神经语言模型的句子表示中得到有力编码,这为语言模型句法知识本质的争论提供了新证据,也提供了不依赖字符串概率的语法能力评估框架。

英文摘要

Whether neural language models (NLMs) possess the ability to distinguish strings on the basis of their grammaticality remains a debated topic in the computational linguistics literature. Existing evidence has largely relied on probability-based measures, testing whether models assign higher probabilities to grammatical than ungrammatical strings. However, probability comparisons have been criticized as a measure for grammatical knowledge based on the assumption that grammaticality is inherently entangled with likelihood. Model-assigned probability is a function of many related sentence properties, such as lexical frequency, plausibility, and world knowledge. In this work, we move beyond probability-based evaluations and investigate whether grammaticality is encoded in the internal representations of NLMs. Using mass-mean probing, we test whether grammatical and ungrammatical sentences are systematically separated in representational space. We further examine the extent to which these representations are independent of sentence properties that are correlated with grammaticality, as well as their generalization across grammatical phenomena and languages. Our results provide evidence that grammaticality is robustly encoded in sentence representations of a wide range of pretrained NLMs, yielding clear representational separation on the dimension of grammaticality that cannot be fully explained by alternative sentence-level factors. Moreover, this encoding generalizes across a broad range of grammatical phenomena and to some degree, across languages, suggesting that grammaticality constitutes a coherent representational dimension in contemporary NLMs. These findings contribute new evidence to debates about the nature of syntactic knowledge in language models and offer a complementary framework for evaluating grammatical competence that is not dependent on string probabilities alone.

URL PDF HTML 收藏
2607.14499 2026-07-17 cs.AI 新提交

Contextualized Evaluation of Vision Language Models through Dynamic, Multi-turn Interactions

通过动态多轮交互对视觉语言模型进行情境化评估

Yijiang Li, Huiqi Zou, Bingyang Wang, Ziang Xiao

机构 * UC San Diego(加州大学圣地亚哥分校) Northeastern University(东北大学) Georgia Institute of Technology(佐治亚理工学院) Johns Hopkins University(约翰·霍普金斯大学)

AI总结 研究多模态大语言模型现实有效性问题,提出CEDI框架,通过三方交互、多轮半结构化对话及多种策略评估,应用于视觉幻觉,发现能揭示更多接近实际情况的幻觉,凸显其对MLLMs能力评估的作用。

详情
AI中文摘要

多模态大语言模型(MLLMs)在基准测试中取得了显著进展,但其在现实世界中的有效性仍不确定。这种差距源于受控静态环境中的基准测试与现实世界应用的动态、交互和情境化性质之间的根本错位。为弥合这一差距,我们提出了CEDI(通过动态多轮交互对MLLMs进行情境化评估)框架,将评估重新构建为被评估模型、自动考官和评分者之间的三方交互。考官通过基于任务的图形表示进行多轮半结构化对话。通过导航状态空间转换,CEDI部署从澄清请求到对抗性探测等各种策略,以获取性能证据。我们将CEDI应用于视觉幻觉。多个模型、不同设置、数据集和领域的实证结果表明,情境化、交互式评估不仅比传统静态评估揭示出更多幻觉,而且揭示出的幻觉更接近实际用例中出现的幻觉。我们还表明,幻觉往往会通过自我强化的对话历史在长语境中累积,并且模型特别容易受到需要拒绝前提或拒绝的问题的影响。这些发现共同凸显了CEDI是朝着对MLLMs能力进行现实、系统和生态有效评估迈出的一步。

英文摘要

Multi-modal Large Language Models (MLLMs) have made substantial advances on benchmarks, yet their real-world effectiveness remains uncertain. This gap stems from the fundamental misalignment between benchmarks in controlled, static settings and the dynamic, interactive, and contextualized nature of real-world applications. To bridge this gap, we propose CEDI (Contextualized Evaluations of MLLMs through Dynamic, multi-round Interactions), a framework that recasts evaluation as a three-party interaction between an evaluatee model, an automated examiner, and a grader. The examiner conducts multi-turn, semi-structured conversation guided by a graph-based representation of the task. By navigating state-space transitions, CEDI deploys diverse strategies, from clarification requests to adversarial probes, to elicit performance evidence. We apply CEDI to visual hallucinations. Empirical results across multiple models, diverse settings, datasets, and domains show that contextualized, interactive evaluations reveal not only significantly more hallucinations than conventional static evaluation but also ones that more closely resemble those arising in practical use cases. We further show that hallucinations often accumulate over long contexts, through self-reinforcing dialogue history, and models are particularly vulnerable to questions requiring premise rejection or refusal. Together, these findings highlight CEDI as a step toward realistic, systematic, and ecologically valid assessments of MLLMs' capabilities. Code is available at github.com/williamium3000/cedi.

URL PDF HTML 收藏
2607.14193 2026-07-17 stat.ML cs.LG cs.NA math.NA physics.med-ph 新提交

Operator-Informed Gaussian Processes for Complex Helmholtz Wavefields: From Synthetic Benchmarks to In Vivo Brain Elastography

用于复杂亥姆霍兹波场的算子引导高斯过程:从综合基准测试到体内脑弹性成像

Boyuan Deng, Kshitiz Upadhyay, Michael Shields

机构 * Department of Civil and Systems Engineering, Johns Hopkins University(约翰霍普金斯大学土木与系统工程系) Department of Aerospace Engineering and Mechanics, University of Minnesota(明尼苏达大学航空航天工程与力学系)

AI总结 研究将算子引导高斯过程回归扩展到复值亥姆霍兹问题,通过转化复算子用实值GP条件推断。在基准问题和脑弹性成像中表现良好,能返回波场后验,确定了精度上限与未校准的不确定性,校准不确定性是后续核心步骤。

Comments 26 pages, 7 figures

详情
AI中文摘要

亥姆霍兹方程支配时谐波传播,在耗散介质中,复模量使波数平方κ²为复数。从稀疏、有噪声的数据推断此类场需要能量化自身不确定性的求解器。物理引导的高斯过程(GP)回归通过返回解的后验来提供此功能,但算子条件公式几乎仅针对实值场开发。我们通过将复算子转化为等效耦合实块,将算子引导的GP回归扩展到复值亥姆霍兹问题,从而能用标准实值GP条件进行推断。该构造允许一系列先验,从适当的对角先验到共区域化和多尺度变体,并基于偏微分方程残差和边界迹进行条件设定。在一到三维的基准问题上,该求解器在内部约束预算小得多的情况下与有限差分和神经网络基线具有竞争力。与那些确定性基线不同,它返回复波场的后验而不是点估计。应用于体内脑磁共振弹性成像时,适当的多尺度先验将剪切卷曲场重建与测量的相关性提高到0.77,高于目标值0.75。增益来自多尺度核而非实虚耦合。我们还确定了由模型失配设定的低频精度上限以及尚未校准的后验不确定性。因此,校准不确定性成为耗散介质中概率波场推断的核心下一步。

英文摘要

The Helmholtz equation governs time-harmonic wave propagation, and in dissipative media a complex modulus renders its squared wavenumber $κ^2$ complex. Inferring such fields from sparse, noisy data calls for solvers that also quantify their own uncertainty. Physics-informed Gaussian-process (GP) regression supplies this by returning a posterior over the solution, yet operator-conditioned formulations have been developed almost exclusively for real-valued fields. We extend operator-informed GP regression to complex-valued Helmholtz problems by realifying the complex operator into an equivalent coupled real block, which enables inference with standard real-valued GP conditioning. The construction admits a family of priors, from a proper diagonal prior to coregionalized and multiscale variants, and conditions on PDE residuals and boundary traces. On benchmark problems in one to three dimensions, the solver is competitive with finite-difference and neural-network baselines at a far smaller interior-constraint budget. Unlike those deterministic baselines, it returns a posterior over the complex wavefield rather than a point estimate. Applied to \textit{in vivo} brain magnetic resonance elastography, a proper multiscale prior reconstructs the shear curl field to a correlation of $0.77$ with measurement, above a $0.75$ target. The gain arises from the multiscale kernel rather than from real--imaginary coupling. We further identify a low-frequency accuracy ceiling set by model mismatch and a posterior uncertainty that is not yet calibrated. Calibrated uncertainty therefore emerges as the central next step for probabilistic wavefield inference in dissipative media.

URL PDF HTML 收藏
2607.06214 2026-07-17 cs.AI 版本更新

A framework for single and multi-agent human-AI curiosity ecosystems

用于单智能体和多智能体人类-人工智能好奇心生态系统的玩具框架

Ilya E. Monosov

机构 * Johns Hopkins University(约翰斯·霍普金斯大学)

AI总结 该论文提出一个玩具框架,将好奇心视为生态系统,探讨单智能体询问策略及相关决策项权重随经验的变化,还扩展到多智能体,追踪多种指标,为研究好奇心生态及设计多智能体发现系统提供概念框架。

Comments fixed abstract in this version

详情
AI中文摘要

本文提供了一个将好奇心视为生态系统的玩具框架。首先,提出单智能体的询问策略(智能体如何、何时以及为何提问)取决于其对即时不确定性降低、成本、延迟回报以及保持问题开放的价值的重视程度。框架中的关键概念是这些与决策相关的项的权重会随经验变化。例如,一段低成本、快速得到答案的时期可能在短时间内改变询问成本,并在较长时间内改变智能体倾向回答的问题类型。其次,这些想法扩展到多个探索共享知识领域的智能体,框架追踪询问量、主题多样性、前沿导向询问、冗余性和可复用知识。结果是一个用于研究好奇心生态以及未来设计用于发现的多智能体人工智能系统的概念性玩具框架。它是《神经科学趋势》中一篇正在审稿的论文的配套文章。

英文摘要

This paper offers a framework for considering curiosity as an ecosystem. First, it suggests that a single agent's inquiry policy (how, when, and why an agent asks a question) depends on how the agent values immediate uncertainty reduction, costs, delayed return, and the value of keeping the question open. A key concept in the framework is that the weights on these decision-related terms can change with experience. For example, a period of cheap, quickly answered questions may change the cost of inquiry on a short timescale and change which kinds of questions the agent is drawn to answer over a longer timescale. Second, these ideas are extended to many agents exploring a shared knowledge landscape, and there the framework tracks inquiry volume, topic diversity, frontier-directed inquiry, redundancy, and reusable knowledge. The result is a conceptual framework for studying curiosity ecology and for future efforts towards designing multi-agent AI systems for discovery.

URL PDF HTML 收藏
2607.13841 2026-07-16 cs.LG stat.ML 新提交

Heavy-Tailed Flow Matching via Random Clocks

通过随机时钟进行重尾流匹配

Zhouhao Yang, Yezhen Wang, Kenji Kawaguchi, Vladimir Braverman, Haoyang Cao

机构 * Johns Hopkins University(约翰斯·霍普金斯大学) National University of Singapore(新加坡国立大学)

AI总结 研究重尾数据匹配问题,提出HTFM框架,将重尾源视为时钟条件高斯源混合,用截断对数签名特征编码时钟。实验显示其在多领域优于高斯流匹配等基线,保留低NFE采样优势,还提供尾部控制接口。

详情
AI中文摘要

重尾数据出现在许多领域,如不平衡图像数据集、金融回报和极端天气等,其中罕见事件具有不成比例的重要性。标准扩散和流匹配模型通常从高斯噪声或高斯源分布开始,对重尾数据的归纳匹配较差。我们提出了通过随机时钟进行重尾流匹配(HTFM)框架,将重尾源描绘为时钟条件高斯源的混合。给定时钟路径时,源分布和流是高斯的;对时钟求边缘分布得到覆盖高斯、α稳定和学生t族的高斯尺度混合。为使时钟条件向量场实用,我们使用截断对数签名特征编码路径值时钟,使速度场能以可忽略的开销适应已实现的条件空间。实验表明,在二维不平衡α稳定混合、CIFAR10-LT和HRRR天气场中,HTFM在模式覆盖、样本质量和尾部统计恢复方面优于高斯流匹配和有竞争力的重尾基线,同时保留了流匹配的低NFE采样优势。此外,随机时钟公式还提供了一个实用的尾部控制接口。

英文摘要

Heavy-tailed data arise in many domains where rare events carry disproportionate importance, such as imbalanced image datasets, financial returns, and weather extremes. Standard diffusion and flow-matching models typically begin from Gaussian noise or Gaussian source distributions, which yield tractable training targets but provide a poor inductive match for heavy-tailed data. We propose Heavy-Tailed Flow Matching via Random Clocks (HTFM), a framework that portrays heavy-tailed sources as mixtures of clock-conditioned Gaussian sources. Conditioning on a given clock path, the source distribution and flow are Gaussian; marginalizing over the clock gives a Gaussian scale mixture covering Gaussian, $α$-stable, and Student-t families. To make the clock-conditioned vector field practical, we encode the path-valued clock using truncated logsignature features, allowing the velocity field to adapt to the realized conditional space with negligible overhead. Empirically, on 2D imbalanced $α$-stable mixtures, CIFAR10-LT, and HRRR weather fields, HTFM improves mode coverage, sample quality, and tail-statistic recovery over Gaussian flow matching and competitive heavy-tailed baselines, while retaining the low-NFE sampling advantage of flow matching. Moreover, the random-clock formulation further provides a practical tail-control interface: by varying only the clock law or tail parameter, the same architecture can calibrate the ``heaviness'' of generated tails across different distribution families.

URL PDF HTML 收藏
2607.13837 2026-07-16 cs.LG cs.AI 新提交

NodeImport: Imbalanced Node Classification with Node Importance Assessment

NodeImport:基于节点重要性评估的不平衡节点分类

Nan Chen, Zemin Liu, Bryan Hooi, Bingsheng He, Jun Hu, Jia Chen

机构 * Johns Hopkins University(约翰斯·霍普金斯大学) Zhejiang University(浙江大学) National University of Singapore(新加坡国立大学) Grabtaxi Holdings Pte Ltd(Grab出租车控股私人有限公司)

AI总结 针对图节点分类的类别不平衡问题,提出利用平衡元集评估节点重要性的方法,推导公式并开发新框架,能过滤有价值节点,构建高质量元集,实验证明该框架在缓解不平衡上具优越性。

Journal ref KDD '25: Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.1, 2025, Pages 94 - 105

详情
AI中文摘要

在实际应用中,图上的节点分类常面临类别不平衡挑战,多数类主导训练导致模型性能有偏差。传统GNN在此场景中表现不佳。现有解决方案存在不足。本文提出利用平衡元集进行重要性度量的方法,识别可抵消类别不平衡的重要节点用于模型训练,理论推导直接评估节点重要性的公式,开发新框架过滤有价值节点,还介绍构建高质量元集的策略。通过实验证明该框架在缓解类别不平衡方面的优越性。

英文摘要

In real-world applications, node classification on graphs often faces the challenge of class imbalance, where majority classes dominate training, resulting in biased model performance. Traditional GNNs often struggle in such scenarios, as they tend to overfit to majority classes while underrepresenting minority classes. Existing solutions, which either prioritize nodes based on class size or synthesize new nodes for minority classes, often fall short of effectively addressing this imbalance issue. This paper introduces an approach to class-imbalanced node classification by utilizing a balanced meta-set for importance measurement, where a training node is considered significant if it enhances model performance under an unbiased setting. Our method identifies important nodes that can counteract class imbalance and utilizes them for model training, allowing for fine-grained and dynamic node selection throughout the training process. We theoretically derive a formula to directly assess node importance, reducing computational overhead and providing an intuitive threshold for node selection. Guided by this metric, we develop a novel framework that filters valuable labeled, unlabeled, and synthetic nodes that enhance model performance in an unbiased context. A key advantage of this framework is its separation of the synthetic node generation process from the filtering process, ensuring compatibility with various node generation methods. Furthermore, we introduce a strategy to construct a high-quality meta-set that closely approximates the overall feature distribution, ensuring robust representation of each class. We evaluate our framework, NodeImport, across multiple datasets using popular GNN architectures, demonstrating its superiority over existing baselines. Our results highlight the flexibility and effectiveness of the framework in mitigating class imbalance, leading to improved outcomes.

URL PDF HTML 收藏
2607.13456 2026-07-16 cs.CV cs.CG cs.GR 新提交

TreeSRNF: Square-Root Normal Fields for Generative Modelling of the Geometric and Structural Variability in Tree-like 3D Objects

TreeSRNF:用于树状3D对象几何和结构变异性生成建模的平方根法向场

Tahmina Khanam, Hamid Laga, Mohammed Bennamoun, Guanjin Wang, Ferdous Sohel, Farid Boussaid, Anuj Srivastava

机构 * Murdoch University(莫道克大学) University of Western Australia(西澳大利亚大学) Johns Hopkins University(约翰霍普金斯大学)

AI总结 该研究针对树状3D对象几何与结构变异性问题,将平方根法向场表示推广到此类对象,构建黎曼树状空间及度量,开发相关算法,可计算统计摘要并合成新对象,性能显著优于现有技术。

Comments ECCV 2026

详情
AI中文摘要

我们引入了一个新颖的数学框架,用于分析和生成复杂的树状3D对象,如植物树木和植物,它们在3D几何形状和分支结构上都会变形。与之前只考虑树状对象骨骼结构或用分支厚度近似其3D几何形状的工作不同,该框架能精确地对树枝的3D几何形状及其相互连接方式进行建模。本文首先将最初为零亏格曲面统计分析提出的平方根法向场(SRNF)表示推广到树状3D对象。接着将树状三维物体视为一个新的黎曼树状空间上的点,该空间配备了一种新的黎曼度量,用以测量表面弯曲和拉伸的程度以及将一个3D树形与另一个对齐所需的结构变化。这样,变形就成为这个新树状空间中的轨迹。我们分析了这个新树状空间及其相应度量的理论性质,并开发了用于计算复杂3D树之间逐点及分支对应和测地线的算法。最后展示了如何使用这些构建模块来计算树状3D对象集合的统计摘要,即变异均值和模式,以及通过从拟合到树状3D对象群体的概率分布中采样来合成新的树状3D对象。我们在真实和合成的植物及树木上展示了该框架的性能和实用性,并表明它显著优于现有技术。

英文摘要

We introduce a novel mathematical framework for analyzing and generating complex tree-shaped 3D objects, such as botanical trees and plants, which deform both in their 3D geometry and branching structure. Unlike previous works, which either consider only the skeletal structure of tree-like objects or approximate their 3D geometry using branch thickness, the proposed framework accurately models both the 3D geometry of the tree branches and the way they are interconnected. In this paper, we first generalize the Square Root Normal Fields (SRNF) representation, originally proposed for the statistical analysis of genus-0 surfaces, to tree-shaped 3D objects. We then treat tree-shaped 3D objects as points on a novel Riemannian tree-shape space equipped with a novel Riemannian metric that measures the amount of surface bending and stretching, and structural changes one needs to apply to one 3D tree-shape to align it with another. This way, deformations become trajectories in this novel tree-shape space. We analyze the theoretical properties of this novel tree-shape space and the corresponding metric and develop algorithms for computing point-wise and branch-wise correspondences and geodesic paths between complex 3D trees. We finally show how to use these building blocks for (1) computing statistical summaries, \ie means and modes of variation, of collections of tree-shaped 3D objects, and (2) synthesizing novel tree-shaped 3D objects by sampling from probability distributions fitted to a population of tree-shaped 3D objects. We demonstrate the performance and utility of the proposed framework on real and synthetic plants and botanical trees and show that it significantly outperforms the state-of-the-art.

URL PDF HTML 收藏
2607.12939 2026-07-15 cs.CV 新提交

Point Tracking in Surgery--The 2025 Surgical Tattoos in Infrared Challenge (STIRC2025)

手术中的点跟踪——2025年红外手术纹身挑战(STIRC2025)

Adam Schmidt, Mert Asim Karaoglu, Zijian Wu, Jiaming Zhang, Yuxin Chen, Tim Salcudean, Ho-Gun Ha, Minkang Jang, Kyungmin Jung, Ihsan Ullah, Hyunki Lee, Suresh Guttikonda, Sarah Latus, Alexander Schlaefer, Xinkai Zhao, Yuichiro Hayashi, Masahiro Oda, Takayuki Kitasaka, Kensaku Mori, Peng Liu, Chenyang Li, Stefanie Speidel, Aoife Gardiner, Agostino Stilli, Danail Stoyanov, Francisco Vasconcelos, Anwesa Choudhuri, Meng Zheng, Zhongpai Gao, Benjamin Planche, Van Nguyen Nguyen, Terrence Chen, Ziyan Wu, Alexander Ladikos, Omid Mohareri

机构 * Intuitive Surgical Inc.(直观外科公司) ImFusion GmbH(ImFusion有限公司) Technical University of Munich(慕尼黑工业大学) University of British Columbia(英属哥伦比亚大学) Johns Hopkins University(约翰·霍普金斯大学) Daegu Gyeongbuk Institute of Science and Technology(大邱庆北科学技术院) Hamburg University of Technology(汉堡工业大学) SustAInLivWork Center of Excellence(可持续生活工作卓越中心) Graduate School of Informatics, Nagoya University(名古屋大学信息科学研究生院) Information Technology Center, Nagoya University(名古屋大学信息技术中心) Department of Information Science, Aichi Institute of Technology(爱知工业大学信息科学系) Research Center for Medical Bigdata, National Institute of Informatics(国立信息学研究所医学大数据研究中心) Department of Translational Surgical Oncology, National Center for Tumor Diseases (NCT), NCT/UCC Dresden, a partnership between DKFZ, Faculty of Medicine and University Hospital Carl Gustav Carus, TUD Dresde(德累斯顿国家肿瘤疾病中心(NCT)转化外科肿瘤学系,NCT/UCC德累斯顿,由德国癌症研究中心、医学院和卡尔·古斯塔夫·卡鲁斯大学医院、德累斯顿工业大学合作成立)

AI总结 介绍2025年手术点跟踪挑战(STIRC2025),参与者提交算法基于红外手术纹身数据集(STIR)评估,含准确性和效率两组件,作为MICCAI EndoVis 2025一部分进行,总结结果与方法,数据集和代码可获取。

Comments 9 pages, 12 figures. arXiv admin note: substantial text overlap with arXiv:2503.24306

详情
AI中文摘要

手术中的点跟踪对于实现诸如分割、3D重建、虚拟组织地标定位、基于自主探头的扫描和子任务自主等下游任务至关重要。本文介绍了点跟踪挑战的2025年迭代,参与者提交算法进行量化。使用名为红外手术纹身(STIR)的数据集评估算法,该挑战称为2025年STIR挑战(STIRC2025)。STIRC2025包括准确性和效率两个定量组件。准确性组件测试算法在体内和体外序列上的准确性,效率组件测试算法推理延迟。该挑战作为MICCAI EndoVis 2025的一部分进行,七个团队参与。本文总结了挑战结果和参与者方法。挑战数据集和基线模型及指标计算代码可通过链接获取。

英文摘要

Point tracking in surgery is crucial to enable applications in downstream tasks such as segmentation, 3D reconstruction, virtual tissue landmarking, autonomous probe-based scanning, and subtask autonomy. This paper introduces the 2025 iteration of a point tracking challenge to address this, wherein participants submit their algorithms for quantification. Their algorithms are evaluated using a dataset named surgical tattoos in infrared (STIR), with the challenge named the STIR Challenge 2025 (STIRC2025). The STIR Challenge 2025 comprises two quantitative components: accuracy and efficiency. The accuracy component tests the accuracy of algorithms on in vivo and ex vivo sequences. The efficiency component tests algorithm inference latency. The challenge was conducted as a part of MICCAI EndoVis 2025, and seven teams participated in this challenge. In this paper we summarize the challenge results and participant methods. The challenge dataset is available at: https://zenodo.org/records/20191078, and the code for baseline models and metrics calculation is available here: https://github.com/athaddius/STIRMetrics

URL PDF HTML 收藏
2607.12450 2026-07-15 cs.CV 新提交

Let RGB Be the Language of Vision

让 RGB 成为视觉的语言

Timing Yang, Jinrui Yang, Xinlong Li, Yuhan Wang, Haoran Li, Yanqing Liu, Guoyizhe Wei, Jixuan Ying, Chen Wei, Rama Chellappa, Yuyin Zhou, Cihang Xie, Alan Yuille, Feng Wang

机构 * Johns Hopkins University(约翰·霍普金斯大学) UC Santa Cruz(加州大学圣克鲁兹分校) Carnegie Mellon University(卡内基梅隆大学) Rice University(莱斯大学)

AI总结 该研究为视觉模型引入统一表述 RGB In and RGB Out(RINO),将多种视觉信息转为 RGB 图像编辑问题,共享架构参数,基于通用主干无需微调,在多视觉任务上有强大零样本性能,为统一视觉语言系统提供见解。

详情
AI中文摘要

这项工作为视觉模型引入了一种统一的表述方式,其中除自然图像之外的各种视觉信息形式,如掩码、深度图和其他结构化视觉信号,都被表示为 RGB 图像,而一般视觉任务可转化为常见的 RGB 到 RGB 图像编辑问题。在这种范式下,不同类型的视觉信息在内部与自然图像共享相同的编码、解码架构和参数,使单个模型能够通过统一视觉接口跨任务转移,类似于语言模型处理文本的方式。我们将此表述称为 RGB 输入与 RGB 输出(RINO)。基于通用图像编辑主干且无需特定任务微调,RINO 在密集理解任务(如分割和深度估计,我们将输出统一为 RGB)以及密集条件生成任务(如姿态到图像生成,我们将输入统一为 RGB)上展示了强大且具有竞争力的零样本性能。我们希望这项研究能为通用统一视觉语言系统提供有用见解,即通过共享视觉语言来表达、解释和解决各种视觉任务。代码可从此 https 网址获取。

英文摘要

This work introduces a unified formulation for vision models, where diverse forms of visual information beyond natural images, such as masks, depth maps, and other structured visual signals, are all represented as RGB images, while general visual tasks can be converted into a common RGB-to-RGB image editing problem. In this paradigm, different types of visual information internally share the same encoding and decoding architecture and parameters as natural images, enabling a single model to transfer across tasks through a unified visual interface, in a way analogous to how language models operate over text. We refer to this formulation as RGB In and RGB Out (RINO). Built upon a generic image editing backbone without task-specific fine-tuning, RINO demonstrates robust and competitive zero-shot performance on both dense understanding tasks such as segmentation and depth estimation (where we unify outputs as RGB), and dense-conditioned generation tasks such as pose-to-image generation (where we unify inputs as RGB). We hope this study provides useful insights toward general unified vision-language systems, where diverse visual tasks can be expressed, interpreted, and solved through a shared visual language. Code is available at https://github.com/yangtiming/RINO.

URL PDF HTML 收藏
2607.12042 2026-07-15 cs.CV cs.LG 新提交

SymbOmni: Evolving Agentic Omni Models via Symbolic Concept Learning

SymbOmni:通过符号概念学习进化智能全能模型

Jinxiu Liu, Jianru Li, Tanqing Kuang, Xuanming Liu, Kangfu Mei, Yandong Wen, Weiyang Liu

机构 * South China University of Technology(华南理工大学) Westlake University(西湖大学) Johns Hopkins University(约翰·霍普金斯大学) The Chinese University of Hong Kong(香港中文大学) Shenzhen Loop Area Institute(深圳环区研究院)

AI总结 针对视觉生成模型无法累积学习与自主进化的问题,提出SymbOmni智能全能模型,通过符号概念学习和归纳-转导循环运行,经言语反向传播训练。实验验证其在多方面性能优越,能有效降低令牌消耗并实现持续学习。

Comments ECCV 2026 (49 pages, 10 figures, project page: https://spherelab.ai/symbomni)

详情
AI中文摘要

视觉生成在诸多领域日益普遍,但现有整体模型存在‘永远的新手’问题,无法累积学习与自主进化。为此提出SymbOmni,一种通过符号概念学习进行累积进化的智能全能模型。其核心是符号概念盒,将低级操作抽象为可复用的符号工作流指令。通过归纳-转导循环运行,训练采用基于语言反馈的言语反向传播。实验表明,它在迭代创建中显著优于现有基于代理的系统,图像质量和任务成功率超闭源模型,有效降低令牌消耗,实现有效持续学习。

英文摘要

Visual generation is increasingly ubiquitous in diverse domains, from text-to-image/video synthesis to multimodal interactive creation. Yet prevailing monolithic models remain fundamentally constrained by their inability to learn cumulatively and evolve autonomously, which is a limitation we term the "perpetual novice" problem. They lack mechanisms for structuring experience into reusable knowledge and therefore rely on brittle, "from-scratch" reasoning for each task, resulting in poor compositional generalization and inefficient knowledge retention. Motivated by these limitations, we propose SymbOmni, an agentic omni-model designed for cumulative evolution through Symbolic Concept Learning. At its core is the Symbolic Concept Box, an optimizable memory module that abstracts low-level operations into reusable Symbolic Workflow Instructions. SymbOmni operates through an induction-transduction cycle: experiences are abstracted into symbolic concepts (induction), which are then adaptively composed to solve novel tasks (transduction). The training is done by verbalized backpropagation with language-based feedback to enable continuous self-improvement without gradient-based model fine-tuning. Comprehensive experiments validate that (I) SymbOmni significantly outperforms existing agent-based systems for iterative creation and also surpasses closed-source models (e.g., Nano Banana, GPT-Image-1) in both image quality and task success rates; (II) SymbOmni effectively reduces token consumption by over 40% while maintaining competitive generation quality; and (III) SymbOmni enables effective continual learning by achieving cumulative gains across multiple online-learning benchmarks and setting a new state of the art.

URL PDF HTML 收藏
2602.15892 2026-07-15 cs.CV cs.AI 版本更新

Egocentric Bias in Vision-Language Models

视觉语言模型中的自我中心偏差

Maijunxian Wang, Yijiang Li, Bingyang Wang, Tianwei Zhao, Ran Ji, Qingying Gao, Emmy Liu, Hokin Deng, Dezhi Luo

机构 * Cognitive Science Program, University of California, Berkeley(加州大学伯克利分校认知科学项目) Department of Electrical and Computer Engineering, University of California San Diego(加州大学圣地亚哥分校电气与计算机工程系) School of Computer Science, Georgia Institute of Technology & Emory University(佐治亚理工学院计算机科学学院及埃默里大学) Department of Computer Science, Johns Hopkins University(约翰霍普金斯大学计算机科学系) Department of Cognitive Science, University of California San Diego(加州大学圣地亚哥分校认知科学系) Equal Advising Department of Computer Science & Wilmer Eye Institute, Johns Hopkins University(约翰霍普金斯大学计算机科学系及威尔默眼科研究所) Language Technologies Institute, Carnegie Mellon University(卡内基梅隆大学语言技术研究所) Robotics Institute, Carnegie Mellon University(卡内基梅隆大学机器人研究所) Weinberg Institute for Cognitive Science, University of Michigan(密歇根大学韦恩伯格认知科学研究所)

AI总结 本文提出FlipSet基准,揭示视觉语言模型在第二级视觉视角推理中存在系统性自我中心偏差,表明模型在整合社会认知与空间操作方面存在根本性不足。

Comments Accepted at CogSci 2026 (Best Undergraduate Student Paper)

详情
AI中文摘要

视觉视角的推理——推断从他人视角世界如何呈现——是社会认知的基础。我们引入FlipSet,一个用于评估视觉语言模型中第二级视觉视角推理(L2 VPT)的诊断基准。该任务要求模拟从另一个代理视角对二维字符字符串进行180度旋转,隔离空间变换与三维场景复杂性。评估103个VLMs发现系统性的自我中心偏差:绝大多数表现低于随机水平,其中约四分之三的错误重现了摄像机视角。控制实验揭示了组合缺陷——模型在孤立情况下实现高理论-心准确率和高于随机水平的内部旋转,但当需要整合时却彻底失败。这种分离表明当前VLMs缺乏将社会意识与空间操作结合的机制,暗示了基于模型的空间推理的根本限制。FlipSet为多模态系统诊断视角推理能力提供了认知基础的测试平台。

英文摘要

Visual perspective taking--inferring how the world appears from another's viewpoint--is foundational to social cognition. We introduce FlipSet, a diagnostic benchmark for Level-2 visual perspective taking (L2 VPT) in vision-language models. The task requires simulating 180-degree rotations of 2D character strings from another agent's perspective, isolating spatial transformation from 3D scene complexity. Evaluating 103 VLMs reveals systematic egocentric bias: the vast majority perform below chance, with roughly three-quarters of errors reproducing the camera viewpoint. Control experiments expose a compositional deficit--models achieve high theory-of-mind accuracy and above-chance mental rotation in isolation, yet fail catastrophically when integration is required. This dissociation indicates that current VLMs lack the mechanisms needed to bind social awareness to spatial operations, suggesting fundamental limitations in model-based spatial reasoning. FlipSet provides a cognitively grounded testbed for diagnosing perspective-taking capabilities in multimodal systems.

URL PDF HTML 收藏
2511.18089 2026-07-15 cs.CV 版本更新

Together, Then Apart: Balancing Alignment and Distinctiveness for Multimodal Survival Analysis

在一起,然后分开:平衡多模态生存分析中的对齐与独特性

Wenjing Liu, Qin Ren, Wen Zhang, Yuewei Lin, Chenyu You

机构 * Stony Brook University(石溪大学) Stanford University(斯坦福大学) Johns Hopkins University(约翰霍普金斯大学) Brookhaven National Laboratory(布鲁赫林国家实验室)

AI总结 针对多模态生存分析,提出TTA框架,先基于原型对齐捕获跨模态共享结构,再通过锚定引导对比目标鼓励模态特定独特性,用不平衡最优传输处理模态不平衡和噪声对应,在多个癌症队列上评估,提升了生存预测并揭示可解释模式。

详情
AI中文摘要

多模态生存分析旨在利用组织病理学图像和基因组图谱等异构生物医学数据改善癌症预后。常见策略是跨模态对齐表示以捕获共享信号,但强跨模态对齐会消除对生存预测至关重要的模态特定证据。本文从一个简单观察重新审视多模态生存学习:有效模型应先发现跨模态共享模式,再保留模态特定信号。这激发了“在一起然后分开”的表示学习原则。基于此,我们提出TTA框架,它平衡跨模态对齐和表示独特性。TTA先基于原型对齐捕获模态间共享生存相关结构,再通过锚定引导对比目标鼓励模态特定独特性。为进一步考虑模态不平衡和噪声对应,用不平衡最优传输建模跨模态交互。我们在多个TCGA癌症队列上评估该方法,TTA持续优于近期多模态生存模型,且学习到原型结构揭示了与临床结果相关的可解释跨模态模式。

英文摘要

Multimodal survival analysis aims to improve cancer prognosis using heterogeneous biomedical data, such as histopathology images and genomic profiles. A common strategy is to align representations across modalities so that shared signals can be captured. However, strong cross-modal alignment can also remove modality-specific evidence that is critical for survival prediction. In this paper, we revisit multimodal survival learning from a simple observation: effective models should first discover shared patterns across modalities, and then preserve modality-specific signals. This motivates a representation learning principle that we refer to as Together Then Apart. Based on this idea, we propose TTA, a framework that balances cross-modal alignment and representation distinctiveness. TTA first performs prototype-based alignment to capture shared survival-related structures between modalities. It then encourages modality-specific distinctiveness through an anchor-guided contrastive objective. To further account for modality imbalance and noisy correspondences, we model cross-modal interactions using unbalanced optimal transport. We evaluate the proposed approach on multiple TCGA cancer cohorts with paired histopathology and genomic data. TTA consistently improves survival prediction over recent multimodal survival models. Moreover, the learned prototype structures reveal interpretable cross-modal patterns associated with clinical outcomes.

URL PDF HTML 收藏
2607.11570 2026-07-14 cs.RO cs.HC 新提交

ERR@HRI 3.0 Challenge: Multimodal Detection of Errors and Anticipation in Human-Robot Interactions

ERR@HRI 3.0 挑战赛:人机交互中错误与预期的多模态检测

Maria Teresa Parreira, Micol Spitale, Maia Stiber, Shiye Cao, Amama Mahmood, Chien-Ming Huang, Hatice Gunes, Wendy Ju

机构 * Cornell University(康奈尔大学) Microsoft Research(微软研究院) Johns Hopkins University(约翰霍普金斯大学) University of Cambridge(剑桥大学)

AI总结 ERR@HRI 3.0 挑战赛提供自然场景视频数据集,供研究者开发多模态机器学习模型检测人机交互错误与预期,三个团队提交的有效模型超卷积神经网络基线,为构建相关检测系统提供了数据、任务、基线及结果等参考。

详情
AI中文摘要

随着机器人越来越融入人类环境,其检测和应对错误的能力对于维持用户信任和交互质量至关重要。尽管机器学习的最新进展提高了错误检测能力,但大多数方法仅限于特定上下文、受控设置或预提取特征,限制了其在现实世界条件下的通用性和适用性。为应对这一挑战,ERR@HRI 3.0 挑战赛为研究人员提供了两个互补数据集,以实现人机交互中错误检测和预防方法的端到端创新。挑战赛提供了来自自然场景的原始、非匿名视频数据:(1)旁观者影响检测(BAD)数据集,包含 45 名参与者对机器人和人类失败场景的自发反应的网络摄像头记录;(2)坏主意数据集,包含 29 名参与者在预测失败发生前的行动结果时的预期面部反应。两个数据集均通过众包收集,捕捉了现实世界条件的固有变异性。这种自然变异性虽然具有挑战性,但为开发强大的错误检测系统提供了一个真实的测试平台。参与者开发了用于旁观者反应检测(赛道 1)和预期结果预测(赛道 2)的多模态机器学习模型,以及一个可选的跨数据集泛化赛道(赛道 3)。三个团队提交了有效模型,所有模型均超过了我们的卷积神经网络基线。本文描述了 ERR@HRI 3.0 的数据集、任务、基线和结果,并讨论了对构建用于人机交互的通用、上下文感知和预期错误检测系统的意义。

英文摘要

As robots become increasingly integrated into human environments, their ability to detect and respond to errors remains critical for maintaining user trust and interaction quality. While recent advances in machine learning have improved error detection capabilities, most approaches are limited to specific contexts, controlled settings, or pre-extracted features, limiting their generalizability and applicability to real-world conditions. To address this challenge, the third edition of the ERR@HRI Challenge (ERR@HRI 3.0) provided researchers with two complementary datasets that enable end-to-end innovation in methods for both detecting and preventing errors in human-robot interaction. The challenge offered raw, non-anonymized video data from naturalistic settings: (1) the Bystander Affect Detection (BAD) dataset, containing webcam recordings of 45 participants' spontaneous reactions to robot and human failure scenarios; and (2) the Bad Idea dataset, featuring 29 participants' anticipatory facial responses while predicting action outcomes before failures occur. Both datasets were collected via crowdsourcing, capturing the inherent variability of real-world conditions. This naturalistic variability, while challenging, provides an authentic testbed for developing robust error detection systems. Participants developed multimodal machine learning models for bystander reaction detection (Track 1) and anticipatory outcome prediction (Track 2), with an optional cross-dataset generalization track (Track 3). Three teams submitted valid models, all of which surpassed our convolutional neural network baselines. This paper describes the datasets, tasks, baselines, and results of ERR@HRI 3.0, and discusses implications for building generalizable, context-aware, and anticipatory error detection systems for human-robot interaction.

URL PDF HTML 收藏
2607.11560 2026-07-14 cs.CV cs.AI 新提交

Technical Report on the CVPR 2026@AdvML Workshop Challenge

关于CVPR 2026@AdvML研讨会挑战赛的技术报告

Tianyuan Zhang, Zonglei Jing, Jiangfan Liu, Ligong Zhang, Ke Ma, Chengzhi Sun, Xiaohai Xu, Zhirui Zhang, Qianqian Xu, Qingming Huang, Hanyu Fang, Junhua Liu, Zheng Wang, Xiaoliang Liu, Yuanbo Li, Shuai Gui, Bin Wang, Menghe Zheng, Jing Nie, Hanyang Meng, Zeyang Zhang, Xiang Zhang, Yongxuan Zhu, Rui Ding, Hainan Li, Yongkang Zhang, Zhilei Zhu, Xianglong Kong, Jin Hu, Zonghao Ying, Yisong Xiao, Lei Chen, Haotong Qin, Jiakai Wang, Aishan Liu, Ruikai Li, Julia Karbing, Yinpeng Dong, Zhenfei Yin, Shao Jing, Xia Hu, Jingyi Xu, Juntao Dai, Xinyun Chen, Vishal M. Patel, Xianglong Liu, Dawn Song, Alan Yuille, Philip H. S. Torr, Dacheng Tao

机构 * Beihang University(北京航空航天大学) University of Chinese Academy of Sciences(中国科学院大学) Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所) Tongji University(同济大学) iFLYTEK Co., Ltd.(科大讯飞股份有限公司) Anhui Laboratory for Safe Artificial Intelligence in the Yangtze River Delta(长三角安全人工智能安徽实验室) Wenzhou Business College(温州商学院) Jiangnan University(江南大学) Guangzhou City University of Technology(广州理工学院) Inceptio Technology(智元机器) Institute of Dataspace(数据空间研究所) Zhongguancun Laboratory(中关村实验室) Tsinghua University(清华大学) ETH Zürich(苏黎世联邦理工学院) University of Oxford(牛津大学) Shanghai AI Laboratory(上海人工智能实验室) BAAI(北京智源人工智能研究院) Meta(元公司) Johns Hopkins University(约翰·霍普金斯大学) University of California, Berkeley(加州大学伯克利分校) Nanyang Technological University(南洋理工大学)

AI总结 介绍CVPR 2026@AdvML研讨会针对自动驾驶VLAs的对抗性多模态攻击挑战赛,基于多视图视觉问答,参赛者要生成对抗图像和文本扰动。阐述任务设计等,研究领先提交作品发现后缀惩罚等模式,为多模态自动驾驶系统相关工作提供参考。

详情
AI中文摘要

视觉语言智能体(VLAs)越来越多地用于解释复杂驾驶场景并支持安全关键推理。本报告介绍了针对自动驾驶VLAs的对抗性多模态攻击的CVPR 2026@AdvML研讨会挑战赛。该挑战赛基于DriveLM风格的多视图视觉问答构建,用六个同步相机图像和一组结构化的驾驶相关问答对来表示每个场景。参与者生成对抗性图像和仅后缀的文本扰动,使模型响应偏离参考答案,同时保持图像保真度并限制文本成本。竞赛包括两个阶段,第二阶段增加了一个隐藏的黑盒模型来评估可迁移性。我们描述了任务设计、提交规则、评估协议和排行榜结果,然后研究了五份有技术报告的领先提交作品。在这些报告中出现了几个反复出现的模式:后缀惩罚有利于图像侧攻击;场景级、多视图优化比单独处理视图更有效;问答类型和图结构为分配攻击预算提供了有用的先验信息;特征空间目标可以提高黑盒迁移能力;相机图像中嵌入的排版内容暴露了驾驶VLAs中持续存在的漏洞。这些发现为未来多模态自动驾驶系统的鲁棒性评估和防御设计提供了实际参考。

英文摘要

Vision-language agents (VLAs) are increasingly used to interpret complex driving scenes and support safety-critical reasoning. This report presents the CVPR 2026@AdvML Workshop Challenge on adversarial multimodal attacks against autonomous-driving VLAs. Built on DriveLM-style multi-view visual question answering, the challenge represents each scene with six synchronized camera images and a structured collection of driving-related question-answer pairs. Participants generate adversarial images and suffix-only textual perturbations that induce model responses to deviate from reference answers while preserving image fidelity and limiting textual cost. The competition comprises two phases, with Phase II adding a hidden black-box model to assess transferability. We describe the task design, submission rules, evaluation protocol, and leaderboard results, and then examine five leading submissions for which technical reports were available. Across these reports, several recurring patterns emerge: image-side attacks are favored by the suffix penalty; scene-level, multi-view optimization is more effective than treating views in isolation; QA types and graph structure provide useful priors for allocating attack budget; feature-space objectives can improve black-box transfer; and typographic content embedded in camera images exposes a persistent vulnerability in driving VLAs. These findings provide a practical reference for future robustness evaluation and defense design in multimodal autonomous-driving systems.

URL PDF HTML 收藏
2607.10873 2026-07-14 cs.CV cs.RO eess.IV 新提交

X-GuideAR: An Augmented Reality Framework to Mitigate Radiation Exposure during Fluoroscopic Guidance

X-GuideAR:一种在荧光镜引导下减轻辐射暴露的增强现实框架

Mingxu Liu, Zixuan Liu, Ruchen Cai, Yu-Chen Ku, Suxi Gu, Amit Jain, Alejandro Martin-Gomez, Mehran Armand

机构 * Johns Hopkins University(约翰霍普金斯大学) University of Arkansas(阿肯色大学)

AI总结 研究针对骨科手术中“荧光镜搜索”增加辐射暴露的问题,提出X-GuideAR增强现实框架,通过合成X射线预览加速搜索、减少辐射,经初步研究,该框架在S2AI螺钉放置中减少X射线数量,提高螺钉直径安全性,助力更安全高效手术。

详情
AI中文摘要

在骨科手术中实现最佳螺钉放置需要在X射线下频繁进行对齐检查并获取多个解剖视图,这一过程会增加患者和手术团队的辐射暴露。本文介绍了X-GuideAR,这是一种用于识别最佳X射线视图的增强现实框架,旨在减少辐射暴露并确保准确的螺钉放置。以S2翼状髂骨(S2AI)螺钉放置为例,该系统通过生成合成X射线预览来加速荧光镜搜索,为视图获取和钻孔提供无辐射引导。初步研究表明,使用该系统可使X射线数量减少62.3%。术后评估显示,与传统工作流程相比,使用X-GuideAR的轨迹支持的平均安全螺钉直径更大,表明骨包容性得到改善,具有潜在的生物力学益处。X-GuideAR在减少辐射暴露和简化S2AI螺钉放置方面具有巨大潜力,为更安全、高效的手术提供了一个有前景的方向。

英文摘要

Achieving optimal screw placement for orthopedic surgeries requires frequent alignment checks and multiple anatomical views under X-ray -- a process known as "fluoro-hunting" that increases radiation exposure to patients and surgical teams. This work introduces X-GuideAR, an augmented reality (AR) framework for identifying optimal X-ray views, aimed at reducing radiation exposure while ensuring accurate screw placement. To exemplify the benefits of X-GuideAR, we focus on S2 alar-iliac (S2AI) screw placement. Our system provides radiation-free guidance for view acquisition and drilling by generating synthetic X-ray previews that accelerate fluoro-hunting. Once the required anatomical views are identified using these previews, a real X-ray is acquired, and the preview of the drilling trajectory is augmented onto it, facilitating precise screw placement with minimal additional radiation. A preliminary study involving eight S2AI trajectories performed by an expert spine surgeon demonstrated a 62.3% reduction in the number of X-rays. Post-procedure evaluations showed that trajectories done with X-GuideAR supported an average safe screw diameter of 12.95 mm compared to 5.9 mm under the conventional workflow, suggesting improved bony containment and potential biomechanical benefit. X-GuideAR shows great potential to reduce radiation exposure and streamline S2AI screw placement, offering a promising direction toward safer and more efficient surgeries.

URL PDF HTML 收藏
2607.10608 2026-07-14 cs.AI 新提交

The Compliance Trap: Diagnosing How AI Agents Consume Conflicting Memory

合规陷阱:诊断人工智能代理如何处理冲突记忆

Yixiong Chen, Xinyi Bai, Alan Yuille

机构 * Johns Hopkins University(约翰·霍普金斯大学) Cornell University(康奈尔大学)

AI总结 研究人工智能代理处理冲突记忆的问题,提出E-P-R框架,通过在WebArena和MemTrapBench上实例化发现主要失败始于入口,冲突记忆导致合规陷阱,强调评估记忆增强型代理应考虑其处理记忆的全过程。

详情
AI中文摘要

记忆正成为长期人工智能代理的核心组件,现有工作多将记忆视为供应问题。但我们仍缺乏对模型在多步行动轨迹中如何使用检索到的记忆的清晰认识。为此提出轨迹级框架E-P-R来诊断此过程,在WebArena和MemTrapBench上进行实例化。发现主要失败常始于入口,冲突记忆会导致合规陷阱,各模型的合规率相似,但合规后成功率极低。结果表明,对记忆增强型代理的评估不仅应基于检索质量或最终成功率,还应考虑其在整个轨迹中如何处理记忆。

英文摘要

Memory is becoming a core component of long-horizon AI agents, allowing agents to reuse past experience when operating web browsers, software tools, and other interactive environments. Existing work mostly treats memory as a supply problem, asking what experience to write, how to store it, and which entry to retrieve for the next task. Yet we still lack a clear account of how models consume retrieved memory across a multi-step action trajectory. This consumption process matters because it determines not only what memories should be retrieved, but also what models and control policies are needed to use them safely. To diagnose this process, we propose Entry--Propagation--Recovery (E-P-R), a trajectory-level framework that asks where memory first changes an action, whether that change carries forward, and whether the agent can recover after leaving a correct path. We instantiate E-P-R on WebArena and on MemTrapBench, a controlled benchmark we build to isolate these phases. We find that the main failure often begins at entry: agents adopt conflicting memory at the first exposed decision point even when it is task-wrong. Repeated exposure then amplifies this early error, while recovery after divergence is weak. Together, these effects create a compliance trap: across models, conflicting memory induces similar compliance rates, but once agents comply, their success rates collapse to a low floor. Stronger agents therefore suffer larger absolute damage because each compliance event erases more baseline capability. These results suggest that memory-augmented agents should be evaluated not only by retrieval quality or final success rate, but by how they consume memory throughout the trajectory.

URL PDF HTML 收藏
2607.10601 2026-07-14 cs.AI 新提交

Agentic-DPO: From Imitation to Agentic Policy Optimization on Expert Trajectories

Agentic-DPO:从专家轨迹上的模仿到智能体策略优化

Yixiong Chen, Alan Yuille

机构 * Johns Hopkins University(约翰·霍普金斯大学)

AI总结 研究针对大语言模型智能体基于专家轨迹训练只学动作序列、难应对错误的问题,提出Agentic-DPO方法,通过转化专家轨迹为状态条件偏好监督,结合策略保持增强,实现低成本智能体策略优化,实验验证其有效性。

详情
AI中文摘要

大语言模型智能体通常通过监督微调(SFT)从专家轨迹进行训练,将多轮智能体行为视为普通文本模仿。这种方法简单低成本,但只学习模仿专家动作序列,而非训练智能体在每个状态针对可能的错误选择正确动作。现有缓解此问题的方法包括偏好学习或强化学习,但通常需要高成本的环境展开和奖励模型。我们提出Agentic-DPO,一种轻量级离线智能体策略优化方法,将专家轨迹转化为状态条件偏好监督。在每个专家动作状态,Agentic-DPO从当前状态采样一个单步动作,将可能的错误动作视为负样本,并使用DPO风格的偏好目标与专家动作进行对比。为避免在偏好学习中混合策略和模式,我们引入策略保持增强(PPA),在保持专家策略不变的同时,在多个模式下生成相同的潜在轨迹。Agentic-DPO无需在线环境展开、奖励模型或全轨迹学生探索。我们在StableToolBench、tau-bench零售和Mind2Web上进行实验,Agentic-DPO在不同模型规模下持续改进智能体,超越模仿效果。特别是,对于9B模型,它将tau-bench准确率从21.7%(SFT)提高到41.4%,在相同骨干网络下仅通过步级展开且在梯度步骤中无环境交互就匹配了在线GRPO。结果表明,当专家轨迹从示范转换为状态级动作偏好时,可支持低成本的智能体策略优化。Agentic-DPO的代码在这个https URL发布。

英文摘要

Large Language Model (LLM) agents are commonly trained from expert trajectories using supervised fine-tuning (SFT), which treats multi-turn agent behavior as ordinary text imitation. This recipe is simple and low-cost, but it only learns to imitate the sequence of expert actions, rather than training the agent to choose the right action against plausible mistakes at each state. Existing methods to mitigate this problem include preference learning or reinforcement learning, but they usually need high-cost environment rollouts and reward models. We propose Agentic-DPO, a lightweight offline agent policy optimization method that turns expert trajectories into state-conditioned preference supervision. At each expert action state, Agentic-DPO samples a one-step action from the current state, treats plausible wrong actions as negatives, and contrasts them with the expert action using a DPO-style preference objective. To avoid mixing both policy and schema in preference learning, we introduce Policy-Preserving Augmentation (PPA), which renders the same latent trajectory under multiple schemas while keeping the expert policy fixed. Agentic-DPO requires no online environment rollout, reward model, or full-trajectory student exploration. We conduct experiments across StableToolBench, tau-bench retail, and Mind2Web, where Agentic-DPO consistently improves agents at different model scales beyond imitation. In particular, it raises tau-bench accuracy from 21.7% (SFT) to 41.4% for a 9B model, matching online GRPO under the same backbone with only step-level rollouts and without environment interaction during gradient steps. The results suggest that expert trajectories can support low-cost agentic policy optimization when converted from demonstrations into state-level action preferences. Code for Agentic-DPO is released at https://github.com/Schuture/Agentic-DPO.

URL PDF HTML 收藏
2607.10074 2026-07-14 cs.LG 新提交

Distance-Preserving Embeddings in Inhomogeneous Random Graphs

非均匀随机图中的距离保持嵌入

My Le, Luana Ruiz, Souvik Dhara

机构 * Johns Hopkins University(约翰斯·霍普金斯大学) Georgia Institute of Technology(佐治亚理工学院)

AI总结 研究非均匀随机图中距离保持嵌入,基于地标嵌入分析最短路径近似,扩展失真保证到全局平均并统一分析,引入GNN增强变体,使模型能从小图学习并推广到大型真实网络,保持或超越经典嵌入保真度。

详情
AI中文摘要

图机器学习为理解复杂网络和学习有意义的节点表示提供了强大工具。然而,设计能最小化局部和全局功能(如最短路径长度)失真的嵌入是一个核心挑战。以往距离保持嵌入的失真保证本质上是最坏情况,产生过于悲观的界限。为解决此问题,我们分析了基于地标嵌入在非均匀随机图上的最短路径近似。通过保留到一小组称为地标的参考节点的最短路径,基于地标的方法有效地充当虚拟图扳手。我们将这些保证扩展到全局、组件范围的平均值,并通过新颖的度量三明治框架统一了有限类型和连续潜在空间的分析。最后,我们引入了一种GNN增强变体,用灵活的、结构感知的神经替代物取代了刚性的、计算昂贵的精确最短路径查询。

英文摘要

Graph machine learning provides powerful tools for understanding complex networks and learning meaningful node representations. A central challenge, however, is designing embeddings with minimal distortion of both local and global functionals, such as shortest path lengths. Prior distortion guarantees for distance-preserving embeddings are worst-case in nature, producing overly pessimistic bounds that fail to capture the structure of typical large-scale networks. To address this, we analyze shortest-path approximation via landmark-based embeddings on inhomogeneous random graphs, a general model with type-dependent edge probabilities. By retaining shortest paths to a small set of reference nodes called landmarks, landmark-based methods effectively function as virtual graph spanners, where structural heterogeneity and controlled neighborhood expansion modeled via multi-type branching processes enable significantly tighter dimension-distortion trade-offs than classical worst-case bounds. We extend these guarantees to global, component-wide averages and unify the analysis across finite-type and continuous latent spaces through a novel metric sandwiching framework, establishing universal distortion bounds for general $L^2$ kernel models, including heavy-tailed and power-law networks. Finally, we introduce a GNN-augmented variant that replaces rigid, computationally expensive exact shortest-path queries with flexible, structure-aware neural surrogates. By leveraging the inherent alignment between graph neural message-passing and the dynamic programming principles of shortest-path algorithms, our approach demonstrates that models trained on small-scale random graphs learn to extract universal distance-preserving features, achieving robust generalization to large-scale, real-world networks that match or exceed the fidelity of classical, exact landmark-based embeddings.

URL PDF HTML 收藏
2606.27537 2026-07-14 cs.CV 版本更新

MemoBench: Benchmarking World Modeling in Dynamically Changing Environments

MemoBench: 动态变化环境中的世界建模基准测试

Haoyu Chen, Kaichen Zhou, Hang Hua, Kaile Zhang, Jingwen Qian, Wufei Ma, Haonan Chen, Chunjiang Liu, Yizhou Zhao, Xiaoyuan Wang, Weiyue Li, Alan Yuille, Paul Pu Liang, Yilun Du

机构 * Harvard University(哈佛大学) MIT(麻省理工学院) MIT-IBM Watson AI Lab(MIT-IBM沃森人工智能实验室) Boston University(波士顿大学) Google(谷歌) JHU(约翰霍普金斯大学) CMU(卡内基梅隆大学) Kempner Institute(肯普纳研究所)

AI总结 提出MemoBench基准,通过目标消失-重现范式评估视频生成模型在动态环境中的记忆一致性,涵盖合成与真实场景,揭示现有模型的关键挑战。

详情
AI中文摘要

视频生成模型旨在模拟动态环境,已有多个基准测试评估帧间的记忆一致性。然而,大多数基准仅在目标保持在视野内时评估一致性,少数迫使目标离开视野的基准则评估遮挡期间无变化的静态场景。为弥补这一差距,我们引入了MemoBench,这是一个围绕动态变化环境中消失-重现范式构建的诊断基准:目标对象经历物理过程,从视野中消失,并必须在重新出现时以更新后的状态正确恢复。我们整理了涵盖合成和真实场景的360个真实剪辑,并设计了一个评估套件,结合自动指标和基于VQA的评估,涵盖四个诊断支柱。对八个最先进模型的评估揭示了在消失-重现范式下关于记忆一致性的关键见解和开放挑战。

英文摘要

Video generation models aspire to simulate dynamic environments, and several benchmarks now evaluate memory consistency across frames. However, most assess consistency only while the target remains in view, and the few that force objects out of view evaluate static scenes where nothing changes during occlusion. To bridge this gap, we introduce MemoBench, a diagnostic benchmark built around the disappear-and-reappear paradigm in dynamically changing environments: a target object undergoes a physical process, disappears from view, and must be correctly recovered in its updated state upon reappearance. We curate 360 ground-truth clips spanning synthetic and real-world scenes, and design an evaluation suite combining automated metrics with VQA-based assessment across four diagnostic pillars. Evaluation of eight state-of-the-art models reveals key insights and open challenges regarding memory consistency under the disappear-and-reappear paradigm.

URL PDF HTML 收藏
2606.23525 2026-07-14 cs.CL 版本更新

Self-Compacting Language Model Agents

自压缩语言模型智能体

Tianjian Li, Jingyu Zhang, William Jurayj, Xi Wang, Chuanyang Jin, Mehrdad Farajtabar, Eric Nalisnick, Daniel Khashabi

机构 * Johns Hopkins University(约翰霍普金斯大学) Apple(苹果公司)

AI总结 提出SelfCompact框架,让语言模型自主决定何时及如何压缩上下文,通过调用压缩工具和轻量级规则实现自适应压缩,无需微调或外部监督,在多个基准上以更低成本匹配或超越固定间隔压缩。

Comments 25 pages, 3 figures

详情
AI中文摘要

由思维链和工具调用组成的长智能体轨迹会积累陈旧内容,锚定后续生成,并最终超出上下文窗口。现有框架通过触发在令牌阈值处的固定间隔压缩来缓解此问题。这种触发不关注轨迹结构,有在推导或搜索过程中丢弃部分结果的风险。我们提出SelfCompact,一种允许模型自身决定何时以及如何压缩的框架。具体来说,它配对两个推理时元素:(i) 模型调用的压缩工具,用于总结累积的上下文,以及(ii) 一个轻量级规则,指定何时触发(子任务已解决,或轨迹正在收敛)以及何时抑制(推导中,或卡住时)。两者都是必需的。仅工具在开放权重模型中使用不均,常常在无帮助的时刻被调用或根本不调用;仅规则无法行动。两者结合,无需任何微调或外部监督即可引发有效的自适应压缩。我们在六个基准(竞赛数学和智能体搜索)和七个模型上展示了实证结果。我们的结果表明,SelfCompact以更低的令牌成本匹配或超过固定间隔总结,在数学上比无总结基线提高最多18.1个百分点,在智能体搜索上提高5-9个百分点,同时每个问题的成本降低30-70%。我们的结果揭示了一个元认知差距:尽管未提示的模型无法可靠地判断其上下文何时在腐化,但一个轻量级规则弥合了这一差距,将何时压缩重新定义为框架无需训练即可提供的能力。

英文摘要

Long agent traces composed of chains of thought and tool calls accumulate stale content that anchor subsequent generations, and eventually outgrow the context window. Existing scaffolds mitigate it with fixed-interval compaction triggered at a token threshold. Such triggers pay no heed to trajectory structure, risking discard of partial results mid-derivation or mid-search. We propose SelfCompact, a scaffold that allows the model itself to decide when and how to compact. Specifically, it pairs two inference-time elements: (i) a compaction tool the model invokes to summarize the accumulated context, and (ii) a lightweight rubric specifying when to fire (a sub-task has resolved, or the trajectory is converging) and when to suppress (mid-derivation, or when stuck). Both are needed. The tool alone is unevenly used across open-weight models, often invoked at unhelpful moments or not at all; the rubric alone cannot act. Together, they elicit effective adaptive compaction without any fine-tuning or external supervision. We present empirical results on six benchmarks (competitive math and agentic search) and seven models. Our results show that SelfCompact matches or exceeds fixed-interval summarization at a fraction of the token cost, improving over a no-summarization baseline by up to 18.1 points on math and 5-9 points on agentic search at 30-70% lower per-question cost. Our results expose a meta-cognitive gap: although unprompted models cannot reliably tell when their own context is rotting, a lightweight rubric closes this gap, reframing when to compact as a capability that scaffolds can supply without training.

URL PDF HTML 收藏
2505.21649 2026-07-14 cs.CV

Seeing Isn't Orienting: A Cognitively Informed Hierarchical Benchmark for Object Orientation in MLLMs

看到并不等于定向:一个认知基础的基准揭示了多模态大语言模型在定向任务中的系统性失败

Nazia Tasnim, Keanu Nichols, Yuting Yan, Nicholas Ikechukwu, Elva Zou, Deepti Ghadiyaram, Bryan A. Plummer

机构 * Boston University(波士顿大学) Johns Hopkins University(约翰霍普金斯大学)

AI总结 本文提出DORI基准,揭示多模态大语言模型在定向理解上的不足,发现其在角度估计和方向跟踪方面存在系统性缺陷,对机器人控制和3D场景重建有重要启示。

Comments Paper accepted to ECCV 2026 (Main Track). Previously, this version appeared as arXiv:2603.11410 which was submitted as a new work by accident

详情
AI中文摘要

物体定向理解是视觉感知中的基本挑战,对机器人操控和增强现实等应用至关重要。当前的视觉-语言基准未能孤立这一能力,常将定向能力与位置关系和一般场景理解混淆。我们引入DORI(Discriminative Orientation Reasoning Intelligence),一个全面的基准,将物体定向感知作为主要评估目标。DORI评估定向理解的四个维度:正面对齐、旋转变换、相对方向关系和标准定向理解。通过精心编纂的11个数据集中的任务,涵盖67个物体类别,跨越合成和现实场景,DORI提供了多模态系统理解物体方向的见解。对15个最先进的视觉-语言模型的评估揭示了关键限制:即使最好的模型在粗粒度任务上也只达到54.2%的准确率,在细粒度方向判断上仅为33.0%,且在需要参考系转换或复合旋转的任务中表现更差。这些发现表明需要专门的定向表示机制,因为模型在精确角度估计、跨视角方向跟踪和复合旋转理解方面存在系统性不足,暗示其内部3D空间表示的局限性。作为首个专门设计用于多模态系统定向意识的诊断框架,DORI对改进机器人控制、3D场景重建和物理环境中的人机交互具有重要启示。DORI数据:https://huggingface.co/datasets/appledora/DORI-Benchmark

英文摘要

Humans develop object orientation understanding progressively, from recognizing which way an object faces to reasoning about orientations across multiple objects. Yet existing vision-language benchmarks largely conflate orientation with broader spatial reasoning. We introduce Discriminative Orientation Reasoning Intelligence (DORI), a cognition-informed hierarchical benchmark that establishes object orientation as the primary evaluation target. DORI decomposes orientation into four dimensions, each evaluated at coarse (categorical) and granular (metric) levels, yielding 33,656 multiple-choice questions over 13,652 real-world and synthetic images from 14 sources. Its design isolates orientation from confounding factors through object isolation, standardized reference frames, and structured prompts. Evaluating 26 state-of-the-art vision-language models reveals a consistent limitation: models strong on general spatial benchmarks remain near-random on object-centric orientation reasoning. The best model achieves only 64.2\% on coarse and 42.9\% on granular judgments, with the largest drops on compound rotations and inter-object reference frame shifts. Large coarse-to-granular performance gaps further indicate reliance on categorical heuristics rather than geometric reasoning. These results establish object orientation understanding as a fundamental open challenge for multimodal systems. Dataset: https://huggingface.co/datasets/appledora/DORI-Benchmark

URL PDF HTML 收藏
2603.06587 2026-07-14 cs.AI q-fin.CP q-fin.RM

Autonomous AI Agents for Option Hedging: Enhancing Financial Stability through Shortfall Aware Reinforcement Learning

自主AI代理用于期权对冲:通过意识短缺的强化学习增强金融稳定性

Minxuan Hu, Ziheng Chen, Jiayu Yi, Wenxi Sun

机构 * Cornell Ann S. Bowers College of Computing and Information Science, Cornell University(康奈尔大学安·索斯·博尔斯计算与信息科学学院) Department of Mathematics, University of Texas at Austin(德克萨斯大学奥斯汀分校数学系) School of Social Sciences, Nanyang Technological University(南洋理工大学社会科学学院) Krieger School of Arts and Sciences, Johns Hopkins University(约翰霍普金斯大学克里eger艺术与科学学院)

AI总结 本文提出RLOP和QLBS两种强化学习框架,通过意识短缺的方法优化期权对冲,提升金融稳定性并改进风险预测。

详情
AI中文摘要

在衍生品市场中部署自主AI代理扩大了静态模型校准与实际对冲结果之间的实践差距。我们引入了两种强化学习框架,一种是新的期权定价复制学习(RLOP)方法和Black-Scholes中Q学习的自适应扩展(QLBS),这些方法优先考虑短缺概率,并将学习目标与敏感于下行的对冲对齐。使用上市的SPY和XOP期权,我们通过实际路径delta对冲结果分布、短缺概率和尾部风险措施如预期短缺来评估模型。实证研究表明,RLOP在大多数切片中减少了短缺频率,并在压力测试中显示出最明显的尾部风险改进,而隐含波动率拟合通常倾向于参数模型,但预测实际成本对冲表现较差。这种摩擦意识的强化学习框架支持了在AI增强的交易系统扩展时的实用衍生品风险管理方法。

英文摘要

The deployment of autonomous AI agents in derivatives markets has widened a practical gap between static model calibration and realized hedging outcomes. We introduce two reinforcement learning frameworks, a novel Replication Learning of Option Pricing (RLOP) approach and an adaptive extension of Q-learner in Black-Scholes (QLBS), that prioritize shortfall probability and align learning objectives with downside sensitive hedging. Using listed SPY and XOP options, we evaluate models using realized path delta hedging outcome distributions, shortfall probability, and tail risk measures such as Expected Shortfall. Empirically, RLOP reduces shortfall frequency in most slices and shows the clearest tail-risk improvements in stress, while implied volatility fit often favors parametric models yet poorly predicts after-cost hedging performance. This friction-aware RL framework supports a practical approach to autonomous derivatives risk management as AI-augmented trading systems scale.

URL PDF HTML 收藏
2602.02905 2026-07-14 cs.AI 版本更新

FIRE-Bench: Evaluating AI Agents on the Rediscovery of Scientific Insights

FIRE-Bench:评估人工智能代理在科学见解重新发现方面的表现

Zhen Wang, Fan Bai, Zhongyan Luo, Jinyan Su, Kaiser Sun, Xinle Yu, Jieyuan Liu, Kun Zhou, Claire Cardie, Mark Dredze, Zhiting Hu, Eric P. Xing

机构 * Johns Hopkins University(约翰霍普金斯大学) Cornell University(康奈尔大学)

AI总结 研究旨在评估大语言模型驱动的人工智能代理在科学见解重新发现上的能力。核心方法是引入FIRE-Bench基准,让代理基于高层次研究问题进行全周期探索。主要贡献是为衡量代理驱动的科学发现进展提供了严格诊断框架,揭示当前代理系统在全周期科研上的挑战。

Comments 34 pages, 3 figures, 16 tables; ICML 2026 Camera-ready Version

详情
AI中文摘要

由大语言模型驱动的自主代理有望加速科学发现,但严格评估其可验证发现的能力仍是一大挑战。现有基准面临权衡。为此,我们引入FIRE-Bench基准,通过重新发现近期高影响力机器学习研究中的既定发现来评估代理。代理仅获一个高层次研究问题,需自主探索、设计实验等。我们用前沿大语言模型骨干评估了一系列代理,结果表明全周期科学研究对当前代理系统仍具挑战性,FIRE-Bench为衡量可靠代理驱动的科学发现进展提供了严格诊断框架。

英文摘要

Autonomous agents powered by large language models (LLMs) promise to accelerate scientific discovery end-to-end, but rigorously evaluating their capacity for verifiable discovery remains a central challenge. Existing benchmarks face a trade-off: they either heavily rely on LLM-as-judge evaluations of automatically generated research outputs or optimize convenient yet isolated performance metrics that provide coarse proxies for scientific insight. To address this gap, we introduce FIRE-Bench (Full-cycle Insight Rediscovery Evaluation), a benchmark that evaluates agents through the rediscovery of established findings from recent, high-impact machine learning research. Agents are given only a high-level research question extracted from a published, verified study and must autonomously explore ideas, design experiments, implement code, execute their plans, and derive conclusions supported by empirical evidence. We evaluate a range of state-of-the-art agents with frontier LLMs backbones like gpt-5 on FIRE-Bench. Our results show that full-cycle scientific research remains challenging for current agent systems: even the strongest agents achieve limited rediscovery success (<50 F1), exhibit high variance across runs, and display recurring failure modes in experimental design, execution, and evidence-based reasoning. FIRE-Bench provides a rigorous and diagnostic framework for measuring progress toward reliable agent-driven scientific discovery.

URL PDF HTML 收藏
2512.10825 2026-07-14 math.ST cs.LG math.OC stat.TH 版本更新

An Elementary Proof of the Near Optimality of LogSumExp Smoothing

对数和指数平滑近似最优性的一个初等证明

Thabo Samakhoana, Benjamin Grimmer

机构 * Johns Hopkins University(约翰霍普金斯大学)

AI总结 研究 \(\mathbb{R}^d\) 中最大值函数平滑设计,通过初等构造给出下界,证明对数和指数平滑近似最优,同时给出更强平滑说明其非完全最优,在低维提出恰好最优平滑。

Comments 15 pages

详情
AI中文摘要

我们考虑在无穷范数下,\(\mathbb{R}^d\) 中(按坐标)最大值函数的平滑设计。对数和指数函数 \(f(x)=\ln(\sum^d_i\exp(x_i))\) 提供了一种经典平滑,与最大值函数的值相差至多 \(\ln(d)\)。我们给出一个下界的初等构造,表明最大值函数的每个高估平滑至少相差约 \(0.8145\ln(d)\)。因此,对数和指数平滑在小常数因子范围内是最优的。然而,我们给出了更强的平滑,表明基于熵的对数和指数方法并非完全最优。在低维情况下,我们提出了恰好最优的平滑,达到了我们的下界。

英文摘要

We consider the design of smoothings of the (coordinate-wise) max function in $\mathbb{R}^d$ in the infinity norm. The LogSumExp function $f(x)=\ln(\sum^d_i\exp(x_i))$ provides a classical smoothing, differing from the max function in value by at most $\ln(d)$. We provide an elementary construction of a lower bound, establishing that every overestimating smoothing of the max function must differ by at least $\sim 0.8145\ln(d)$. Hence, LogSumExp is optimal up to small constant factors. However, we provide strictly stronger smoothings showing the entropy-based LogSumExp approach is not exactly optimal. In small dimensions, we propose exactly optimal smoothings, attaining our lower bound.

URL PDF HTML 收藏
2601.06896 2026-07-14 eess.AS cs.CL 版本更新

TagSpeech: End-to-End Multi-Speaker ASR and Diarization with Fine-Grained Temporal Grounding

TagSpeech:基于细粒度时间定位的端到端多说话人自动语音识别与说话人分离

Mingyue Huo, Yiwen Shao, Yuheng Zhang

机构 * University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校) Johns Hopkins University(约翰霍普金斯大学)

AI总结 研究提出TagSpeech框架,利用时间锚点定位技术联合多说话人ASR与说话人分离。通过关键设计解决细粒度对齐挑战,端到端建模“谁在何时说了什么”。实验显示其在DER上优于基线,采用参数高效训练范式,低计算成本获强性能。

Comments Accepted to ACL2026 Main Oral; v2: added overlap analysis in Section 5.2

详情
AI中文摘要

我们提出了TagSpeech——一个统一的基于大语言模型(LLM)的框架,它利用时间锚点定位技术进行联合多说话人自动语音识别(ASR)和说话人分离。该框架基于两个关键设计:(1)通过序列化输出训练(SOT)微调解耦的语义和说话人流,以学习轮流发言动态;(2)交错时间锚点机制,不仅支持细粒度时间戳预测,还作为语义理解和说话人跟踪之间的同步信号。与之前主要关注说话人归因ASR或隐式说话人分离的工作相比,TagSpeech解决了细粒度说话人-内容对齐的挑战,并以端到端方式明确建模“谁在何时说了什么”。在AMI和阿里会议基准测试上的实验表明,我们的方法在说话人分离错误率(DER)方面比强大的端到端基线(包括Qwen-Omni和Gemini)有持续改进,尤其在处理复杂语音重叠方面。此外,TagSpeech采用参数高效的训练范式,其中LLM主干被冻结,只训练轻量级投影仪,以低计算成本实现强大性能。

英文摘要

We present TagSpeech, a unified LLM-based framework that utilizes Temporal Anchor Grounding for joint multi-speaker ASR and diarization. The framework is built on two key designs: (1) decoupled semantic and speaker streams fine-tuned via Serialized Output Training (SOT) to learn turn-taking dynamics; and (2) an interleaved time anchor mechanism that not only supports fine-grained timestamp prediction but also acts as a synchronization signal between semantic understanding and speaker tracking. Compared to previous works that primarily focus on speaker-attributed ASR or implicit diarization, TagSpeech addresses the challenge of fine-grained speaker-content alignment and explicitly models "who spoke what and when" in an end-to-end manner. Experiments on AMI and AliMeeting benchmarks demonstrate that our method achieves consistent improvements in Diarization Error Rate (DER) over strong end-to-end baselines, including Qwen-Omni and Gemini, particularly in handling complex speech overlaps. Moreover, TagSpeech employs a parameter-efficient training paradigm in which the LLM backbone is frozen and only lightweight projectors are trained, resulting in strong performance with low computational cost.

URL PDF HTML 收藏
2510.09497 2026-07-14 cs.RO cs.AI 版本更新

Toward Autonomous Soft Robotic Endovascular Navigation via Imitation Learning

通过模仿学习实现自主软机器人血管内导航

Noah Barnes, Ji Woong Kim, Lingyun Di, Hannah Qu, Anuruddha Bhattacharjee, Miroslaw Janowski, Dheeraj Gandhi, Bailey Felix, Shaopeng Jiang, Olivia Young, Mark Fuge, Ryan D. Sochol, Jeremy D. Brown, Axel Krieger

机构 * Johns Hopkins University(约翰霍普金斯大学) Stanford University(斯坦福大学) McGill University(麦吉尔大学) University of Maryland(马里兰大学) Swiss Federal Institute of Technology in Lausanne (EPFL)(日内瓦联邦理工学院(EPFL)) ETH Zurich(苏黎世联邦理工学院)

AI总结 研究旨在实现自主软机器人血管内导航,开发基于变压器的模仿学习框架,通过目标条件等实现通用导航,在多种几何结构上训练并评估,策略成功率高,还进行了相关研究及扩展,提升了在未见几何结构上的成功率。

详情
AI中文摘要

在血管内手术中,介入医生在细钢丝引导下将导管推送至患者血管内的治疗部位。机器人导丝虽能增强可操作性,但建模与控制困难。自主软机器人导丝导航有望克服这些挑战,提高血管内导航的精度和安全性。作为第一步,我们建立了用于自主导航的大规模二维投影环境。我们开发了基于变压器的模仿学习框架,通过目标条件、相对动作输出和自动造影剂注射,在动脉瘤靶向任务中实现通用的软机器人导航。我们在36种不同的模块化分叉几何结构上训练策略,在模拟荧光透视下生成647个示范,并在三种未见的血管几何结构上进行评估。该策略在未见几何结构上到达动脉瘤的成功率为83%,优于多个基线。此外,消融和基线研究评估了每个设计和数据收集选择的有效性。最后,我们将该策略扩展到在一个未见的患者来源几何结构上实现75%的成功率。

英文摘要

In endovascular surgery, endovascular interventionists push a thin tube called a catheter, guided by a thin wire to a treatment site inside the patient's blood vessels to treat various conditions such as blood clots, aneurysms, and malformations. Robotic guidewires can enhance maneuverability but are difficult to model and control. Autonomous soft robotic guidewire navigation has the potential to overcome these challenges, increasing the precision and safety of endovascular navigation. As a first step, we establish a large-scale, 2D-projected environment for autonomous navigation. In other surgical domains, end-to-end imitation learning has shown promising results. Thus, we develop a transformer-based imitation learning framework with goal conditioning, relative action outputs, and automatic contrast dye injections to enable generalizable soft robot navigation in an aneurysm targeting task. We train the policy on 36 different modular bifurcated geometries, generating 647 total demonstrations under simulated fluoroscopy, and evaluate it on three previously unseen vascular geometries. The policy reaches the aneurysm with a success rate of 83% on the unseen geometries, outperforming several baselines. In addition, ablation and baseline studies evaluate the effectiveness of each design and data collection choice. Lastly, we extend the policy to achieve 75% success on an unseen patient-derived geometry. Project website: https://softrobotnavigation.github.io/

URL PDF HTML 收藏