arXivDaily arXiv每日学术速递 周一至周五更新

高校专区

Zhejiang University(浙江大学)

至 收录 2867
2607.18198 2026-07-21 cs.LG cs.CV 新提交

Three-Body Scattering for Generative Modeling

用于生成建模的三体散射

Peng Sun, Zhenglin Cheng, Deyuan Liu, Jun Xie, Xinyi Shang, Tao Lin

机构 * Westlake University(西湖大学) Zhejiang University(浙江大学) University College London(伦敦大学学院)

AI总结 研究提出三体散射建模(TBSM)用于生成,将能量距离转化为特定相互作用,为单步生成器提供监督。通过在线跟踪条件期望减少噪声,在ImageNet-256上训练单步生成器取得良好结果,确立了跟踪散射作为高维单步生成的途径。

Comments 31 pages, 5 figures, and 4 tables. Code: https://github.com/sp12138/TBSM

详情
AI中文摘要

现代生成模型通常依赖对抗性判别器、规定的噪声到数据路径或自回归分解。相反,我们表明适当的分布能量可以诱导样本级运动,并为单步生成器提供直接回归监督。用于生成的三体散射建模(TBSM)将能量距离转化为每个射弹的固定大小相互作用:每个射弹被一个真实源吸引并被一个独立生成的源排斥。基于射弹及其条件,其期望等于$\frac12D_E^2(P_{\theta},Q)$的2-瓦瑟斯坦梯度流速度。一批$B$个冻结目标事件产生$O(B)$个样本级损失,每个损失使用一个参考作为其条件,而不是像漂移模型等方法使用的小批量全对场。在线跟踪这种条件期望可以减少场噪声。使用冻结图像特征中的散射,TBSM在ImageNet-256上训练单步生成器,在NFE = 1时,像素空间的PixelDiT-XL实现FID = 2.23,潜在空间的DiT-XL实现FID = 1.63。我们提供了一个设计图,将扩散相关监督、类似漂移的动力学和类似GAN的目标联系起来。这些结果将跟踪散射确立为高维单步生成的一条途径。代码:此https URL。

英文摘要

Modern generative models typically rely on an adversarial critic, a prescribed noise-to-data path, or an autoregressive factorization. Instead, we show that a proper distributional energy can induce sample-level motion and provide direct regression supervision for a one-step generator. Three-Body Scattering Modeling (TBSM) for generation turns the energy distance into a constant-size per-projectile interaction: each projectile is attracted toward one real source and repelled from one independently generated source. Conditioned on the projectile and its condition, its expectation equals the $2$-Wasserstein gradient-flow velocity of $\frac12D_E^2(P_θ,Q)$. A batch of $B$ frozen-target events yields $O(B)$ sample-level losses, each using one reference for its condition instead of the minibatch-wide all-pairs field used by methods such as Drifting Models. Tracking this conditional expectation online can reduce field noise. Using scattering in frozen image features, TBSM trains one-step generators on ImageNet-256, achieving FID${}=2.23$ with pixel-space PixelDiT-XL and FID${}=1.63$ with latent-space DiT-XL at NFE${}=1$. We provide a design map relating diffusion-related supervision, Drift-like dynamics, and GAN-like objectives. These results establish tracked scattering as a route to high-dimensional one-step generation. Code: https://github.com/sp12138/TBSM.

URL PDF HTML 收藏
2607.18154 2026-07-21 cs.RO 新提交

World Translation: Minimizing Sim-to-Real Gap with Backward Dynamics Extraction and Unpaired Domain Translation

世界翻译:通过反向动力学提取和非配对域翻译最小化模拟与现实之间的差距

Xinchen Yao, Leixin Chang, Hua Chen

机构 * Zhejiang University(浙江大学) LimX Dynamics(LimX动力学公司)

AI总结 研究模拟与现实差距问题,提出“世界翻译”方法,利用模拟器和学习动力学互补优势,反向提取动力学信息并跨域转换,实验表明该方法能更准确建模,在特定情况下优势明显,还证实可改进策略转移。

Comments 8 pages, 8 figures

详情
AI中文摘要

在现实世界中部署经过模拟训练的机器人策略时,模拟与现实之间的差距仍然是一个基本挑战。实到模拟方法从现实角度缩小了这一差距,从现实数据中学习过渡动力学以构建更逼真的数字世界,其中学习到的动力学模型是主要实例。然而,此类方法面临部分可观测性问题,相同观测可能因不可观测因素而导致不同过渡。现有方法假定这些因素可从观测历史中恢复,但在观测历史无信息时可能失败。为解决此限制,我们提出了“世界翻译”,利用模拟器和学习到的动力学的互补优势。我们从观测到的过渡中反向提取不可观测的动力学信息,然后将此特征作为非配对域翻译问题在模拟与现实之间进行转换,以在转移域风格的同时保留动力学内容。在人形机器人、四足机器人和操纵器平台上的实验表明,我们的方法比基线实现了更准确的动力学建模,在无法从观测历史中恢复不可观测因素时收益最大。在Go2四足机器人上的实际机器人部署证实了改进的策略转移。

英文摘要

The gap between simulation and reality remains a fundamental challenge in deploying simulation-trained robotic policies in the real world. Real-to-sim methods narrow this gap from the real side, learning transition dynamics from real data to build a more realistic digital world. Learned dynamics models are their dominant instance. Such methods, however, face a partial observability problem: the same observation may branch to different transitions due to unobservable factors. Existing methods assume these factors can be recovered from observation history. However, this may fail whenever observation history is uninformative, such as a sudden contact event with no prior warning. To address this limitation, we propose \textit{World Translation}, which exploits a complementary strength of simulators and learned dynamics. Simulators are deterministic but physically imperfect, while learned models are accurate but underdetermined under partial observability. Rather than predicting transitions forward from history, we extract the unobservable dynamics information backward from an observed transition, then translate this feature across simulation and reality as an unpaired domain-translation problem that preserves dynamics content while transferring domain style. Experiments across humanoid, quadruped, and manipulator platforms show that our method achieves more accurate dynamics modeling than baselines, with the largest gains when unobservable factors cannot be recovered from observation history. Real-robot deployment on Go2 quadruped confirms improved policy transfer.

URL PDF HTML 收藏
2607.18091 2026-07-21 cs.CV cs.GR cs.LG 新提交

SciForma: Structure-Faithful Generation of Scientific Diagrams

SciForma:科学图表的结构忠实生成

Yuxuan Luo, Peng Zhang, Xinjie Zhang, Xun Guo, Zhouhui Lian, Yan Lu

机构 * Wangxuan Institute of Computer Technology, Peking University(北京大学王选计算机技术研究所) State Key Lab of CAD & CG, Zhejiang University(浙江大学CAD&CG国家重点实验室) Microsoft Research Asia(微软亚洲研究院)

AI总结 研究针对科学图表结构保真度问题,提出SciForma框架,分解图表质量为三个结构轴,用M-DPO优化,策划训练和评估数据,实现迭代编辑,使SciForma-9B超越开源基线和GPT-Image-1.5,提升科学图表生成的结构保真度。

Comments 30 pages, 21 figures

详情
AI中文摘要

结构保真度对于科学方法图表至关重要。为传达研究逻辑,此类图表必须如实呈现组件、方向关系和文本注释。由于单个错误(如箭头反向或方程式不可读)会使整个图表无效,结构保真度本质上是联合性的。当前开源模型不满足此标准。监督微调(SFT)学习合理布局但无法可靠确保结构正确性,基于标量奖励的训练后处理会掩盖失败的结构维度。为解决此问题,我们引入SciForma框架。它将图表质量分解为组件、箭头和文本三个结构轴,基于此策划了用于结构化训练的SciFormaData-700K和用于逻辑验证评估的SciFormaBench-2K。我们开发了多维联合偏好优化(M-DPO),在训练后跨所有轴强制同时正确,并将梯度自适应路由到最不足的维度。推理时可进行迭代编辑以纠正残留错误。这使得SciForma-9B在SciFormaBench-2K和AIBench上超越所有开源基线和GPT-Image-1.5,使开放科学图表生成接近专有级别的结构保真度。

英文摘要

Structural fidelity is essential to scientific methodology diagrams. To communicate research logic, these diagrams must faithfully render components, directional relations, and textual annotations. Since a single error, such as a reversed arrow or an unreadable equation, can invalidate the entire figure, structural fidelity is inherently conjunctive: correctness on one axis cannot compensate for failure on another. Current open-source models fail to satisfy this criterion. Supervised fine-tuning (SFT) learns plausible layouts but cannot reliably ensure structural correctness, while scalar reward-based post-training obscures which structural dimension has failed. To address this, we introduce SciForma, a framework for the structure faithful generation of scientific methodology diagrams. Specifically, SciForma decomposes diagram quality into three structural axes: Component, Arrow, and Text, guided by a structural inventory. Built on this foundation, we curate SciFormaData-700K for structured training and SciFormaBench-2K for logic-verified evaluation. To close the gap left by SFT, we develop Multi-Dimensional Conjunctive Preference Optimization (M-DPO), which enforces simultaneous correctness across all axes and adaptively routes gradients to the most deficient dimension in post-training. The same structural inventory also enables iterative editing at inference time to correct residual errors. This combination allows SciForma-9B to exceed all open-source baselines and GPT-Image-1.5 on both SciFormaBench-2K and AIBench, bringing open scientific diagram generation close to proprietary-level structural fidelity. Our code and data will be available at: https://github.com/microsoft/SciForma.

URL PDF HTML 收藏
2607.18082 2026-07-21 cs.LG cs.AI 新提交

Enhancing Rubric-based RL via Self-Distillation

通过自蒸馏增强基于评分标准的强化学习

Mingxuan Xia, Yuhang Yang, Chao Ye, Shuai Zhu, Shenzhi Yang, Guangcheng Zhu, Yuhang Zhang, Cheng Peng, Haobo Wang, Siqing Wang

机构 * Zhejiang University(浙江大学) ByteDance(字节跳动)

AI总结 研究基于评分标准的强化学习中探索有限问题,提出标准蒸馏策略优化(CriPO),通过策略内自蒸馏同时解决未探索标准和被抑制标准问题,在医学和科学基准测试中表现出色,减少优化步骤并提升性能。

详情
AI中文摘要

基于评分标准的强化学习在改进大语言模型的开放式任务中展现出潜力。其公认的局限是探索有限,未探索标准(UC)未获优化信号。近期方法通过在展开过程中纳入评分标准信息解决此问题,但引入了训练与推理不匹配。此外,这些方法忽视了被抑制标准(SC)这一失败模式。我们的分析表明SC很普遍。为同时解决UC和SC且不引入训练推理不匹配,我们提出标准蒸馏策略优化(CriPO),通过策略内自蒸馏增强基于评分标准的强化学习。在医学和科学基准测试中,CriPO持续优于基于评分标准的强化学习,以约少两倍的优化步骤实现更强的最终性能。

英文摘要

Rubric-based RL has recently shown promise in improving LLMs on open-ended tasks. A widely recognized limitation of rubric-based RL is limited exploration: criteria that no rollout manages to satisfy (Unexplored Criteria, UC) receive no optimization signal. Recent methods address this by incorporating rubric information as external guidance during rollout, yet they introduce a train-inference mismatch: the policy is optimized on rollouts produced under external guidance while this guidance is absent at inference time, causing error accumulation through autoregressive decoding. Moreover, these exploration-focused approaches overlook a fundamentally different failure mode that we term Suppressed Criteria (SC) -- criteria that are satisfied by some rollouts yet whose learning signals are lost during optimization because scalar reward aggregation assigns them non-positive aggregate advantages. Our analysis reveals that SC are remarkably prevalent: over 57% of samples exhibit this failure mode throughout training, with an average of 1.8 SC per sample. To simultaneously address both UC and SC without introducing training-inference mismatch, we propose Criterion-Distilled Policy Optimization (CriPO), which enhances rubric-based RL via on-policy self-distillation. For UC, CriPO constructs a criterion-injection self-teacher and computes a localized forward-KL loss to inject missing behaviors into the policy. For SC, CriPO employs a counterfactual self-teacher to locate criterion-relevant tokens in negative-advantage rollouts and flips their token-level advantages to positive values, preserving useful patterns that would otherwise be suppressed. Experiments on medicine and science benchmarks demonstrate that CriPO consistently outperforms rubric-based RL, achieving stronger final performance with approximately $2\times$ fewer optimization steps.

URL PDF HTML 收藏
2607.18072 2026-07-21 cs.LG cs.AI 新提交

SGN: A Similarity-based Generative Network for Data Generation under Distribution Shift

SGN:一种用于分布偏移下数据生成的基于相似度的生成网络

Jiaqi Zhu, Xincheng Chen, Yuncheng Wu, Zhaojing Luo, Beng Chin Ooi

机构 * National University of Singapore(新加坡国立大学) Renmin University of China(中国人民大学) Beijing Institute of Technology(北京理工大学) Zhejiang University(浙江大学)

AI总结 研究分布偏移下数据生成问题,提出基于相似度的生成网络SGN,在源数据训练后无需参数更新用于新目标域,通过学习潜在空间和利用目标域小代表性集,实现目标引导数据增强,实验验证其有效性。

详情
AI中文摘要

在源域上训练的生成模型生成的样本与偏移后的目标域往往匹配不佳,限制了其在目标域数据增强方面的有效性。虽然特定于目标的适应可以减少这种不匹配,但通常需要额外的优化和特定于域的参数。我们提出了一种基于相似度的生成网络(SGN),这是一个可重复使用的框架,在有标签的源数据上训练一次,无需参数更新即可应用于新的目标域。SGN学习由标签诱导的成对相似度构建的潜在空间,同时通过编码器-解码器架构保留重构信息。在生成时,对来自目标域的一个小的有标签代表性集进行编码,并在学习到的潜在空间中进行组合,使生成的样本在保持类一致性的同时继承目标特定特征。我们进一步分析了所提出的相似度结构的可实现性和维度要求。在图像和表格数据集上的实验证明了SGN在源到目标分布偏移下进行目标引导数据增强的有效性。

英文摘要

Generative models trained on a source domain often produce samples that are poorly aligned with shifted target domains, limiting their effectiveness for target-domain data augmentation. Although target-specific adaptation can reduce this mismatch, it typically requires additional optimization and domain-specific parameters. We propose a Similarity-based Generative Network (SGN), a reusable framework that is trained once on labeled source data and applied to new target domains without parameter updates. SGN learns a latent space structured by label-induced pairwise similarities while preserving reconstructive information through an encoder-decoder architecture. At generation time, a small labeled representative set from the target domain is encoded and combined in the learned latent space, allowing the generated samples to inherit target-specific characteristics while maintaining class consistency. We further analyze the realizability and dimensionality requirements of the proposed similarity structure. Experiments on image and tabular datasets demonstrate the effectiveness of SGN for target-guided data augmentation under source-to-target distribution shifts.

URL PDF HTML 收藏
2607.17843 2026-07-21 cs.LG cs.CL cs.DC 新提交

Mobius Learning: Cyclic Depth Folding in Transformers

莫比乌斯学习:Transformer中的循环深度折叠

Tongtian Zhu

机构 * Zhejiang University(浙江大学)

AI总结 研究挑战Transformer中块作用与位置绑定的传统观点,提出基于循环深度折叠的莫比乌斯学习,同一组块在不同数据流中有不同应用时机,实现深度角色叠加,实验表明其在分布式训练中有优势,开辟新设计空间。

详情
AI中文摘要

基于Transformer的语言模型沿有序深度轴组织计算,浅块和深块常发挥不同表征作用。本文挑战了这些作用必须与块在有序序列中的位置绑定的传统观点。引入基于循环深度折叠的训练架构莫比乌斯学习,不同数据流遵循循环移位的块顺序。同一组块对一些数据流在序列早期应用,对另一些在晚期应用,实现深度角色叠加。在四 worker 实验中,用Muon在25亿FineWeb令牌上训练修改后的GPT - 2小模型(1.24亿参数),莫比乌斯学习在更多Transformer块序列轮次时验证损失更低。这表明块组不必局限于序列中固定的浅或深角色,基于循环深度折叠开辟了新设计空间,且该结构适合内存受限的分布式训练。

英文摘要

Transformer-based language models organize computation along an ordered depth axis, where shallow and deep blocks often develop distinct representational roles. We challenge the conventional view that these roles must remain tied to a block's position in the ordered sequence. We introduce Mobius Learning, a training architecture based on cyclic depth folding, in which different data streams follow cyclically shifted block orders. The same block group is therefore applied early in the block sequence for some data streams and late for others, so it is optimized in both shallow and deep roles, a phenomenon we call depth-role superposition. Surprisingly, in four-worker experiments with a modded GPT-2 small (124M) model trained on 2.5B FineWeb tokens using Muon, Mobius Learning achieves lower validation loss than a fixed-order looped Transformer at larger numbers of Transformer block-sequence passes. This counterintuitive result shows that a block group need not remain confined to one fixed shallow or deep role within the block sequence and opens a new design space based on cyclic depth folding. Crucially, this structure makes Mobius Learning particularly well suited to memory-constrained distributed training: raw training data remain local, while each worker stores one block group rather than the complete Transformer block stack.

URL PDF HTML 收藏
2607.17742 2026-07-21 cs.AI 新提交

Semantically Similar, Logically Distinct: Diagnosing the Semantic-Answerability Gap in Table RAG

语义相似,逻辑不同:诊断表格检索增强生成中的语义可回答性差距

Jiaming Tian, Liyao Li, Wentao Ye, Haobo Wang, Lihua Yu, Zujie Ren, Gang Chen, Junbo Zhao

机构 * Zhejiang University(浙江大学) Bank of Hangzhou Co., Ltd.(杭州银行股份有限公司) Zhejiang Lab(之江实验室)

AI总结 研究表格检索增强生成中语义可回答性差距问题,通过TCR-Bench基准评估发现密集检索器存在差距,分析其相关因素,用Answerability-Aware Reranking两阶段管道测试,提升了检索性能,揭示差距源于缺少可回答性验证步骤。

详情
AI中文摘要

表格是检索增强生成(RAG)中的关键知识源,但检索到的表格可能缺乏足够证据来回答查询,即存在可回答性问题。为研究此问题,我们引入TCR-Bench基准,围绕具有高度相似模式但内容有细微差异的兄弟表构建。我们评估的密集检索器存在语义-可回答性差距,从0.755(神谕)降至0.330(前5检索)。分析表明差距与语义积累等有关。作为诊断工具,我们测试了Answerability-Aware Reranking(AAR)两阶段管道,它将顶级目标检索从18.2%提高到57.4%,证明许多失败反映了缺少可回答性验证步骤。

英文摘要

Tables are a critical knowledge source in retrieval-augmented generation (RAG), but a retrieved table may lack sufficient evidence to answer a query, a property we call answerability. While answerability broadly concerns whether a source or collection of sources contains sufficient evidence, retrieval models optimized for semantic relevance do not guarantee it even in the single-source case, creating a fundamental mismatch. To study this, we introduce TCR-Bench, a diagnostic benchmark for Table Content-level Answerability in RAG, built around sibling tables, i.e., tables with highly similar schemas but subtle content differences. On TCR-Bench, the dense retrievers we evaluate persistently exhibit a Semantic-Answerability Gap: they often retrieve the correct sibling group yet struggle to pinpoint the uniquely answerable table within it, dropping QA performance from 0.755 (oracle) to 0.330 (top-5 retrieved). Our analysis suggests this gap is associated with semantic accumulation, schema-level cue dependence, and weak row-column binding. As a diagnostic probe into the source of this gap, we test whether a lightweight two-stage pipeline, Answerability-Aware Reranking (AAR), applying direct query-table answerability judgment, can recover performance: it raises top-1 target retrieval from 18.2% to 57.4%, and this large gain is itself evidence that much of the observed failure reflects a missing answerability verification step, rather than an inherent limitation of model capacity alone.

URL PDF HTML 收藏
2607.17598 2026-07-21 cs.AI cs.CL cs.SE 新提交

Is Progressive Disclosure All You Need for Long-Context Agents?

长上下文智能体仅靠渐进式披露就够了吗?

Yifeng He, Yinzhe Zhao, Jicheng Wang, Hao Chen

机构 * University of California, Davis(加利福尼亚大学戴维斯分校) Zhejiang University(浙江大学) The University of Hong Kong(香港大学)

AI总结 研究长文档问答中智能体获取文档路径自行决定读取内容的模式,通过对照研究,在InfiniteBench上对比原始文档导航、Agent Skills包设计与经典混合检索器,发现不同情况下各方法表现不同,渐进式披露在语料库大时作用关键。

详情
AI中文摘要

长文档问答通常要在将整个文档加载到上下文窗口和附加单独的检索器之间做出选择。智能体人工智能提出了一个更广泛的选项,即给智能体文档路径并让它决定如何以及读取什么内容。Agent Skills作为一种将专业知识打包成智能体按需加载的文件夹的标准,提供了一种现成的机制:渐进式披露,它仅暴露查询所需的内容,从简短描述到具体段落。从业者迅速采用这种模式进行书籍长度的理解任务,但支持此类选择的证据一直是传闻。我们对这种模式进行了首次对照研究,在InfiniteBench上,针对三种智能体框架和三个模型家族,将原始文档导航和几种Agent Skills包设计与经典混合检索器进行比较。在单本书籍的情况下,收益取决于框架,当智能体对原始文档导航不佳时收益很大,而当强大的智能体框架已经自行划分和检索时收益接近零。当扩展到跨多本书籍的任务时,原始文档导航失效,而一级渐进式披露退化更慢并领先。第二个更深的路由级别毫无帮助,有时甚至会直接破坏准确性,所以一个级别就足够了。渐进式披露带来的是上下文,而非智能:当强大的智能体可以自行定位正确段落时它是多余的,而一旦语料库变得太大而无法通过阅读导航时它是决定性的。

英文摘要

Long-document question answering usually forces a choice between loading the whole document into the context window and bolting on a separate retriever. Agentic AI suggests a broader option, giving the agent the document path and letting it decide how and what to read. Agent Skills, a standard for packaging expertise into folders an agent loads on demand, supply a ready mechanism: progressive disclosure, which exposes only what a query needs, from a short description down to the specific passages. Practitioners rapidly adopted this pattern for book-length understanding tasks, but the evidence to support such choices has been anecdotal. We run the first controlled study of the pattern, comparing raw-document navigation and several designs of Agent Skills packs against a classical hybrid retriever across three agent harnesses and three model families on InfiniteBench. On a single book, the gain depends on the harness, running large when the agent navigates the raw document poorly but near zero when a strong agent harness already divides and retrieves on its own. When scaling up to tasks that span many books, raw-document navigation collapses while one-level progressive disclosure degrades more slowly and pulls ahead. A second, deeper routing level never helps and sometimes breaks accuracy outright, so one level is enough. Progressive disclosure buys context, not intelligence: it is redundant while a strong agent can locate the right passages itself, and decisive once the corpus grows too large to navigate by reading.

URL PDF HTML 收藏
2607.17521 2026-07-21 cs.RO 新提交

GeoWorldAD: Geometry World Action Model for Autonomous Driving

GeoWorldAD:用于自动驾驶的几何世界行动模型

Songyan Zhang, Jinyuan Tian, Hanbing Li, Daqi Liu, Hao Chen, Wenhui Huang, Fang Li, Guang Chen, Hangjun Ye, Long Chen, Kuiyuan Yang, Chen Lv

机构 * Nanyang Technological University(南洋理工大学) Xiaomi EV(小米汽车) Zhejiang University(浙江大学) Harvard University(哈佛大学)

AI总结 研究自动驾驶中安全高效规划决策问题,提出GeoWorldAD模型,通过在自我对齐3D空间中规划轨迹、用潜在未来几何标记预测场景演变,并逐步聚合多尺度几何线索,实验证明该模型在自动驾驶方面性能先进。

详情
AI中文摘要

自动驾驶需要在动态3D环境中做出安全且高效的规划决策。尽管近期的视觉/视频行动模型能直接从视觉观察中学习策略并随视觉Transformer和大规模训练数据发展良好,但常缺乏明确的几何基础和对未来的空间引导。本文提出GeoWorldAD,一种在自我对齐的3D空间中进行轨迹规划并通过潜在未来几何标记预测短视距场景演变的几何世界行动模型。当前几何为安全规划提供基本空间约束,未来几何揭示周围物体和以自我为中心的自由空间如何演变,减少过度保守决策且不牺牲安全性。通过迭代轨迹细化逐步聚合多尺度当前几何和潜在未来几何以有效利用这些几何线索。在NAVSIM v1和v2上的实验证明了其先进性能,凸显了明确的3D几何基础和未来几何世界建模对安全高效自动驾驶的有效性。

英文摘要

Autonomous driving requires both safe and efficient planning decisions in dynamic 3D environments. Although recent Vision/Video-Action models learn policies directly from visual observations and scale well with advances in vision transformers and large-scale training data, they often lack explicit geometric grounding and future-aware spatial guidance, limiting their ability to balance collision avoidance and driving progress. In this work, we propose GeoWorldAD, a geometry world action model that grounds trajectory planning in ego-aligned 3D space and anticipates short-horizon scene evolution with latent future geometry tokens. Present geometry provides essential spatial constraints for safe planning, while future geometry reveals how surrounding agents and ego-centric free space may evolve, reducing overly conservative decisions without sacrificing safety. To efficiently exploit these geometric cues, GeoWorldAD progressively aggregates multi-scale present geometry and latent future geometry through iterative trajectory refinement. Experiments on NAVSIM v1 and v2 demonstrate state-of-the-art performance, highlighting the effectiveness of explicit 3D geometry grounding and future geometry world modeling for safe and efficient autonomous driving.

URL PDF HTML 收藏
2607.17460 2026-07-21 cs.AI 新提交

AEC-DS: Adaptive Erasure Coding with PDP-Triggered Reputation and QoS-Aware Migration for Decentralized Storage

AEC-DS:用于去中心化存储的具有PDP触发声誉和QoS感知迁移的自适应纠删编码

Shuaiwen Li, Weihang Yu, Ke Wang, Meng Han

机构 * Zhejiang University(浙江大学) Central University of Finance and Economics(中央财经大学) Binjiang Institute of Zhejiang University(浙江大学滨江学院)

AI总结 针对去中心化存储系统中审计结果未有效指导决策的问题,提出AEC-DS机制,利用PDP反馈驱动,结合QoS感知迁移策略调整分片放置,模拟显示其能保持数据耐久性,减少恢复操作,类迁移作用显著,为自我修复的去中心化存储提供实用路径。

详情
AI中文摘要

在去中心化存储系统中,审计结果常未直接用于指导后续冗余和分片放置决策,导致资源分配低效和恢复延迟。我们提出AEC-DS,一种由可验证数据拥有(PDP)反馈驱动的闭环自适应纠删编码机制。PDP审计持续更新节点声誉,QoS感知迁移策略根据节点可靠性和数据优先级调整分片放置。该策略将高优先级分片从不稳定节点移至冷层中更可靠节点,并在后续放置决策中惩罚不稳定节点。对800个节点和500个文件的模拟表明,AEC-DS在冗余因子为1.25倍的评估故障模型下保持100%数据耐久性。与Static-EC、Dynamic-EC和DRD-EC相比,AEC-DS将累积恢复操作减少66.8%-75.2%。消融结果进一步表明类迁移在防止数据丢失中起主要作用,将测量的防丢失能力提高176.8%。这些结果表明PDP反馈可将完整性审计与冗余和放置适配联系起来,为实现自我修复的去中心化存储提供了一条实用途径,同时考虑了迁移的额外成本。

英文摘要

In decentralized storage systems, audit results are often not used directly to guide later redundancy and shard-placement decisions, which can lead to inefficient resource allocation and delayed recovery. We propose AEC-DS, a closed-loop adaptive erasure coding mechanism driven by Provable Data Possession (PDP) feedback. PDP audits continuously update node reputation, while a QoS-aware migration policy adjusts shard placement according to node reliability and data priority. The policy moves high-priority shards from unstable nodes to more reliable nodes in the cold tier and penalizes unstable nodes in subsequent placement decisions. Simulations with 800 nodes and 500 files show that AEC-DS maintains 100% data durability under the evaluated fault model with a redundancy factor of 1.25x. Compared with Static-EC, Dynamic-EC, and DRD-EC, AEC-DS reduces cumulative recovery operations by 66.8%-75.2%. Ablation results further show that class migration plays a major role in preventing data loss, improving the measured loss-prevention capability by 176.8%. These results indicate that PDP feedback can connect integrity auditing with redundancy and placement adaptation, providing a practical path toward self-healing decentralized storage while accounting for the additional cost of migration.

URL PDF HTML 收藏
2607.17423 2026-07-21 cs.CV 新提交

TimeLens2: Generalist Video Temporal Grounding with Multimodal LLMs

TimeLens2:使用多模态大语言模型进行通用视频时间定位

Yuhan Zhu, Changlian Ma, Xiangyu Zeng, Xinhao Li, Zhiqiu Zhang, Songze Li, Jun Zhang, Tianxiang Jiang, Yuandong Yang, Ziang Yan, Zikang Wang, Xinyu Chen, Haoran Chen, Shaowei Zhang, Limin Wang

机构 * Nanjing University(南京大学) Shanghai AI Laboratory(上海人工智能实验室) Shanghai Jiao Tong University(上海交通大学) Zhejiang University(浙江大学) University of Science and Technology of China(中国科学技术大学) Fudan University(复旦大学)

AI总结 研究通用视频时间定位问题,TimeLens2将时间证据视为区间集,通过多种策略构建多跨度监督,其时间瓦瑟斯坦奖励等方法提供反馈,在多个基准测试中表现出色,不同变体均有性能提升。

Comments Technical Report

详情
AI中文摘要

视频多模态大语言模型能描述视频中发生的事情,但很少能确定支持证据出现的时间。我们研究通用视频时间定位,即一个模型预测跨视频长度、领域、查询形式和视角的可变基数证据区间集。现有训练策略与该集值任务不匹配。TimeLens2在整个监督和优化过程中将时间证据视为区间集。TimeLens2-93K通过字幕衍生提案、独立定位、跨智能体共识、语义验证和边界细化构建可靠的多跨度监督。我们的时间瓦瑟斯坦奖励计算合并区间支持上均匀分布之间精确的一维\(W_1\),在不等基数和等效碎片化情况下提供密集、无匹配反馈;时间交并比用精确重叠反馈进行补充。在七个基准测试中,TimeLens2-2B在每个基准上均优于所有大小匹配的基线,4B和8B变体达到了当前最优性能。2B、4B和8B变体分别比其Qwen3-VL主干提高了14.2、13.0和18.1的平均交并比分数。

英文摘要

Video multimodal large language models (MLLMs) can describe what happens in a video, but rarely identify when the supporting evidence occurs. We study generalist video temporal grounding, in which one model predicts a variable-cardinality set of evidence intervals across video lengths, domains, query forms, and viewpoints. Existing training strategies are misaligned with this set-valued task: long-video labels often rely on brittle one-pass annotation, while reinforcement-learning rewards either fail to distinguish non-overlapping predictions or require fragile segment matching. TimeLens2 treats temporal evidence as an interval set throughout supervision and optimization. TimeLens2-93K constructs reliable multi-span supervision through caption-derived proposals, independent localization, cross-agent consensus, semantic verification, and boundary refinement. Our temporal Wasserstein reward computes exact one-dimensional \(W_1\) between uniform distributions over merged interval supports, providing dense, matching-free feedback under unequal cardinalities and equivalent fragmentation; temporal IoU complements it with precise-overlap feedback. Across seven benchmarks, TimeLens2-2B outperforms all size-matched baselines on every benchmark, while the 4B and 8B variants achieve state-of-the-art performance, surpassing open-source models with up to 397B parameters. The 2B, 4B, and 8B variants improve over their Qwen3-VL backbones by 14.2, 13.0, and 18.1 mIoU points, respectively.

URL PDF HTML 收藏
2607.17329 2026-07-21 cs.CV 新提交

MIS-HCC: Hierarchical Channel Clustering for Efficient Medical Image Segmentation

MIS-HCC:用于高效医学图像分割的分层通道聚类

Bo Zhao, Haoran Yu, Lifei Liu, Zongcheng Chu, Yining Liu, Chang Liu, Szu-Yu Chen, Zequn Xie

机构 * Yale University(耶鲁大学) University of Florida(佛罗里达大学) Wichita State University(威奇托州立大学) Purdue University(普渡大学) University of California, Berkeley(加利福尼亚大学伯克利分校) Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所) Stevens Institute of Technology(史蒂文斯理工学院) Zhejiang University(浙江大学)

AI总结 针对医学图像分割模型在资源受限平台部署的挑战,提出分层聚类压缩方法MIS-HCC,利用瓦瑟斯坦距离表示通道相似性形成矩阵指导聚类,融合通道生成压缩网络,实验证明其在精度和压缩效率上优于现有方法。

详情
AI中文摘要

医学图像分割模型需要高精度和轻量级设计以适应实际医学应用。由于其高计算和参数要求,在资源有限的医学平台上部署这些模型仍然是一项重大挑战。现有的模型压缩剪枝方法大多忽略了复杂深度神经网络内部结构之间的内在联系和相似性。因此,压缩模型可能无法有效保留预训练网络的基本特征。为解决此问题,我们提出了一种用于医学图像分割模型的分层聚类压缩方法(MIS-HCC)。该方法采用分层聚类对通道进行划分并有效融合其参数。具体而言,它利用瓦瑟斯坦距离来表示预训练网络各层内通道的相似性,形成一个指导聚类过程的相似性矩阵。然后将每个聚类中的通道融合以生成压缩网络。在三个医学图像数据集上的实验结果表明,MIS-HCC在准确性和压缩效率方面均优于现有方法,为在资源有限的医学平台上部署医学图像分割模型提供了有效解决方案。

英文摘要

Medical image segmentation models require both high accuracy and lightweight design to accommodate real-world medical applications. The deployment of these models on resource-limited medical platforms remains a significant challenge due to their high computational and parameter requirements. Existing pruning methods for model compression mostly overlook the intrinsic connections and similarity between the internal structures of complex deep neural networks. As a result, compressed models may not effectively retain the basic features of the pretrained network. To solve this problem, we propose a hierarchical clustering compression method for medical image segmentation models (MIS-HCC). This approach employs hierarchical clustering to partition channels and fuse their parameters efficiently. Specifically, it leverages the Wasserstein distance to represent similarity of channels within layers of pre-trained network, forming a similarity matrix that guides the clustering process. Channels within each cluster are then fused to produce a compressed network. Experimental results on three medical image datasets application demonstrate that MIS-HCC outperforms the state-of-the-art methods in both accuracy and compression efficiency, offering an effective solution for deploying medical image segmentation models on resource-limited medical platforms.

URL PDF HTML 收藏
2607.17305 2026-07-21 cs.AI cs.CR 新提交

Learning-Driven Adaptive Audit Scheduling: A Sequential Decision Approach to Off-Chain Data Integrity

学习驱动的自适应审计调度:一种用于链下数据完整性的序贯决策方法

Changting Lin, Fan Li, Weihang Yu, Keyang He, Mingyuan Yan, Yourong Chen, Meng Han

机构 * Hangzhou Yunphant Network Technology Co., Ltd.(杭州云幻网络科技有限公司) Zhejiang University(浙江大学) Central University of Finance and Economics(中央财经大学) University of North Georgia(北佐治亚大学) Zhejiang Shuren University(浙江树人大学)

AI总结 研究链下数据完整性审计问题,提出DRQN-CMDP方法,结合GRU层与拉格朗日对偶上升,利用无配对同态MAC原语,相比其他13种方法,在燃气成本、误报率和检测延迟上取得良好平衡。

详情
AI中文摘要

我们将链下数据的加密审计建模为部分可观测性下的约束马尔可夫决策过程(CMDP):存储节点的隐藏类型和损坏状态使问题成为部分可观测马尔可夫决策过程(POMDP),而误报率上限ρ施加了明确的安全约束。我们提出了DRQN-CMDP,一种深度循环Q网络,其门控循环单元(GRU)层维持对潜在节点类型的信念,并与拉格朗日对偶上升相结合,自动调整误报率惩罚λ。一种无配对的同态消息认证码(MAC)原语提供O(1)的链上验证成本。在13种方法中,DRQN-CMDP实现了良好的平衡:比固定高频审计的燃气成本低83%,误报率为个位数(7.5%),检测延迟适中,这是其他方法在所有三个目标上无法同时匹配的组合。

英文摘要

We model cryptographic auditing of off-chain data as a Constrained MDP (CMDP) under partial observability: the storage node's hidden type and corruption state make the problem a POMDP, while a miss-rate ceiling rho imposes an explicit security constraint. We propose DRQN-CMDP, a Deep Recurrent Q-Network whose GRU layer maintains a belief over the latent node type, paired with Lagrangian dual ascent that adapts the miss-rate penalty lambda automatically. A pairing-free homomorphic-MAC primitive supplies O(1) on-chain verification cost. Across 13 methods--four DQN variants, PPO, A2C, PPO-Lagrangian, a stateful Bayesian heuristic, three fixed-rule baselines, and an oracle-informed heuristic--DRQN-CMDP achieves a favourable balance: 83% lower gas than fixed high-frequency auditing, single-digit miss rate (7.5%), and moderate detection latency--a combination no other method matches across all three objectives simultaneously.

URL PDF HTML 收藏
2607.17247 2026-07-21 cs.LG cs.AI 新提交

Distilled Reinforcement Learning for LLM Post-training

用于大语言模型训练后处理的蒸馏强化学习

Chen Wang, Zhaochun Li, Jionghao Bai, Yining Zhang, Hexuan Deng, Ge Lan, Yue Wang

机构 * College of Elite Engineers, Nankai University(南开大学精英工程师学院) Zhongguancun Academy(中关村学院) Beijing Institute of Technology(北京理工大学) Zhejiang University(浙江大学) Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所) Harbin Institute of Technology(哈尔滨工业大学) College of Software, Nankai University(南开大学软件学院)

AI总结 研究大语言模型训练后处理问题,提出蒸馏强化学习方法,集成教师监督到RL目标,含反向重要性采样等三个组件,能有效转移知识,在家族内和跨家族蒸馏实验中性能大幅优于标准RL和OPD。

详情
AI中文摘要

大语言模型训练后处理对于提升推理、适应性和对齐性至关重要。现有方法主要有强化学习(RL)和策略蒸馏(OPD)两种范式。RL依赖粗粒度结果监督,存在信用分配困难和获取新知识能力有限的问题。OPD通过KL散度无条件匹配教师逻辑,导致类似教师提供新知识少,差异大的教师指导无效,限制其用于家族内蒸馏。我们提出蒸馏强化学习(Distilled RL),将教师监督集成到RL目标中,提供细粒度指导,选择性转移新知识并避免无条件模仿。它包含反向重要性采样、负样本重置和序列级几何归一化三个组件。通过案例研究表明其能有效转移知识,实验显示在家族内和跨家族蒸馏设置中,Distilled RL在pass@1和pass@k方面均大幅优于标准RL和OPD。

英文摘要

Large language model (LLM) post-training is essential for improving reasoning, adaptation, and alignment. Existing methods mainly follow two paradigms: reinforcement learning (RL) and on-policy distillation (OPD). However, RL relies on coarse-grained outcome supervision, resulting in difficult credit assignment and limited capability to acquire new knowledge. OPD, meanwhile, unconditionally matches teacher logits through KL divergence, which creates a dilemma: similar teachers provide little new knowledge, while substantially different teachers often yield ineffective guidance, largely restricting OPD to within-family distillation. We propose Distilled Reinforcement Learning (Distilled RL), which integrates teacher supervision into the RL objective to provide fine-grained guidance, selectively transfer new knowledge and avoid unconditional imitation. Distilled RL contains three components: reverse importance sampling with clipping, negative sample reset, and sequence-level geometric normalization. Through a concise and interpretable case study, we demonstrate that Distilled RL can effectively transfer previously unavailable knowledge from a teacher model to a student model. Extensive experiments across both within-family and cross-family distillation settings show that Distilled RL substantially outperforms standard RL and OPD in terms of both pass@1 and pass@k. Our code is available at https://github.com/597358816/Distilled-RL.

URL PDF HTML 收藏
2607.17244 2026-07-21 cs.LG 新提交

DynImmune-BERT: Dynamic Immune Repertoire Modeling with Neural ODE Driven Continuous Transformers

DynImmune-BERT:基于神经常微分方程驱动的连续变换器的动态免疫组库建模

Rong Fu, Yongtai Liu, Xiaowen Ma, Haoyu Zhao, Shuo Yin, Yiqing Lyu, Long Zhang, Wangyu Wu

机构 * University of Macau(澳门大学) Hanyang University(汉阳大学) Zhejiang University(浙江大学) Wuhan University(武汉大学) Tsinghua University(清华大学) South China University of Technology(华南理工大学) University of Liverpool(利物浦大学)

AI总结 研究针对纵向T细胞受体组库建模问题,提出DynImmune-BERT模型,结合多种方法,能在有纵向组库结构时补充静态编码器,通过特定评估方式得出结果,对小外部队列和协议差异需谨慎解读。

详情
AI中文摘要

纵向T细胞受体组库包含免疫扰动后克隆扩增、收缩、消失和重新出现的信号。静态组库语言模型通常将样本总结为一袋序列,因此采样间隔、测序深度和克隆存在模式仅得到微弱体现。本文提出了DynImmune-BERT,一种用于患者水平免疫状态预测的连续时间组库模型。该方法结合了深度自适应中心对数比初始化、克隆存在门控神经常微分方程动力学、有界邻域自注意力、基于事件的状态重启以及监督优势和稀有克隆质量的混合传输目标。一个低秩元适配器初始化重新出现的克隆型,同时保持参数数量与观察到的克隆数量无关。评估将文献报道的基线与内部控制的时间比较分开,报告小外部队列的不确定性,添加校准和阈值诊断,并可视化潜在克隆轨迹和注意力邻域。结果表明,当纵向组库结构可用时,事件感知时间建模可以补充强大的静态编码器,而小外部队列和协议差异需要谨慎解释。

英文摘要

Longitudinal T cell receptor repertoires contain signals of clonal expansion, contraction, disappearance, and reappearance after immune perturbation. Static repertoire language models usually summarize a sample as a bag of sequences, so the sampling interval, sequencing depth, and clone presence pattern are only weakly represented. This paper presents DynImmune-BERT, a continuous time repertoire model for patient level immune status prediction. The method combines depth adaptive centered log ratio initialization, clone presence gated Neural ordinary differential equation dynamics, bounded neighborhood self attention, event based state restart, and a hybrid transport objective that supervises dominant and rare clone mass. A low rank meta adapter initializes reappearing clonotypes while keeping the parameter count independent of the number of observed clones. The evaluation separates literature reported baselines from internally controlled temporal comparisons, reports uncertainty for small external cohorts, adds calibration and threshold diagnostics, and visualizes latent clone trajectories and attention neighborhoods. The results indicate that event aware temporal modeling can complement strong static encoders when longitudinal repertoire structure is available, while small external cohorts and protocol differences require cautious interpretation.

URL PDF HTML 收藏
2607.17152 2026-07-21 cs.CR cs.CL 新提交

How Jailbreak Attacks Inform Safety Alignment: A Defender-Centric, Shapley-Based Evaluation of Jailbreak Contributions

越狱攻击如何为安全对齐提供信息:以防御者为中心、基于Shapley值的越狱贡献评估

Yukai Zhou, Feiyang Lu, Xiaokai Mao, Jinfei Liu, Wenjie Wang

机构 * ShanghaiTech University(上海科技大学) Zhejiang University(浙江大学)

AI总结 研究以防御者为中心评估越狱攻击,提出A - MESS框架,通过AttackSHAP估计攻击效用,在不同场景下实验发现攻击成功率排名与防御者效用弱对齐,有限查询能准确估计,直接优化子集安全效用更强,建议将越狱攻击视为安全改进资源。

详情
AI中文摘要

对大语言模型的越狱攻击通常以攻击者为中心的指标(如攻击成功率)来评估,但能攻破模型的攻击不一定对提升其安全性有用。我们提出以防御者为中心的越狱评估观点,即通过将攻击用作安全训练的红队数据时所带来的下游安全改进来评估。在此基础上,我们引入A - MESS(最小有效攻击子集选择),这是一个与设置无关的框架,用于从黑盒子集效用观察中归因和选择越狱攻击。A - MESS估计AttackSHAP,这是基于Shapley值的分数,可将边际效用归因于单个攻击,并通过贪婪或基于代理的优化在用户指定预算下选择紧凑攻击子集。在受控效用场景和真实大语言模型安全设置中,我们发现攻击成功率排名与以防御者为中心的效用弱对齐,有限效用查询能准确估计AttackSHAP,直接优化子集产生的安全效用比以攻击者为中心或仅归因选择更强。这些结果表明应将越狱攻击视为提升安全的资源,而非仅用于攻破模型的工具。

英文摘要

Jailbreak attacks on large language models are usually evaluated by attacker-centric metrics such as attack success rate (ASR), yet an attack that breaks a model is not necessarily useful for improving its safety. We propose a defender-centric view of jailbreak evaluation, where attacks are evaluated by the downstream safety improvements they enable when used as red-teaming data for safety training. Building on this view, we introduce A-MESS (Minimal Effective Attack-Subset Selection), a setting-agnostic framework for attributing and selecting jailbreak attacks from black-box subset utility observations. A-MESS estimates AttackSHAP, a Shapley-based score that attributes marginal utility to individual attacks and selects compact attack subsets under user-specified budgets via greedy or surrogate-based optimization. Across controlled utility landscapes and real LLM safety settings, we find that ASR rankings are weakly aligned with defender-centric utility, that AttackSHAP can be estimated accurately with limited utility queries, and that directly optimizing subsets yields stronger safety utility than attacker-centric or attribution-only selection. These results suggest evaluating jailbreak attacks as resources for improving safety, not only as tools for breaking models.

URL PDF HTML 收藏
2607.17139 2026-07-21 cs.SE cs.CL 新提交

SynH-Rank: Quality-Aware Code Search via Diverse Data Synthesis and Hierarchical Ranking Training

SynH-Rank:通过多样化数据合成和分层排序训练实现质量感知代码搜索

Keyu Liang, Haoye Wang, Yanfu Yan, Zhiyuan Wan, Zhongxin Liu

机构 * College of Computer Science and Technology(计算机科学与技术学院) The State Key Laboratory of Blockchain and Data Security(区块链与数据安全国家重点实验室) Zhejiang University(浙江大学) Hangzhou City University(杭州市城市大学) Zhejiang University and Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security(浙江大学和杭州高新区(滨江)区块链与数据安全研究院)

AI总结 研究针对代码搜索中重排器忽视非功能质量的问题,提出SynH-Rank框架,结合大语言模型数据合成与分层排序训练,采用三级标注方案及新指标,有效提升了代码搜索质量,在多方面取得优于传统方法的性能。

详情
AI中文摘要

代码搜索通过实现高效的代码重用提高开发者生产力。当前代码搜索系统常采用检索-重排管道,重排器专注于建模查询与代码间的语义相关性,却忽略了执行速度、内存使用和可维护性等关键非功能质量。实现质量感知代码搜索面临两大挑战:高质量标注数据集稀缺及标准对比学习目标的局限性。为解决这些挑战,我们提出SynH-Rank,一种质量感知代码重排框架,它将基于大语言模型的多样化数据合成与分层排序训练相结合。SynH-Rank采用三级标注方案明确建模层次结构:高质量相关>低质量相关>不相关。此外,我们引入一个包含4209对的新基准和两个新指标:用于评估高质量代码优先级的质量偏好准确率(QPA)和用于评估复杂条件下性能的多条件准确率(MCA)。实验结果表明,SynH-Rank比主干模型将QPA提高了20.15%,比仅基于相关性的标准对比训练性能提升了15.80%,同时增强了传统相关性指标和多条件泛化能力。

英文摘要

Code search enhances developer productivity by enabling efficient code reuse. Current code search systems often use a retrieve-then-rerank pipeline, where rerankers focus on modeling semantic relevance between queries and code. However, these rerankers overlook critical non-functional qualities like execution speed, memory usage, and maintainability, which are essential for practical software development. Studies reveal developers expect results to maintain high coding standards and satisfy specific needs, such as resource optimization, highlighting the importance of quality-aware code search. Achieving quality-aware code search faces two major challenges: the scarcity of quality-annotated datasets for effective training and the limitations of standard contrastive learning objectives, which fail to capture the ordinal relationships among high-quality, low-quality, and irrelevant code. Although contrastive learning excels in distinguishing relevant from irrelevant code, its binary objective does not support nuanced quality distinctions.To address these challenges, we propose SynH-Rank, a quality-aware code reranking framework that combines LLM-driven diverse data synthesis with hierarchical ranking training. SynH-Rank employs a three-level labeling scheme to explicitly model the hierarchy: high-quality relevant > low-quality relevant > irrelevant. Additionally, we introduce a new benchmark with 4,209 pairs and two novel metrics: Quality Preference Accuracy (QPA) for assessing prioritization of high-quality code and Multi-Condition Accuracy (MCA) for evaluating performance under complex constraints.Experimental results show SynH-Rank improves QPA by 20.15\% over backbone models and outperforms standard relevance-only contrastive training by 15.80\%, while simultaneously enhancing traditional relevance metrics and multi-condition generalizability.

URL PDF HTML 收藏
2607.16947 2026-07-21 cs.CV 新提交

When Physical Preferences Meet Semantic Constraints: Physical and Semantic Direct Preference Optimization for Text-to-Video Generation

当物理偏好遇到语义约束:用于文本到视频生成的物理和语义直接偏好优化

Siwei Meng, Yawei Luo, Shu Zhang, Ping Liu

机构 * University of Nevada, Reno(内华达大学雷诺分校) Zhejiang University(浙江大学)

AI总结 文本到视频生成模型存在物理合理性与语义一致性的矛盾,提出物理和语义直接偏好优化方法(PSDPO),通过调节偏好对贡献,限制语义漂移,实现更可靠平衡,实验验证其在提高物理合理性和保持语义一致性上优于基线和现有方法。

Comments This work is accepted by ACM MM 2026

详情
AI中文摘要

文本到视频(T2V)生成模型已实现强大的视觉逼真度,但提高物理合理性可能会以与输入文本的语义一致性为代价。这种矛盾的产生是因为物理偏好通常通过比较两个视频之间的动态来确定,而不考虑任何一个视频是否忠实地描绘了提示指定的场景,这使得物理语义冲突在这种监督范式下成为一种系统趋势。我们将此挑战表述为一个约束偏好优化问题,并提出物理和语义直接偏好优化(PSDPO),它基于每个偏好对的物理和语义信号之间的一致性来调节其贡献。梯度级分析表明,PSDPO将冲突对的语义漂移限制在可控的残差范围内,并进一步推动了一种可证明减少累积漂移的分阶段优化协议。实验表明,PSDPO在VideoPhy-2上比基线提高了高达2倍的物理合理性,同时在VBench上保持了强大的语义一致性,比现有的基于偏好的方法实现了更可靠的平衡。

英文摘要

Text-to-video (T2V) generation models have achieved strong visual realism, but improving physical plausibility can come at the cost of semantic consistency with the input text. This tension arises because physical preference is typically determined by comparing dynamics between two videos, without accounting for whether either video faithfully depicts the scene specified by the prompt, making physical-semantic conflict a systematic tendency under this supervision paradigm. We formulate this challenge as a constrained preference optimization problem and propose Physical and Semantic Direct Preference Optimization (PSDPO), which modulates each preference pair's contribution based on the agreement between its physical and semantic signals. A gradient-level analysis shows that PSDPO bounds the semantic drift from conflicting pairs to a controllable residual, and further motivates a staged optimization protocol that provably reduces cumulative drift. The resulting method operates entirely within the standard DPO framework, requiring no auxiliary models or additional loss terms. Experiments show that PSDPO improves physical plausibility by up to $2\times$ over the baseline on VideoPhy-2, while maintaining strong semantic consistency on VBench, achieving a more reliable balance than existing preference-based methods.

URL PDF HTML 收藏
2607.16806 2026-07-21 cs.RO 新提交

Token-Wise Latent Streaming from Slow Reasoners to Fast Planners for Dynamic Vision Language Navigation

用于动态视觉语言导航的从慢速推理器到快速规划器的逐令牌潜在流

Tianshuai Hu, Yangyi Zhong, Zeying Gong, Lingdong Kong, Xiaodong Mei, Guoyang Zhao, Xiaolu Liu, Song Wang, Rong Li, Junwei Liang

机构 * The Hong Kong University of Science and Technology(香港科技大学) The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州)) National University of Singapore(新加坡国立大学) Zhejiang University(浙江大学)

AI总结 针对动态视觉语言导航中语言推理慢与规划需即时的矛盾,提出SPARK-VLN双系统框架,通过三个模块将慢速VLM推理器知识流到快速规划器,引入新基准套件,提高了导航成功率、社会合规性及推理效率。

详情
AI中文摘要

在动态、以人类为中心的环境中的视觉语言导航存在一个基本矛盾:语言推理缓慢且深思熟虑,而安全、符合社会规范的规划应该即时且具有反应性。由此产生的观测陈旧性对安全至关重要:推理过程中选择的动作在执行时可能已经不安全。我们观察到,在VLM完成推理之前很久,其中间隐藏状态就已经编码了与动作相关的意图。我们提出了SPARK-VLN,这是一个用于动态社会VLN的双系统框架,在整个令牌生成过程中将慢速VLM推理器的知识流到快速流匹配专家规划器,在推理过程中提供新的和不断演变的指导。该设计由三个模块实现:一个逐令牌隐藏流提取器,一个序列到插槽潜在桥接器,一个不断演变的潜在调节器。我们还引入了一个用于动态社会视觉语言导航的以人类为中心的基准套件,该套件在整个推理过程中使行人和机器人保持活跃,并报告导航成功、社会合规性、人类碰撞和明确的陈旧性统计数据。在这些设置中,SPARK-VLN提高了导航成功率和社会合规性,并保持了推理效率。

英文摘要

Vision-Language Navigation in dynamic, human-centric environments exposes a fundamental tension: linguistic reasoning is slow and deliberative, whereas safe, socially compliant planning should be instant and reactive. The resulting observation staleness is safety-critical: a maneuver chosen during inference can already be unsafe by the time it executes. We observe that, long before a VLM finishes its inference, its intermediate hidden states already encode action-relevant intent. We propose SPARK-VLN, a dual-system framework for dynamic social VLN that streams the slow VLM reasoner's knowledge to a fast flow-matching expert planner throughout token generation, providing fresh and evolving guidance during inference. This design is realized by three modules: a Token-Wise Hidden Streamer that extracts intermediate hidden states along the token generation process, a Sequence-to-Slot Latent Bridge that projects them into fixed-size latent slots, and an Evolving Latent Conditioner that infuses them into the expert planner. We also introduce a human-centric benchmark suite for dynamic social vision-language navigation that keeps pedestrians and the robot active throughout inference and reports navigation success, social compliance, human collisions, and explicit staleness statistics. Across these settings, SPARK-VLN mproves navigation success and social compliance while sustaining inference efficiency. Webpage: https://hutslib.github.io/SPARK-VLN/.

URL PDF HTML 收藏
2607.16692 2026-07-21 cs.SE cs.CL 新提交

Dependency-Guided Code Generation: Structured Matrix Decomposition and Consistency-Guided Refinement

依赖引导的代码生成:结构化矩阵分解与一致性引导的细化

Mingqiao Mo, Yangchen Zeng, Zikai Xiao, Xin Xiao, Wenhua Nie, Zhaolu Kang, Guangyuan Dong, Kai Shu, Hao Zhang, Xiaodong Fan

机构 * University of the Chinese Academy of Sciences(中国科学院大学) ByteDance Inc.(字节跳动公司) Zhejiang University(浙江大学) National Taiwan University(台湾国立大学) Peking University(北京大学) Alibaba Group(阿里巴巴集团) Tsinghua University(清华大学) Liaoning Technical University(辽宁技术大学)

AI总结 针对现有代码生成方法无法充分捕捉代码实体依赖关系的问题,提出依赖感知代码生成框架,通过结构化矩阵分解和一致性引导细化生成代码,经实验验证该方法能生成语义对齐和结构保真度更高的代码。

Comments 12 pages

详情
AI中文摘要

现代软件系统日益复杂,使自动代码生成成为软件工程中的一项基本任务。然而,现有方法往往无法充分捕捉代码实体间复杂的多层次依赖关系,导致生成的代码逻辑不完整或难以集成到实际系统中。为解决此局限,我们提出一个依赖感知代码生成框架,通过基于图的表示明确建模代码实体间的交互。我们将依赖分解为两个互补组件:一个捕获强显式关系的量化矩阵和一个对弱隐式交互建模的稀疏低秩分解。通过交替优化过程有效学习分解。在代码生成期间,将学习到的依赖结构作为约束纳入,确保生成代码的语义连贯和结构一致。此外,我们为强依赖引入稀疏三元组表示,显著提高存储效率和计算可扩展性。大量实验表明,与现有方法相比,我们的方法始终能生成具有更高语义对齐和结构保真度的代码。

英文摘要

The increasing complexity of modern software systems has made automated code generation a fundamental task in software engineering. However, existing approaches often fail to adequately capture the intricate, multi-level dependencies among code entities, leading to generated code that is logically incomplete or difficult to integrate into real-world systems. To address this limitation, we propose a dependency-aware code generation framework that explicitly models interactions among code entities through a graph-based representation. We decompose dependencies into two complementary components: a quantized matrix that captures strong, explicit relations, and a sparse low-rank factorization that models weaker, implicit interactions. The decomposition is efficiently learned via an alternating optimization procedure. During code generation, the learned dependency structure is incorporated as a constraint, ensuring both semantic coherence and structural consistency of the generated code. Furthermore, we introduce a sparse triplet representation for strong dependencies, significantly improving storage efficiency and computational scalability. Extensive experiments demonstrate that our approach consistently produces code with superior semantic alignment and structural fidelity compared to existing methods.

URL PDF HTML 收藏
2607.16286 2026-07-21 cs.CV cs.AI cs.LG 新提交

Depth Estimators Are Implicit Neural Fields for 3D Scene Geometry Inpainting and Reconstruction

深度估计器是用于3D场景几何修复和重建的隐式神经场

Yingzhao Jian, Zihao Lin, Hehe Fan

机构 * College of Artificial Intelligence, Zhejiang University(浙江大学人工智能学院)

AI总结 针对现实世界场景数据3D几何不完整问题,提出神经深度场(NDF),将深度估计器视为场景级隐式场解决问题,通过单测试时优化实现,实验表明其在3D场景几何修复中性能优异。

详情
AI中文摘要

现实世界场景数据的3D几何通常不完整。主流方法使用深度估计器修复缺失结构,但预测结果可能与观测几何不一致,或在分布外数据上不可靠。为解决这些问题,我们提出神经深度场(NDF)。关键见解是深度估计器也可是场景级隐式场。作为估计器,通过学习观测深度数据适应目标域;作为隐式场,拟合现有几何保持一致性。实验表明,NDF在不同场景数据上产生高保真且全局一致的几何,减少跨视图不一致63.3%,提高修复精度23.1%,在3D场景几何修复中达到了当前最优性能。

英文摘要

The 3D geometry of real-world scene data is often incomplete. Mainstream methods use depth estimators to inpaint missing structure. However, their prediction results can be inconsistent with observed geometry, or unreliable on out-of-distribution data. To solve these problems, we propose Neural Depth Field (NDF). Our key insight is that a depth estimator can also be a scene-level implicit field. As an estimator, it adapts to the target domain by learning observed depth data. As an implicit field, it fits the existing geometry to maintain consistency. Under this view, NDF addresses both problems through a single test-time optimization. Experiments show that NDF produces high-fidelity and globally consistent geometry across diverse scene data, ranging from indoor scans to satellite imagery. It reduces cross-view inconsistency by 63.3\% and improves inpainting accuracy by 23.1\%, achieving state-of-the-art performance in 3D scene geometry inpainting. The code is available at: https://github.com/Shadow-Dream/Neural-Depth-Field.

URL PDF HTML 收藏
2607.16284 2026-07-21 cs.CV cs.MM 新提交

MAC 2026: Advancing Micro-Action Analysis Towards Fine-Grained Understanding

MAC 2026:推动微动作分析迈向细粒度理解

Kun Li, Dan Guo, Jihao Gu, Pengyu Liu, Xiaobai Li, Haoyu Chen, Yanbin Hao, Guoying Zhao, Meng Wang

机构 * United Arab Emirates University(阿联酋大学) Hefei University of Technology(合肥工业大学) University College London(伦敦大学学院) Zhejiang University(浙江大学) University of Oulu(奥卢大学) CMVS, University of Oulu(奥卢大学计算机视觉与媒体研究中心)

AI总结 本文介绍第三届MAC 2026,以从识别到细粒度微动作理解为主题,扩大挑战范围,引入新任务并借助多模态大语言模型评估,总结了相关数据集、设置、结果等,还探讨了微动作分析未来方向及在视频理解中的作用。

Comments Challenge Summary Paper of the 3rd Micro-Action Analysis Grand Challenge (MAC 2026) at ACM Multimedia 2026

详情
AI中文摘要

微动作是人类微妙且自发的行为,在社交互动和情感交流中提供重要的非语言线索。然而,其持续时间短、运动模式弱以及细粒度语义差异,使得难以对其进行标准化的标注、建模和评估。为推动微动作分析的学术研究,我们提出并每年组织微动作分析大挑战(MAC)作为该新兴领域的公共基准平台。前两版MAC为微动作识别和检测建立了标准化评估设置,并提供了可公开访问的数据集和协议。在此基础上,本文介绍了与ACM多媒体2026联合举办的第三届MAC。该版本以从识别到细粒度微动作理解为主题,进一步扩大了挑战范围。特别引入了细粒度微动作理解新任务,借助多模态大语言模型进行评估,旨在评估模型捕捉细粒度语义线索和深入解释人类微动作的能力。我们总结了数据集、任务设置、评估协议、竞赛结果以及顶尖团队的代表性解决方案。最后,我们讨论了微动作分析的未来方向及其在以人类为中心的视频理解中的更广泛作用。

英文摘要

Micro-Actions (MAs) are subtle and spontaneous human behaviors that provide important non-verbal cues in social interaction and affective communication. However, their short duration, weak motion patterns, and fine-grained semantic differences make them difficult to annotate, model, and evaluate in a standardized manner. To promote academic research on micro-action analysis, we proposed and have annually organized the Micro-Action Analysis Grand Challenge (MAC) as a public benchmark platform for this emerging field. The first two editions of MAC established standardized evaluation settings for micro-action recognition and detection, providing publicly accessible datasets and protocols. Building upon these editions, this paper presents the 3rd MAC, held in conjunction with ACM Multimedia 2026. Under the theme of moving from recognition to fine-grained micro-action understanding, this edition further expands the scope of the challenge beyond conventional recognition and detection. In particular, we introduce a new task named fine-grained micro-action understanding, evaluated with the assistance of multimodal large language models, aiming to assess models' ability to capture fine-grained semantic cues and interpret subtle human micro-actions at a deeper level. We summarize the datasets, task settings, evaluation protocols, competition results, and representative solutions from top-performing teams. Finally, we discuss future directions for micro-action analysis and its broader role in human-centric video understanding.

URL PDF HTML 收藏
2607.14616 2026-07-21 cs.AI cs.CV 版本更新

SportD: Can VLMs Physically Strategize?

SportD:视觉语言模型能进行物理策略制定吗?

Jasin Cekinmez, Addison J. Wu, Haotian Xia, Akshaya Bharadhwaj, Anay Putty, Anirudh Ravishankar, Jaewoong Lee, Jinglin Xiao, Kyumin Andrew Shim, Mishika Ahuja, Nisarga Patil, Leo Liu, Zhuohan Liu, Weining Shen

机构 * Princeton University(普林斯顿大学) Rice University(莱斯大学) UC Irvine(加州大学欧文分校) POSTECH(浦项科技大学) NYU Shanghai(纽约大学上海分校) UC Santa Barbara(加州大学圣塔芭芭拉分校) Zhejiang University(浙江大学)

AI总结 研究视觉语言模型在足球场景中能否进行物理策略制定,引入SportD基准,通过控球价值模型评估模型决策,发现模型表现低于职业球员,有偏好低方差低回报动作等问题,为衡量模型物理策略推理提供测试平台。

详情
AI中文摘要

视觉语言模型在解释视觉场景方面能力日益增强,但能否利用信息做出策略有效决策尚不明晰。我们在足球场景中研究此问题,模型观察球上决策前几秒,选择射门或传球给特定队友。不同于传统视觉理解任务,足球决策可通过估计每个可用动作的值进行定量评估。我们引入SportD基准,包含2022年世界杯478个球上决策。通过与控球价值模型对比评估模型选择,衡量最优动作准确率和次优决策损失的值。研究发现,三个前沿视觉语言模型最佳表现仅31.4%选最高价值动作,远低于职业球员的38.9%,且都有更大遗憾。进一步分析显示模型偏好低方差、低回报动作,还会部分模仿熟悉模式而非评估反事实替代方案。SportD为衡量视觉语言模型的物理策略推理提供了基于价值的测试平台。

英文摘要

Vision--language models have become increasingly capable of interpreting visual scenes, but it remains unclear whether they can use information to make strategically effective decisions. We investigate this question in soccer, where models observe the seconds preceding an on-ball decision and must choose whether to shoot or pass to a specific teammate. Unlike conventional visual-understanding tasks, soccer enables decisions to be evaluated quantitatively by estimating the value of every available action. We introduce SportD, a benchmark comprising 478 on-ball decisions from the 2022 FIFA World Cup. Each model choice is evaluated against a possession-value model that estimates the action that most increases the attacking team's probability of scoring, allowing us to measure both optimal-action accuracy and the value forfeited by suboptimal decisions. Across three frontier VLMs, the best selects the highest-valued action on 31.4% of events, compared with 38.9% for the professional players, and all models incur significantly greater regret. Further analysis reveals a systematic preference for lower-variance and lower-reward actions: VLMs shoot less often and select substantially less progressive passes than either the optimal policy or the real players. The models also reproduce the player's specific action above chance even when that action is suboptimal, suggesting partial imitation of familiar play patterns rather than consistent evaluation of counterfactual alternatives. SportD provides a value-grounded testbed for measuring physical strategic reasoning in VLMs.

URL PDF HTML 收藏
2607.13586 2026-07-21 cs.CV 版本更新

UniPhysGen: Unified Physical Grounding for Simulation-Ready 3D Assets

UniPhysGen:用于可模拟3D资产的统一物理基础

Xian Li, Rong Wei, Lujie Yang, Haolin Huang, Junyuan Fang, Siliang Tang, Jun Xiao, Rui Tang, Juncheng Li

机构 * Zhejiang University(浙江大学) Manycore Tech Inc.(众核科技公司) University of Electronic Science and Technology of China(电子科技大学)

AI总结 研究针对现有3D资产缺乏统一物理语义问题,提出UniPhys框架及UniPhysGen模型,通过联合推理关节语义和固有物理属性,减轻几何捷径偏差,经实验验证其性能先进,生成资产可用于机器人模拟环境。

Comments 39 pages, 12 figures

详情
AI中文摘要

物理基础的3D资产在具身人工智能和机器人模拟中日益重要。然而,大多数现有3D资产缺乏现实交互所需的统一物理语义,包括关节语义和固有物理属性。当前方法要么独立处理这些语义,要么依赖规范化对象结构,限制了跨异构3D资产的鲁棒性。我们提出了UniPhys,一个可扩展框架,能自动将原始3D资产转换为具有统一物理语义的可模拟资产。在此基础上构建了大规模数据集UniPhys-40K和统一物理基础评估基准UniPhys-Bench。还引入了统一物理基础模型UniPhysGen,其联合推理关节语义和固有物理属性,减轻了异构部件分解下的几何捷径偏差。大量实验证明其在关节基础和固有物理属性估计任务上的先进性能,生成的资产可直接用于机器人模拟环境进行现实物理交互。

英文摘要

Physically grounded 3D assets are increasingly important for embodied AI and robotic simulation. However, most existing 3D assets lack unified physical semantics, including articulation semantics and intrinsic physical properties, required for realistic interaction. Current approaches either treat these semantics independently or rely on canonicalized object structures, limiting robustness across heterogeneous 3D assets. We present UniPhys, a scalable framework for automatically transforming raw 3D assets into simulation-ready assets with unified physical semantics. Based on UniPhys, we construct UniPhys-40K, a large-scale physically grounded dataset, together with UniPhys-Bench, a carefully verified benchmark for unified physical grounding evaluation. We further introduce UniPhysGen, a unified physical grounding model that jointly reasons over articulation semantics and intrinsic physical properties. UniPhysGen incorporates geometry-robust articulation grounding to mitigate geometric shortcut bias under heterogeneous part decompositions. Extensive experiments demonstrate state-of-the-art performance across articulation grounding and intrinsic physical property estimation tasks, while the resulting assets can be directly deployed in robotic simulation environments for realistic physical interaction. Our code and dataset will be available at https://github.com/breezexian/UniPhysGen.

URL PDF HTML 收藏
2606.00680 2026-07-21 cs.AI cs.LG 版本更新

Regularized Offline Policy Optimization with Posterior Hybrid Bayesian Belief

具有后验混合贝叶斯信念的正则化离线策略优化

Hongqiang Lin, Pengfei Wang, Nenggan Zheng

机构 * Zhejiang University, Hangzhou, China(浙江大学)

AI总结 提出后验混合贝叶斯信念(PhyB)以统一量化离线强化学习中的认知不确定性,并基于此开发迭代正则化策略优化算法,实现单调改进直至收敛。

详情
AI中文摘要

离线强化学习旨在从预先收集的数据集中优化策略。该范式的一个瓶颈是管理认知不确定性,这种不确定性源于有限的数据覆盖(样本层面)以及从有限数据中识别转移动态的模糊性(模型层面)。为了统一量化这些不确定性,贝叶斯强化学习通过将动态模型视为随机变量并维护相应的信念而被提出。尽管具有理论吸引力,贝叶斯强化学习中的策略优化在计算上仍然具有挑战性,因为它需要求解带有期望的复合目标。先前的方法要么采用计算可扩展性差的基于搜索的技术,要么施加牺牲贝叶斯强化学习适应性的限制性后验假设。为了解决这些局限性,我们提出了后验混合贝叶斯信念(PhyB),它将期望重新表述为动态模型子集上的凸组合。理论分析表明,这种近似引起的目标差异是有界的。基于PhyB,我们开发了一种迭代正则化策略优化算法,该算法为单调改进直至收敛提供了与度量无关的保证。实验结果表明,PhyB在各种基准测试中达到了最先进的性能。

英文摘要

Offline reinforcement learning (RL) aims to optimize policies from pre-collected datasets. A bottleneck of this paradigm is managing epistemic uncertainty, which arises from limited data coverage (sample-level) and the ambiguity in identifying transition dynamics from finite data (model-level). To provide a unified quantification of these uncertainties, Bayesian RL has been proposed by treating the dynamics model as a random variable and maintaining a corresponding belief. Despite its theoretical appeal, policy optimization in Bayesian RL remains computationally challenging as it requires solving composite objectives with expectations. Prior methods either employ search-based techniques with poor computational scalability or impose restrictive posterior assumptions that sacrifice the adaptability of Bayesian RL. To address these limitations, we propose Posterior Hybrid Bayesian Belief (PhyB), which reformulates the expectation as a convex combination over a subset of dynamics models. Theoretical analysis demonstrates that the objective discrepancy induced by this approximation remains bounded. Based on PhyB, we develop an iterative regularized policy optimization algorithm that provides metric-agnostic guarantees for monotonic improvement until convergence. Empirical results demonstrate that PhyB achieves state-of-the-art performance on various benchmarks.

URL PDF HTML 收藏
2604.08523 2026-07-21 cs.CL cs.AI 版本更新

ClawBench: Can AI Agents Complete Everyday Online Tasks?

ClawBench:AI代理能否完成日常在线任务?

Yuxuan Zhang, Yubo Wang, Yipeng Zhu, Penghui Du, Junwen Miao, Xuan Lu, Zhuofeng Li, Xingwei Qu, Zhengkang Guo, Yuanzhe Shen, Dingjie Song, Han Zhou, Tuney Zheng, Xian Wu, Hao Yu, Songcheng Cai, Yi Lu, Yunzhuo Hao, Minyi Lei, Liang Chen, Kai Zou, Huifeng Yin, Wendong Xu, Dongfu Jiang, Ping Nie, Jiaheng Liu, Wenhu Chen, Kelsey R. Allen

机构 * University of British Columbia(不列颠哥伦比亚大学) Vector Institute(向量研究所) Etude AI Carnegie Mellon University(卡内基梅隆大学) University of Waterloo(滑铁卢大学) Shanghai Jiao Tong University(上海交通大学) UniPat AI Zhejiang University(浙江大学) HKUST(香港科技大学) Tsinghua University(清华大学)

AI总结 ClawBench通过153个日常任务测试AI代理能力,涵盖15类144个平台,挑战多步骤流程和复杂操作,揭示现有模型在真实环境中的局限性。

Comments Project page: https://claw-bench.com

详情
AI中文摘要

AI代理可能能自动化邮箱,但能否自动化其他日常任务?日常在线任务为评估下一代AI代理提供了现实且未解的测试环境。我们引入ClawBench,一个包含153个简单任务的评估框架,涵盖144个活跃平台,从完成购买和预约到提交工作申请。这些任务需要超越现有基准的能力,如从用户提供的文档中获取信息、跨不同平台的多步骤流程导航以及填写大量详细表单。不同于现有在离线沙盒中评估的基准,ClawBench在生产网站上运行,保留了真实世界网页交互的全部复杂性、动态性和挑战。轻量级拦截层仅捕获并阻止最终提交请求,确保安全评估无实际影响。对7个前沿模型的评估显示,无论是专有还是开源模型,只能完成少量任务。例如,Claude Sonnet 4.6仅完成33.3%。ClawBench的进步使我们更接近于能够作为可靠通用助手的AI代理。

英文摘要

AI agents may be able to assist with emails and documents, but can they reliably complete everyday online workflows on real websites? Everyday online tasks offer a realistic yet unsolved testbed for evaluating the next generation of AI agents. To this end, we introduce ClawBench, an evaluation framework comprising 153 everyday online tasks that people need to accomplish regularly in their lives and work, spanning 144 platforms across 15 categories, from completing purchases and booking appointments to submitting job applications. These tasks require capabilities beyond existing benchmarks, such as obtaining relevant information from user-provided documents, navigating multi-step workflows across diverse platforms, and write-heavy operations like filling in many detailed forms correctly. Unlike existing benchmarks that evaluate agents in offline sandboxes with static pages, ClawBench operates on production websites, preserving the full complexity, dynamic nature, and interaction challenges of real-world web environments. An interception layer captures and blocks the final submission request, ensuring safe evaluation without real-world side effects. Our evaluations of 8 frontier models show that both proprietary and open-source models complete only a small portion of these tasks. For example, Claude Sonnet 4.6 achieves only 33.3%, which exposes gaps in current AI agents. Progress on ClawBench brings us closer to AI agents that can function as general-purpose assistants.

URL PDF HTML 收藏
2507.04384 2026-07-21 cs.RO cs.SY eess.SY 版本更新

Rapid and Safe Trajectory Planning over Diverse Scenes through Diffusion Composition

通过扩散合成在多样场景中进行快速且安全的轨迹规划

Wule Mao, Zhouheng Li, Yunhao Luo, Fangguo Zhao, Lei Xie

机构 * College of Control Science and Engineering, Zhejiang University(浙江大学控制科学与工程学院) University of Michigan(密歇根大学) MIT(麻省理工学院)

AI总结 针对动态环境规划难题,提出能量参数化扩散规划框架,结合轻量级安全过滤器和场景无关的数据生成管道,实现多样场景下快速安全规划,仿真与真实实验验证了其有效性、鲁棒性和适用性。

详情
AI中文摘要

在动态环境中实现安全、高效且运动学上可行的规划仍是重大挑战,规划者须同时应对移动障碍物、传感器不确定性和严格运动约束。为此,我们提出能量参数化扩散规划框架,学习保守能量场以在多样场景中实现安全稳定泛化。该公式能灵活整合多种约束,无需重新训练即可泛化到未见环境。为确保部署时的实时安全,还纳入轻量级安全过滤器。此外,开发了基于场景无关的MPC数据生成管道。仿真中,平均规划时间0.21秒,规划失败率0.57%。F1TENTH平台的真实实验进一步验证其有效性,在未见动态环境中能生成无碰撞轨迹,平均避障间隙0.26米,展现出强大的鲁棒性和实际适用性。

英文摘要

Achieving safe, efficient, and kinematically feasible planning in dynamic environments remains a significant challenge, as planners must simultaneously handle moving obstacles, sensor uncertainty, and strict motion constraints. To address this problem, we propose an energy-parameterized diffusion planning framework that learns a conservative energy field to realize safe and stable generalization across diverse scenarios. The energy-parameterized diffusion formulation enables flexible integration of multiple constraints, allowing the planner to generalize to previously unseen environments without retraining. To ensure real-time safety during deployment, we further incorporate a lightweight safety filter that enforces safety and kinematic feasibility constraints in real-time. Additionally, we develop a scene-agnostic, MPC-based data generation pipeline to produce large-scale, dynamically feasible training trajectories. In simulation, the proposed method achieves real-time performance with a mean planning time of 0.21s and a low planning failure rate of 0.57%. Real-world experiments on the F1TENTH platform further validate the effectiveness of the proposed framework. Under sensor uncertainty in previously unseen dynamic environments, the planner consistently generates collision-free trajectories, which remain safe after being tracked by a simple controller, maintaining a mean obstacle clearance of 0.26 m, demonstrating strong robustness and practical applicability. Project page: https://rstp-comp-diffuser.github.io.

URL PDF HTML 收藏
2510.05969 2026-07-21 cs.CL cs.AI 版本更新

Probing the Difficulty Perception Mechanism of Large Language Models

探究大语言模型的难度感知机制

Sunbowen Lee, Qingyu Yin, Chak Tou Leong, Jialiang Zhang, Yicheng Gong, Shiwen Ni, Min Yang, Xiaoyu Shen

机构 * Institute of Digital Twin, EIT(数字孪生研究所,EIT) Wuhan University of Science and Technology(武汉科技大学) Zhejiang University(浙江大学) Hong Kong Polytechnic University(香港理工大学) Shenzhen Institutes of Advanced Technology, CAS(深圳先进技术研究院,中国科学院)

AI总结 研究大语言模型内部评估问题难度的能力,通过线性探测和定位特定注意力头实现难度感知建模,经消融实验验证,为用LLMs作自动难度注释器提供支持,揭示token级差异,表明难度感知有结构组织,提供新方向。

详情
AI中文摘要

大语言模型(LLMs)越来越多地应用于复杂推理任务,但对其内部评估问题难度的能力知之甚少,而这是自适应推理和高效资源分配的关键能力。本文研究LLMs是否在其内部表示中隐式编码问题难度。通过对LLMs最终token表示进行线性探测,证明数学问题难度可线性建模。进一步定位了最终Transformer层的特定注意力头,其对简单和困难问题有相反激活模式以实现难度感知,消融实验验证了定位准确性。实验为将LLMs用作自动难度注释器提供支持,还揭示了token级熵和难度感知的显著差异。研究表明LLMs中的难度感知不仅存在且有结构组织,为未来研究提供了新理论见解和实践方向。

英文摘要

Large language models (LLMs) are increasingly deployed on complex reasoning tasks, yet little is known about their ability to internally evaluate problem difficulty, which is an essential capability for adaptive reasoning and efficient resource allocation. In this work, we investigate whether LLMs implicitly encode problem difficulty in their internal representations. Using a linear probe on the final-token representations of LLMs, we demonstrate that the difficulty level of math problems can be linearly modeled. We further locate the specific attention heads of the final Transformer layer: these attention heads have opposite activation patterns for simple and difficult problems, thus achieving perception of difficulty. Our ablation experiments prove the accuracy of the location. Crucially, our experiments provide practical support for using LLMs as automatic difficulty annotators, potentially substantially reducing reliance on costly human labeling in benchmark construction and curriculum learning. We also uncover that there is a significant difference in entropy and difficulty perception at the token level. Our study reveals that difficulty perception in LLMs is not only present but also structurally organized, offering new theoretical insights and practical directions for future research. Our code is available at https://github.com/Aegis1863/Difficulty-Perception-of-LLMs.

URL PDF HTML 收藏
2607.15995 2026-07-20 cs.CV cs.LG 新提交

CanonicalPhys: Pose-Robust Remote Photoplethysmography via Canonical-Space Priors

CanonicalPhys:通过规范空间先验实现姿态鲁棒的远程光电容积脉搏波描记法

Hui Wei, Seyedata Jodeiri Seyedian, Xiaobai Li, Guoying Zhao

机构 * Center for Machine Vision and Signal Analysis (CMVS), University of Oulu(奥卢大学机器视觉与信号分析中心) ELLIS Institute Finland(芬兰ELLIS研究所) Zhejiang University(浙江大学)

AI总结 研究针对深度远程光电容积脉搏波描记法在头部姿态下性能下降问题,提出CanonicalPhys方法,通过添加可微四点单应性变换固定面部锚点到规范位置,利用规范框架表示相关先验,有效降低姿态影响,减少MAE退化。

Comments Accepted by IJCB 2026. Code: https://github.com/infraface/CanonicalPhys

详情
AI中文摘要

深度远程光电容积脉搏波描记法(rPPG)在正面、静止面部的心率误差可达到次bpm,但在头部姿态下性能会急剧下降。在MMPD数据集上,最先进的FactorizePhys主干网络的平均绝对误差(MAE)从正面(|偏航角|<15°)到大偏航角(|偏航角|≥45°)帧增长了1.60倍。我们认为姿态是一种“坐标结构”干扰,而非数据增强问题。我们引入了CanonicalPhys,它在前面添加了一个可微的四点单应性变换,将四个面部锚点固定在规范位置。在这个规范框架中,三个先验可以表示为每个像素的朗伯权重、跨感兴趣区域的时间一致性损失以及来自窗口化POS的知识蒸馏。在相同参数数量下,CanonicalPhys将MMPD数据集上从正面到大幅偏航的MAE退化从1.60倍降低到1.33倍,并将轻度偏航区间从1.32倍降低到1.07倍,在姿态丰富的目标上跨数据集MAE减少高达32%。

英文摘要

Deep remote photoplethysmography (rPPG) attains sub-bpm heart-rate error on frontal, stationary faces yet degrades sharply under head pose: on MMPD, the state-of-the-art FactorizePhys backbone's MAE grows $1.60\times$ from frontal ($|\text{yaw}|{<}15^\circ$) to large-yaw ($|\text{yaw}|{\geq}45^\circ$) frames. We argue that pose is a \emph{coordinate-structural} nuisance rather than a data-augmentation problem: in image coordinates the same pixel maps to different anatomy at different poses, blocking three priors otherwise natural for rPPG, namely the dichromatic reflection model, pulse-phase invariance across skin regions, and the POS/CHROM chromaticity projection, each of which presumes a stable anatomy-to-pixel mapping. We introduce \textbf{CanonicalPhys}, which prepends a differentiable four-point homography that fixes four facial anchors at canonical positions; in this canonical frame the three priors become expressible as a per-pixel Lambertian weight, a cross-ROI temporal consistency loss, and knowledge distillation from windowed POS, none of which adds trainable parameters over the backbone. At an identical parameter count, CanonicalPhys reduces MMPD's frontal-to-large-yaw MAE degradation from $1.60\times$ to $1.33\times$ and flattens the mild-yaw bin from $1.32\times$ to $1.07\times$ (across CanonicalPhys variants), with matched cross-dataset MAE reductions of up to $32\%$ on pose-rich targets. Code: https://github.com/infraface/CanonicalPhys

URL PDF HTML 收藏
2607.15776 2026-07-20 cs.AI 新提交

NeurOWL: An LLM-Based Neural-symbolic Framework for Incomplete OWL Ontology Reasoning

NeurOWL:基于大语言模型的神经符号框架用于不完整OWL本体推理

Hui Yang, Jiaoyan Chen, Yiping Song, Renate Schmidt, Wen Zhang

机构 * The University of Manchester(曼彻斯特大学) Zhejiang University(浙江大学)

AI总结 研究不完整OWL本体的包含关系推理问题,提出NeurOWL框架,通过大语言模型和本体嵌入联合执行验证和溯因,利用形式与文本语义,在多领域真实本体上评估,展现强大稳健性能。

详情
AI中文摘要

OWL本体提供了一个支持语义推理的形式化知识表示框架,在医疗和生物信息学等领域广泛应用。然而在实际中,现实世界的本体往往不完整,给推理带来挑战。本文聚焦于一个基本的包含关系推理问题:给定一个不完整本体和一个候选(非蕴含)包含关系,判断该包含关系在语义上是否合理,若合理则提供包含潜在缺失公理的逻辑合理的解释。此任务将包含关系验证与本体溯因统一起来,并通过去除对预定义缺失公理候选集的需求对后者进行了推广。为解决此包含关系推理问题,我们提出了NeurOWL,一个端到端的神经符号框架,通过大语言模型和本体嵌入联合执行验证和溯因,利用形式定义语义和文本语义。我们在多个领域的真实世界本体上评估了NeurOWL,展示了其在不同领域的强大且稳健的性能。

英文摘要

OWL ontologies provide a formal knowledge representation framework that enables semantic reasoning, and have been widely adopted across domains such as healthcare and bioinformatics. In practice, however, real-world ontologies are often incomplete, which pose challenges for reasoning. In this work, we focus on a fundamental subsumption reasoning problem: given an incomplete ontology and a candidate (non-entailed) subsumption, determine whether the subsumption is semantically plausible and, if so, providing a logically sound explanation containing potential missing axioms. This task unifies subsumption verification with ontology abduction, and generalizes the latter by removing the need for a predefined candidate set of missing axioms. To address this subsumption reasoning problem, we propose NeurOWL, an end-to-end neuro-symbolic framework that jointly performs verification and abduction, leveraging both formally defined semantics and textual semantics through Large Language Models and ontology embeddings. We evaluate NeurOWL on real-world ontologies across multiple domains, demonstrating strong and robust performance across different domains.

URL PDF HTML 收藏