arXivDaily arXiv每日学术速递 周一至周五更新

高校专区

Shanghai Jiao Tong University(上海交通大学)

至 收录 3215
2607.18084 2026-07-21 cs.AI cs.CL cs.LG 新提交

WorldCupArena: Fine-Grained Evaluation of Language Models and Deep-Research Agents on Football Forecasting

世界杯竞技场:语言模型和深度研究代理在足球预测上的细粒度评估

Zhaokai Wang, Tianlin Gui, Jiayuan Rao, Shangzhe Di, Yihong Tang, Dingli Liang

机构 * Shanghai Jiao Tong University(上海交通大学) McGill University(麦吉尔大学) University College London(伦敦大学学院)

AI总结 介绍世界杯竞技场这一语言模型和深度研究代理的动态基准,用于足球预测。模型赛前接收证据或自行搜索信息进行多项预测,赛后与实际结果对比。通过多指标评估104场比赛和13个系统,展示不同模型表现及与基线对比情况,且新赛程可添加用于评估未来模型。

详情
AI中文摘要

开球前预测足球比赛需要的不仅仅是了解过去的结果:模型必须利用不断变化的信息并在答案可用之前做出明确预测。我们提出了世界杯竞技场,这是一个用于语言模型和深度研究代理的动态基准。2026年国际足联世界杯是其首次评估,相同流程可用于未来联赛和杯赛。每场比赛前,模型要么接收通用证据包,要么自行搜索信息,预测比赛结果、比分、可能的球员和事件、比赛统计数据以及比赛结果。比赛结束后,将这些预测与记录结果进行比较。我们报告了结果准确率、精确比分准确率、当预测比分接近但不准确时给予一定分数的比分线分数,以及其他预测任务的分数。在104场比赛和13个系统中,结果准确率相似的模型在详细预测上差异更明显。与博彩市场和球迷基线相比,最佳系统在结果和精确比分准确率上仅略有提高,但在比分线分数上有更明显的提高。新赛程开始时可添加,使该基准能够评估未来模型而无需使用已知结果。代码、提示、预测和评估脚本在该https网址开源。

英文摘要

Predicting a football match before kickoff requires more than knowing past results: a model must use changing information and make a clear prediction before the answer is available. We present WorldCupArena, a dynamic benchmark for language models and deep-research agents. The 2026 FIFA World Cup is its first evaluation, and the same process can be reused for future leagues and cups. Before each match, a model either receives a common evidence package or searches for information itself. It predicts the result and score, likely players and events, match statistics, and the outcome of the competition. After the match, these predictions are compared with the recorded result. We report result accuracy, exact-score accuracy, and a scoreline score that gives some credit when a predicted score is close but not exact, together with scores for the other prediction tasks. Across 104 matches and 13 systems, models with similar result accuracy differ more clearly on detailed predictions. Compared with betting-market and human-fan baselines, the best system shows only small gains in result and exact-score accuracy, but a clearer gain in Scoreline. New schedules can be added as they begin, allowing the benchmark to evaluate future models without using outcomes that are already known. Code, prompts, predictions, and evaluation scripts are open sourced at https://github.com/wzk1015/WorldCupArena.

URL PDF HTML 收藏
2607.17956 2026-07-21 cs.RO 新提交

Does Robust VIO Need More Learning? Geometry-Verified Visual Measurements under Distribution Shift

鲁棒视觉惯性里程计需要更多学习吗?分布偏移下的几何验证视觉测量

Yangyang Ning, Shu Liang, Quanbo Ge, Tianchen Deng, Yuhua Qi, Shenghai Yuan

机构 * Tongji University(同济大学) Nanjing University of Information Science and Technology(南京信息工程大学) Shanghai Jiao Tong University(上海交通大学) Sun Yat-sen University(中山大学) Nanyang Technological University(南洋理工大学)

AI总结 研究分布偏移下鲁棒视觉惯性里程计是否需更多学习,提出最小学习立体VIO框架,将学习限于视觉测量生成,结合几何验证等,实验表明该方法在多种场景下准确稳定,精心集成学习视觉测量更有效。

详情
AI中文摘要

学习越来越多地被引入视觉惯性里程计(VIO),从学习特征前端到以学习为主的运动和几何估计。然而,当部署条件与训练分布不同时,更多的学习不一定能提高鲁棒性。本文探讨分布偏移下的鲁棒VIO是否真的需要更深层次的学习估计,还是可以将学习局限于视觉测量生成。提出了一个最小学习立体VIO框架,其中SEA-RAFT仅用于提出密集立体对应并预测其不确定性,而时间跟踪、几何验证和状态估计保持显式。在稀疏特征位置采样密集流,使用预测的不确定性和立体极线一致性进行滤波,并通过不确定性加权重投影因子纳入滑动窗口立体惯性估计器。相同的不确定性通过立体三角测量进一步传播用于下游各向异性3D高斯映射。在EuRoC、VIODE和4Seasons上的实验表明,在运动模糊、动态场景、光照变化和从室内到室外的大分布偏移下,能进行准确稳定的估计。消融实验表明,仅靠学习流是不够的:收益来自于将学习到的对应提议与几何验证和不确定性感知加权相结合。这些结果表明,对于OOD鲁棒VIO,精心集成的学习视觉测量可能比学习更大比例的估计管道更有效。基准测试的代码和配置将在接受后开源。可在这个https URL获取补充视频。

英文摘要

Learning is increasingly introduced into visual-inertial odometry (VIO), ranging from learned feature front-ends to learning-dominant motion and geometry estimation. However, learning more of the pipeline does not necessarily improve robustness when deployment conditions differ from the training distribution. This work asks whether robust VIO under distribution shift truly requires deeper learned estimation, or whether learning can be confined to visual measurement generation. We propose a minimal-learning stereo VIO framework in which SEA-RAFT is used only to propose dense stereo correspondences and predict their uncertainty, while temporal tracking, geometric verification, and state estimation remain explicit. Dense flow is sampled at sparse feature locations, filtered using predicted uncertainty and stereo epipolar consistency, and incorporated into a sliding-window stereo-inertial estimator through uncertainty-weighted reprojection factors. The same uncertainty is further propagated through stereo triangulation for downstream anisotropic 3D Gaussian mapping. Experiments on EuRoC, VIODE, and 4Seasons demonstrate accurate and stable estimation under motion blur, dynamic scenes, illumination changes, and large indoor-to-outdoor distribution shifts. Ablations show that learned flow alone is insufficient: the gains arise from combining learned correspondence proposals with geometric verification and uncertainty-aware weighting. These results suggest that, for OOD-robust VIO, carefully integrated learned visual measurements can be more effective than learning a larger fraction of the estimation pipeline. Code and configs for the benchmark will be open-source upon acceptance. A supplementary video is available at https://drive.google.com/file/d/1EVRhOkhanmNXHbQS1Vr80FoEIAYOYOV2/view

URL PDF HTML 收藏
2607.17842 2026-07-21 cs.CV cs.AI 新提交

CaT-GS: Efficient 3DGS Rendering for Large Scale Scenes via Inter-frame Caching and Tile Scheduling

CaT-GS:通过帧间缓存和瓦片调度实现大规模场景的高效3DGS渲染

Tingjia Zhang, Bo Chen, Shengzhong Liu, Fan Wu, Guihai Chen

机构 * Shanghai Jiao Tong University(上海交通大学) University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)

AI总结 针对3DGS在大规模场景中性能下降问题,CaT-GS提出通过推测性多帧预处理、帧间缓存机制及重构光栅化任务来消除冗余与减轻负载不平衡,实验证明其显著提升渲染速度,为大规模场景高保真实时渲染建立新基准。

Comments CVPR 2026

详情
AI中文摘要

近期3D高斯渲染(3DGS)的突破推动了神经渲染的发展,但其在大规模场景中性能因基于瓦片的光栅化计算负担而显著下降。现有优化要么需昂贵的场景重新训练,要么关注管道的狭窄方面。通过分析,发现冗余的帧间预处理、基于视点的遮挡冗余和严重的瓦片级负载不平衡。为此提出CaT-GS,引入推测性多帧预处理和帧间缓存机制消除冗余,用专用内核重构光栅化任务减轻负载不平衡,实验表明其比原3DGS加速达10倍,比先前方法高70%,为大规模场景高保真实时渲染建立了新基准。

英文摘要

Recent breakthroughs in 3D Gaussian Splatting (3DGS) have advanced neural rendering with high fidelity and speed. However, its performance degrades significantly in large-scale scenes due to the computational burden of tile-based rasterization. Existing optimization efforts either require costly scene re-training or focus on narrow aspects of the pipeline, overlooking critical inefficiencies in real-world deployments. Through a comprehensive analysis, we identify three primary sources of redundancy and low GPU utilization: redundant inter-frame pre-processing, viewpoint-based occlusion redundancy, and severe tile-level load imbalance. To address these issues, we propose CaT-GS, a novel and efficient 3DGS rendering pipeline. CaT-GS introduces a speculative multi-frame preprocessing method to eliminate redundant computations across consecutive frames, and an inter-frame caching mechanism to eliminate viewpoint redundant rendering stages. Furthermore, it refactors rasterization tasks with a dedicated kernel to mitigate tile load imbalance, significantly boosting GPU utilization. Extensive experiments demonstrate that CaT-GS achieves a speedup of up to 10 times over the original 3DGS and up to 70% over previous state-of-the-art methods, establishing a new benchmark for high-fidelity, real-time rendering of large-scale scenes.

URL PDF HTML 收藏
2607.17767 2026-07-21 cs.RO 新提交

VLN-AVP: Zero-Shot Vision-Language Navigation with Hybrid Long-Short-Term Memory for Autonomous Valet Parking

VLN-AVP:用于自动代客泊车的基于混合长短时记忆的零样本视觉语言导航

Yijian Li, Xiangru Mu, Changze Li, Hantian Shi, Jiyuan Cai, Jia Cai, Xiaoxue Liu, Yajing Sun, Ming Yang, Tong Qin

机构 * Shanghai Jiao Tong University(上海交通大学) Yinwang Intelligent Technology Co., Ltd.(银望智能科技有限公司)

AI总结 研究针对自主代客泊车依赖预建地图限制可扩展性问题,提出VLN-AVP零样本导航框架,结合BEV模型与VLM,引入混合记忆系统,构建数据集和基准,实验证明该方法在模拟和实际车辆实验中均有良好效果,提升了成功率。

详情
AI中文摘要

自主代客泊车(AVP)中的现有方法通常依赖预建地图,这严重限制了它们在未见环境和开放词汇目标上的可扩展性。受视觉语言模型(VLM)在视觉语言导航(VLN)任务中的应用启发,我们提出了VLN-AVP,一种用于AVP任务的零样本导航框架。通过将鸟瞰图(BEV)模型的精确空间感知与VLM的通用智能相结合,该框架消除了对预建地图的依赖,解释停车场景中的语义环境上下文,并能根据自然语言指令进行直观导航。具体而言,我们引入了一种混合记忆系统:短期感知记忆跟踪语义视觉线索以解决现有方法中VLM单帧推理的局限性,而长期拓扑记忆则促进从过去经验中进行稳定的策略学习。为了弥合现有基准之间的差距,我们还展示了VLN-AVP数据集和基准。它具有10个高保真停车场景和超过1000个导航情节,是迄今为止车库场景数量最多的,也是第一个用于地下停车的VLN基准。大量实验表明,在模拟中,我们的方法与VLN方法相比成功率提高了25%以上,与其他自动驾驶方法相比提高了15%以上。此外,它在实际车辆实验中获得了领先的成功率,证明了其实际可行性。

英文摘要

Existing methods in Autonomous Valet Parking (AVP) typically rely on pre-built maps, which severely restricts their scalability to unseen environments and open-vocabulary targets. Inspired by the application of Vision-Language Models (VLMs) in Vision-Language Navigation (VLN) tasks, we propose VLN-AVP, a zero-shot navigation framework for AVP tasks. By combining the precise spatial perception of a Bird's-Eye-View (BEV) model with the general intelligence of VLMs, our framework 1) eliminates the dependency on pre-built maps, 2) interprets semantic environmental contexts in parking scenarios, and 3) enables intuitive navigation following natural language instructions. Specifically, we introduce a hybrid memory system: a short-term perception memory tracks semantic visual cues to address the limitations of VLM's single-frame reasoning in existing methods, while a long-term topological memory facilitates stable policy learning from past experiences. To bridge the gap in existing benchmarks, we also present the VLN-AVP dataset and benchmark. Featuring 10 high-fidelity parking scenes and over 1,000 navigation episodes, it has the largest number of garage scenes to date and is the first VLN benchmark for underground parking. Extensive experiments demonstrate that in simulation, our method achieves an over 25% improvement in success rate compared to VLN methods and an over 15% improvement compared to other autonomous driving methods. Furthermore, it attains a leading success rate in real-world vehicle experiments, proving its practical feasibility.

URL PDF HTML 收藏
2607.17423 2026-07-21 cs.CV 新提交

TimeLens2: Generalist Video Temporal Grounding with Multimodal LLMs

TimeLens2:使用多模态大语言模型进行通用视频时间定位

Yuhan Zhu, Changlian Ma, Xiangyu Zeng, Xinhao Li, Zhiqiu Zhang, Songze Li, Jun Zhang, Tianxiang Jiang, Yuandong Yang, Ziang Yan, Zikang Wang, Xinyu Chen, Haoran Chen, Shaowei Zhang, Limin Wang

机构 * Nanjing University(南京大学) Shanghai AI Laboratory(上海人工智能实验室) Shanghai Jiao Tong University(上海交通大学) Zhejiang University(浙江大学) University of Science and Technology of China(中国科学技术大学) Fudan University(复旦大学)

AI总结 研究通用视频时间定位问题,TimeLens2将时间证据视为区间集,通过多种策略构建多跨度监督,其时间瓦瑟斯坦奖励等方法提供反馈,在多个基准测试中表现出色,不同变体均有性能提升。

Comments Technical Report

详情
AI中文摘要

视频多模态大语言模型能描述视频中发生的事情,但很少能确定支持证据出现的时间。我们研究通用视频时间定位,即一个模型预测跨视频长度、领域、查询形式和视角的可变基数证据区间集。现有训练策略与该集值任务不匹配。TimeLens2在整个监督和优化过程中将时间证据视为区间集。TimeLens2-93K通过字幕衍生提案、独立定位、跨智能体共识、语义验证和边界细化构建可靠的多跨度监督。我们的时间瓦瑟斯坦奖励计算合并区间支持上均匀分布之间精确的一维\(W_1\),在不等基数和等效碎片化情况下提供密集、无匹配反馈;时间交并比用精确重叠反馈进行补充。在七个基准测试中,TimeLens2-2B在每个基准上均优于所有大小匹配的基线,4B和8B变体达到了当前最优性能。2B、4B和8B变体分别比其Qwen3-VL主干提高了14.2、13.0和18.1的平均交并比分数。

英文摘要

Video multimodal large language models (MLLMs) can describe what happens in a video, but rarely identify when the supporting evidence occurs. We study generalist video temporal grounding, in which one model predicts a variable-cardinality set of evidence intervals across video lengths, domains, query forms, and viewpoints. Existing training strategies are misaligned with this set-valued task: long-video labels often rely on brittle one-pass annotation, while reinforcement-learning rewards either fail to distinguish non-overlapping predictions or require fragile segment matching. TimeLens2 treats temporal evidence as an interval set throughout supervision and optimization. TimeLens2-93K constructs reliable multi-span supervision through caption-derived proposals, independent localization, cross-agent consensus, semantic verification, and boundary refinement. Our temporal Wasserstein reward computes exact one-dimensional \(W_1\) between uniform distributions over merged interval supports, providing dense, matching-free feedback under unequal cardinalities and equivalent fragmentation; temporal IoU complements it with precise-overlap feedback. Across seven benchmarks, TimeLens2-2B outperforms all size-matched baselines on every benchmark, while the 4B and 8B variants achieve state-of-the-art performance, surpassing open-source models with up to 397B parameters. The 2B, 4B, and 8B variants improve over their Qwen3-VL backbones by 14.2, 13.0, and 18.1 mIoU points, respectively.

URL PDF HTML 收藏
2607.17412 2026-07-21 cs.LG 新提交

CORAL: Learning Amyloid Fibril Ligand Docking with Cooperative Binding Rewards

CORAL:通过协同结合奖励学习淀粉样纤维配体对接

Yasheng Sun, Bohan Li, Youqi Tao, Jürgen Schmidhuber

机构 * Center of Excellence for Generative AI, KAUST(沙特阿卜杜拉国王科技大学生成式人工智能卓越中心) MoE Key Lab of Artificial Intelligence, Shanghai Jiao Tong University(上海交通大学人工智能教育部重点实验室) Stern Laboratory, Brigham and Women’s Hospital, Harvard University(哈佛大学布莱根妇女医院斯特恩实验室)

AI总结 研究针对淀粉样纤维配体对接面临的挑战,提出CORAL强化学习框架,通过纳入协同配体 - 配体堆叠能量及蛋白质 - 配体对接亲和力训练模型,引入评估集,实验证明其在姿态质量和结合亲和力相关性上优于现有基线。

Comments 25 pages

详情
AI中文摘要

神经退行性疾病(如阿尔茨海默病和帕金森病)的一个标志是蛋白质异常聚集成淀粉样纤维,能选择性结合这些纤维的小分子有望用作诊断、成像探针和治疗剂。预测此类配体与纤维靶点的结合存在两个基本挑战:一是淀粉样配体复合物的共晶体结构极少,难以进行对接模型的监督训练;二是淀粉样纤维的结合模式与球状蛋白根本不同,现有对接模型无法捕捉。为应对这些挑战,我们提出CORAL,这是一个强化学习框架,训练生成对接模型以生成适合交叉β凹槽几何形状的配体姿态分布。我们的奖励明确纳入了配体 - 配体堆叠能量以及蛋白质 - 配体对接亲和力,直接捕捉淀粉样纤维独特的结合几何形状。我们还引入了由领域专家验证的模型生成姿态构建的淀粉样配体复合物精选评估集。实验表明,与现有对接基线相比,姿态质量和结合亲和力相关性得到了改善。

英文摘要

A hallmark of neurodegenerative diseases such as Alzheimer's and Parkinson's is the aberrant aggregation of proteins into amyloid fibrils, and small molecules that selectively bind to these fibrils hold promise as diagnostics, imaging probes, and therapeutics. Predicting how such ligands bind to fibril targets, however, presents two fundamental challenges. First, resolved co-crystal structures of amyloid-ligand complexes are exceptionally scarce; even with recent advances in cryo-EM only a handful have been structurally characterized, making supervised training of docking models impractical for this target class. Second, amyloid fibrils present a binding mode fundamentally different from globular proteins: ligands intercalate into longitudinal cross-$β$ grooves and stack cooperatively along the fibril axis, a geometry that existing docking models are not designed to capture. To address these challenges, we present CORAL (COopeRative Amyloid Ligand docking), a reinforcement learning framework that trains a generative docking model to produce ligand pose distributions tailored to the cross-$β$ groove geometry. Our reward explicitly incorporates cooperative ligand-ligand stacking energy alongside protein-ligand docking affinity, directly capturing the distinctive binding geometry of amyloid fibrils. We further introduce a curated evaluation set of amyloid-ligand complexes constructed from model-generated poses validated by domain experts. Experiments on both experimentally resolved structures and this evaluation set demonstrate improved pose quality and binding affinity correlation over existing docking baselines.

URL PDF HTML 收藏
2607.17257 2026-07-21 cs.RO cs.AI 新提交

Asynchronous Multimodal Diffusion Policy Composition via Latency-Aware Guidance Fusion

通过延迟感知引导融合实现异步多模态扩散策略组合

Zihao He, Hongjie Fang, Shirun Tang, Cewu Lu, Haoshu Fang

机构 * Shanghai Jiao Tong University(上海交通大学) Noematrix(无矩阵) Shanghai Innovation Institute(上海创新研究院)

AI总结 研究针对多模态扩散策略在模态差异下的融合问题,提出LAG - Fusion框架,通过延迟感知引导融合实现异步策略组合,推导参考帧重定位规则,在接触丰富操纵实验中提升了策略响应性和任务性能。

详情
AI中文摘要

扩散策略在机器人模仿学习中显示出强大潜力,近期扩展引入其他模态以提升操纵性能。但模态在信息内容、传感速率和推理延迟方面存在差异。现有多模态扩散策略通常依赖同步融合或手动设计的多频率架构。我们提出LAG - Fusion,一种用于异步多模态扩散策略组合的延迟感知引导融合框架。它允许特定模态策略以其原生推理速率运行并在可用时提供去噪引导。为使异步组合一致,我们推导了相对动作表示下扩散变量的参考帧重定位规则。通过将低频视觉策略与高频力策略组合在接触丰富的操纵中实例化LAG - Fusion。实验表明,LAG - Fusion在异构模态延迟下比同步融合和专门设计的力感知基线提高了策略响应性和任务性能。

英文摘要

Diffusion policies have shown strong potential for robotic imitation learning, and recent extensions incorporate additional modalities to improve manipulation performance. However, these modalities often differ not only in information content but also in sensing rates and inference latencies. Existing multimodal diffusion policies typically rely on synchronous fusion or manually designed multi-frequency architectures, which either slow down high-frequency feedback or limit extensibility to new modality combinations. We propose LAG-Fusion, a latency-aware guidance fusion framework for asynchronous multimodal diffusion policy composition. LAG-Fusion allows modality-specific policies to operate at their native inference rates and contribute denoising guidance whenever available. To make asynchronous composition consistent, we derive a reference-frame rebasing rule for diffusion variables under relative action representations, enabling delayed guidance to be aligned before fusion. We instantiate LAG-Fusion in contact-rich manipulation by composing a low-frequency vision policy with a high-frequency force policy. Experiments under heterogeneous modality latencies show that LAG-Fusion improves policy responsiveness and task performance over synchronous fusion and specially designed force-aware baselines.

URL PDF HTML 收藏
2607.16742 2026-07-21 cs.CV 新提交

Multi-Dimensional Quality Assessment for AI-Generated Human-Centric Videos: Dataset and Model

人工智能生成的以人为中心的视频的多维质量评估:数据集与模型

Sijing Wu, Yunhao Li, Huiyu Duan, Yucheng Zhu, Xiongkuo Min, Patrick Le Callet, Guangtao Zhai

机构 * Institute of Image Communication and Network Engineering, Shanghai Jiao Tong University(上海交通大学图像通信与网络工程研究所) USC-SJTU Institute of Cultural and Creative Industry, Shanghai Jiao Tong University(上海交通大学南加州大学文化创意产业学院) Polytech Nantes, Université de Nantes(法国南特大学高等理工学院)

AI总结 研究针对AI生成的以人为中心的视频质量评估问题,提出扩展数据集HVEval+并构建MoE-Rater模型,该模型采用专家混合及三阶段训练策略,能统一多种任务,在相关数据集上性能优越,推动了视频质量评估及T2V模型优化。

Comments Accepted for publication in IEEE TCSVT, 2026

详情
AI中文摘要

人工智能生成的以人为中心的视频在众多现代应用中起着关键作用。然而,它们常存在质量问题和语义不匹配,凸显了对此类视频进行有效质量评估的重要性。为此,我们扩展之前的数据集HVEval,加入成对偏好注释,得到HVEval+,这是用于人工智能生成的以人为中心视频的最大整体质量评估数据集。它包含基于综合分类法的1000个提示、由24个文本到视频(T2V)模型生成的20000个视频以及大量人工注释。同时,我们提出MoE-Rater,一种受专家混合(MoE)启发且基于多模态大语言模型(MLLM)的一体化方法,支持在单个模型内进行多维质量评分、多维成对比较和特定类别问答。具体介绍了投影专家混合(MoPE)和LoRA专家混合(MoLE),以及由任务感知预训练、特定任务适应和自适应路由优化组成的三阶段训练策略。大量实验和综合分析表明HVEval+数据集和MoE-Rater方法在推进人工智能生成视频质量评估及促进T2V模型评估与优化方面具有巨大潜力。

英文摘要

AI-generated human-centric videos play a crucial role in a wide range of modern applications. However, they often suffer from quality issues and semantic mismatches, underscoring the importance of effective quality assessment for such videos. To this end, we extend our previous dataset HVEval with pairwise preference annotations, resulting in HVEval+, the largest holistic quality assessment dataset for AI-generated human-centric videos, which comprises 1k prompts based on a comprehensive taxonomy, 20k videos generated by 24 text-to-video (T2V) models, and extensive human annotations, including 60k mean opinion scores (MOSs) and 60k preference pairs across 3 dimensions (i.e., spatial quality, temporal quality, and text-video correspondence), as well as 20k category-specific question-answer (Q&A) pairs. Along with the HVEval+ dataset, we further propose MoE-Rater, a Mixture-of-Experts (MoE)-inspired and multimodal large language model (MLLM)-based all-in-one method that supports multi-dimensional quality rating, multi-dimensional pairwise comparison, and category-specific question answering within a single model. Specifically, we introduce Mixture of Projector Experts (MoPE) and Mixture of LoRA Experts (MoLE), together with a three-stage training strategy consisting of task-aware pre-training, task-specific adaptation, and adaptive routing optimization, to effectively unify multiple tasks, resulting in superior performance on both HVEval+ and Human-AGVQA datasets. Extensive experiments and comprehensive analysis demonstrate the significant potential of the HVEval+ dataset and the MoE-Rater method in advancing AI-generated video quality assessment and further facilitating the evaluation and optimization of T2V models.

URL PDF HTML 收藏
2607.16624 2026-07-21 cs.CV 新提交

SPARE-GS: Structural Parsimony and Resource Efficiency for 3D Gaussian Splatting

SPARE-GS:用于3D高斯点云的结构简约与资源高效性

Zhang Chen, Shuai Wan, Fuzheng Yang, Jiazhi Xia, Weiyao Lin, Junhui Hou

机构 * School of Electronics and Information, Northwestern Polytechnical University(西北工业大学电子信息学院) Department of Computer Science, City University of Hong Kong(香港城市大学计算机科学系) School of Telecommunication Engineering, Xidian University(西安电子科技大学通信工程学院) School of Computer Science and Engineering, Central South University(中南大学计算机科学与工程学院) Department of Electrical Engineering, Shanghai Jiao Tong University(上海交通大学电气工程系)

AI总结 研究针对3D高斯点云训练效率和表示紧凑性问题,提出SPARE-GS框架,通过将结构演化公式化并动态调整基元分布,实现减少高斯数量、训练时间,提升PSNR及下游处理效率,证明全局结构预算调节的广泛适用性。

详情
AI中文摘要

3D高斯点云(3DGS)能实时实现高保真新视图合成,但其训练效率和表示紧凑性受过多基元扩散阻碍。为此,我们将3DGS的结构演化公式化为全局预算约束优化问题并推导最优条件。基于此提出SPARE-GS框架,它动态调整3D高斯基元分布与区域表示需求。实验表明,该框架平均减少30.38%的高斯数量和23.81%的训练时间,提升平均PSNR,还减少下游处理时间,证明全局结构预算调节的广泛适用性。

英文摘要

3D Gaussian Splatting (3DGS) achieves high-fidelity novel view synthesis in real-time; however its training efficiency and representation compactness are hindered by excessive primitive proliferation. To address this challenge, we formulate the structural evolution of 3DGS as a global budget-constrained optimization problem and derive an optimality condition, which requires the marginal utility of structural resources to be balanced across spatial regions under a finite primitive budget. Based on this formulation, we propose SPARE-GS, a general plug-and-play framework that dynamically aligns the distribution of 3D Gaussian primitives with regional representational demand. SPARE-GS estimates capacity-normalized regional demand, assigns adaptive target quotas, and uses regional budget deviations to coordinate densification, pruning and adaptive termination toward a more balanced structural allocation. Extensive experiments across standard, accelerated, and structure-enhanced 3DGS pipelines demonstrate that SPARE-GS reduces the Gaussian count and training time by an average of 30.38% and 23.81%, respectively, while improving the average PSNR. Moreover, the resulting compact representations reduce downstream processing time and improve the rate-distortion performance of diverse compression and pruning methods, demonstrating the broad applicability of global structural budget regulation.

URL PDF HTML 收藏
2607.16355 2026-07-21 cs.CV cs.AI 新提交

PhysAgent: Reflective Agentic Physics Control for Physically Plausible Video Generation

PhysAgent:用于物理上合理的视频生成的反射式智能体物理控制

Qirui Li, Jinkun Hao, Yibo Li, Ran Yi, Paul L. Rosin, Yu-Kun Lai

机构 * Shanghai Jiao Tong University(上海交通大学) Cardiff University(卡迪夫大学)

AI总结 研究物理上合理的视频生成问题,提出PhysAgent反射式智能体框架,通过闭环设计及物理控制API,改善参数控制,实现复杂轨迹等,实验证明其能生成更合理视频,有更好提示对齐及泛化效果。

Comments For project page, see https://iapple233.github.io/PhysAgent

详情
AI中文摘要

基于物理的视频生成的最新进展利用物理模拟作为物理先验来指导视频合成以获得物理上合理的结果。模拟过程由物理规范控制,通常由视觉语言模型一次性生成。这种一次性预测往往无法将用户意图准确转化为可执行的模拟,特别是对于细粒度物体动力学、复杂运动轨迹和时间结构化交互。本文提出了PhysAgent,一个反射式智能体框架,它在物理程序生成、物理模拟、特定阶段验证和目标程序修复之间形成闭环。除了改善对耦合物理参数的控制,我们的框架使智能体能够通过将每个物理程序视为可执行假设来逐步实现复杂轨迹、多阶段交互和精确事件结果。此外,我们设计了一组物理控制API以支持更稳定和复杂的运动行为。大量实验表明,PhysAgent能生成更符合物理原理的视频,实现更好的提示对齐,并在不同物理场景中更有效地泛化。

英文摘要

Recent advances in physics-grounded video generation leverage physics simulation as a physical prior to guide video synthesis toward physically plausible outcomes. The simulation process is controlled by physical specifications, which are typically generated by a vision-language model in a single pass. Such one-shot prediction often fails to accurately translate user intent into executable simulations, particularly for fine-grained object dynamics, complex motion trajectories, and temporally structured interactions. In this paper, we propose PhysAgent, a reflective agentic framework that closes the loop among physical program generation, physics simulation, stage-specific verification, and targeted program repair. Beyond improving the control of coupled physical parameters, our framework enables the agent to progressively realize complex trajectories, multi-stage interactions, and precise event outcomes by treating each physical program as an executable hypothesis. In addition, we design a set of physics-control APIs to support more stable and complex motion behaviors. Extensive experiments demonstrate that PhysAgent produces more physically plausible videos, achieves better prompt alignment, and generalizes more effectively across diverse physical scenarios.

URL PDF HTML 收藏
2607.16327 2026-07-21 cs.CV 新提交

Localization-Infused Vision-Language Semantic Fusion for Text-Guided Medical Image Segmentation

用于文本引导医学图像分割的定位注入视觉语言语义融合

Songyue Han, Mingye Zou, Shuchang Ye, Lei Bi, Mingyuan Meng

机构 * Air Force Engineering University(空军工程大学) Harbin Institute of Technology(哈尔滨工业大学) University of Sydney(悉尼大学) Institute of Translational Medicine, Shanghai Jiao Tong University(上海交通大学转化医学研究院) Zhongguancun Institute of Artificial Intelligence(中关村人工智能研究院)

AI总结 研究针对文本引导医学图像分割,提出LoG框架,通过联合执行多尺度目标定位任务,实现三级定位注入语义融合,在三个基准数据集实验中表现出色,优于现有医学图像分割方法。

详情
AI中文摘要

医学图像分割对现代计算机辅助医学至关重要。近期,文本引导分割通过纳入临床医生制定的文本报告作为图像分割的语义指导展现出前景。现有方法通常通过预训练文本编码器隐式提取文本语义,再经简单图像 - 文本特征融合整合视觉 - 语言语义。但这些方法未明确捕捉文本报告中面向目标的信息,也未探索基础特征级融合之外的多级信息融合策略。本研究提出LoG框架,通过联合执行多尺度目标定位任务,明确捕捉面向目标的视觉 - 语言语义,实现三级定位注入语义融合,包括定位引导特征融合、定位门控注意力融合和定位约束损失融合。在三个基准数据集上的大量实验表明,LoG在QaTa - COV19、MosMedData + 和Kvasir - SEG上分别取得91.59%、80.71%和94.59%的Dice分数,持续优于现有方法。

英文摘要

Medical image segmentation is essential for modern computer-aided medicine. Recently, text-guided segmentation has shown promise by incorporating clinician-formulated textual reports as semantic guidance for image segmentation. These reports describe target appearance, location, and neighboring anatomy, providing explicit guidance for localization and delineation. Existing text-guided segmentation methods typically extract textual semantics implicitly through a pretrained text encoder and then integrate vision-language semantics via straightforward image-text feature fusion. However, these methods do not explicitly capture target-oriented information embedded in textual reports, particularly target location, and do not explore multi-level information fusion strategies beyond basic feature-level fusion, limiting the extraction and integration of critical textual semantics. In this study, we propose LoG, a localization-infused vision-language fusion framework for text-guided medical image segmentation. By jointly performing multi-scale target localization tasks, LoG explicitly captures target-oriented vision-language semantics and enables three-level localization-infused semantic fusion: (i) localization-guided feature fusion that directly infuses location-relevant semantics into visual features, (ii) localization-gated attention fusion that redirects multi-scale localization predictions to reinforce critical regions, and (iii) localization-constrained loss fusion that supervises segmentation based on spatial consistency with target localization. Extensive experiments on three benchmark datasets, involving three medical imaging modalities with paired textual reports, demonstrate that LoG achieves Dice scores of 91.59%, 80.71%, and 94.59% on QaTa-COV19, MosMedData+, and Kvasir-SEG, respectively, consistently outperforming state-of-the-art medical image segmentation methods.

URL PDF HTML 收藏
2607.16267 2026-07-21 cs.RO 新提交

HyperDCM: Dynamic Cluster Memory Replay in Hyperbolic Space for Continual Robotic Navigation Across Scenes

HyperDCM:用于跨场景连续机器人导航的双曲空间动态集群记忆回放

Zhengfei Lu, Jian Yang, Muyu Wang, Shaowen Chen, Jinpeng Mi, Ke Li, Xiong You, Qi Wu, Xuan Tang, Xian Wei

机构 * Software Engineering Institute, East China Normal University(华东师范大学软件工程学院) School of Geospatial Information, Information Engineering University(信息工程大学地理空间信息学院) University of Shanghai for Science and Technology(上海理工大学) Shanghai Jiao Tong University(上海交通大学)

AI总结 针对视觉导航持续学习难题,提出HyperDCM,通过场景图建模和记忆回放增强导航。利用视觉语言模型提取三元组,经R-GCN编码投影到双曲空间,采用动态聚类等策略。实验证明其在保留导航能力和泛化性上优于基线。

详情
AI中文摘要

视觉导航中的持续学习因灾难性遗忘以及适应多样且不断变化的环境的困难而仍具挑战性。为解决这些问题,我们提出双曲动态集群记忆(HyperDCM),这是一种结构感知记忆机制,通过场景图建模和有原则的记忆回放来增强基于扩散策略的导航。HyperDCM使用大型视觉语言模型从RGB观测中提取语义场景三元组,通过关系图卷积网络(R-GCN)将其编码为场景图嵌入,并将嵌入投影到双曲空间以增强连续导航中的结构可分离性和保留性。动态聚类和结构敏感更新策略选择代表性样本进行记忆回放,从而保持知识多样性并减轻灾难性遗忘。在多场景室内和室外数据集上的实验表明,与适用于扩散策略导航的代表性持续学习基线相比,HyperDCM在保留过去导航能力方面表现出色且泛化能力有所提高。

英文摘要

Continual learning in visual navigation remains challenging due to catastrophic forgetting and the difficulties associated with adapting to diverse and evolving environments. To address these issues, we propose Hyperbolic Dynamic Cluster Memory (HyperDCM), a structure-aware memory mechanism that enhances diffusion policy-based navigation through scene graph modeling and principled memory replay. HyperDCM extracts semantic scene triples from RGB observations using large vision-language models, encodes them into scene graph embeddings via a Relational Graph Convolutional Network (R-GCN), and projects the embeddings into hyperbolic space to enhance structural separability and retention in continual navigation. A dynamic clustering and structure-sensitive update strategy selects representative samples for memory replay, thereby preserving knowledge diversity and mitigating catastrophic forgetting. Experiments on multi-scene indoor and outdoor datasets demonstrate that HyperDCM achieves superior retention of past navigation capabilities and improved generalization compared to representative continual learning baselines adapted to diffusion policy navigation.

URL PDF HTML 收藏
2607.16247 2026-07-21 cs.LG cs.CV 新提交

Self-Evolving Just-In-Time Memory for Proactive Embodied Safety

用于主动具身安全的自进化即时记忆

Bingrui Sima, Lizhong Wang, Xiaoya Lu, Kun He, Xiao Yang

机构 * Huazhong University of Science and Technology(华中科技大学) Tsinghua University(清华大学) Shanghai Jiao Tong University(上海交通大学)

AI总结 研究视觉语言模型在闭环交互中应对动态危险的问题,提出自进化即时记忆框架,含RSG、事实记忆和经验记忆,并通过自动测试-验证-写入循环完善元技能,实验证明该框架大幅提升安全成功率且不阻碍任务进展。

详情
AI中文摘要

虽然视觉语言模型(VLMs)使具身智能体能够执行复杂的家庭任务,但它们在闭环交互中难以主动应对动态出现的危险。现有安全方法常依赖运行时护栏来阻止不安全行为或导致过度谨慎,严重阻碍任务进展。为打破安全与进展的权衡,我们引入自进化即时记忆框架,将具身安全从阻碍进展的护栏转变为主动减轻危险。该框架由用于部分可观测下持续安全相关状态跟踪的风险充足拓扑信念图(RSG)、用于精确危险预测的基于智能体的事实记忆以及注入程序元技能以指导可执行、保持进展的减轻危险的经验记忆组成。此外,我们提出自动测试-验证-写入循环,让智能体在测试时能从执行轨迹中不断完善减轻危险的元技能。在IS-Bench上的实验表明,我们的框架大幅提高了多个VLM主干的安全成功率(如在Qwen3-VL-8B上提高30.3%),使智能体能够主动减轻危险而不阻碍任务进展。

英文摘要

While Vision-Language Models (VLMs) have empowered embodied agents to execute complex household tasks, they struggle to proactively handle dynamically emerging hazards during closed-loop interactions. Existing safety approaches often rely on runtime guardrails to block unsafe actions or induce excessive caution, which severely stalls task progress instead of actively resolving the underlying risks. To break this safety-progress trade-off, we introduce the Self-Evolving Just-In-Time Memory framework, which reframes embodied safety from progress-stalling guardrails to proactive hazard mitigation. The framework consists of a Risk-Sufficient Topological Belief Graph (RSG) for persistent safety-relevant state tracking under partial observability, an Agency-Grounded Factual Memory for precise hazard anticipation, and an Experience Memory that injects procedural Meta-Skills to guide executable, progress-preserving mitigation. Furthermore, we propose an automated Test-Verify-Write loop, allowing agents to continually refine their mitigation Meta-Skills from execution traces at test time. Experiments on IS-Bench demonstrate that our framework substantially boosts the Safe-Success rate across multiple VLM backbones (e.g., +30.3% on Qwen3-VL-8B), enabling agents to proactively mitigate hazards without stalling task progress. Code is available at https://github.com/DyMessi/JIT-Memory.

URL PDF HTML 收藏
2607.13597 2026-07-21 cs.RO cs.AI cs.CV 版本更新

Semantic Anchoring for Robotic Action Representations

用于机器人动作表示的语义锚定

Yuan Xu, Youheng Shi, Chengyang Li, Wentao Zhu, Yizhou Wang

机构 * Peking University(北京大学) Eastern Institute of Technology, Ningbo(宁波东方理工大学) Shanghai Jiao Tong University(上海交通大学)

AI总结 研究VLA模型微调后动作表示结构受损问题,受镜像神经元理论启发,通过系统探测证实结构变化与任务表现相关。提出即插即用方法,将动作表示锚定到语义流形并分解通道,经多基准测试验证,有效提升了模型在真实世界任务中的表现。

Comments Project Page: https://xy02-05.github.io/SemanticMN

详情
AI中文摘要

视觉-语言-动作(VLA)模型从预训练的视觉-语言模型继承了丰富的语义表示,但在有限的机器人演示上进行微调会破坏这种结构并削弱泛化能力。由此产生一个基本问题:什么构成良好的动作表示?受镜像神经元理论启发,研究机器人动作表示是否保留预训练编码器捕获的语义结构。系统探测证实微调期间该结构会受损,其质量与任务成功和分布外泛化同步。还引入一种即插即用方法,将动作表示锚定到语义流形,同时将表示分解为共享语义通道和私有通道,推理时丢弃这些通道,使部署模型不变。在不同VLA主干上通过模拟和真实世界基准测试验证,该方法在真实世界分布内任务上提高了18.7%,在分布外泛化上提高了21.5%。

英文摘要

Vision-Language-Action (VLA) models inherit rich semantic representations from pretrained Vision-Language Models, yet fine-tuning on limited robot demonstrations degrades this structure and undermines generalization. A fundamental question therefore arises: what constitutes a good action representation? Inspired by the mirror neuron theory's insight that observation and execution share an intention-level encoding, we examine whether a robot's action representations preserve the semantic structure captured by pretrained encoders. Systematic probing confirms that this structure erodes during finetuning, and that its quality synchronizes with both task success and out-of-distribution generalization. We further introduce a plug-and-play method that anchors action representations to a semantic manifold while decomposing representations into a shared semantic channel and a private channel, all discarded at inference, leaving the deployed model unchanged. Validated on different VLA backbones across simulation and real-world benchmarks, our method yields up to +18.7% on real-world in-distribution tasks and +21.5% on out-of-distribution generalization.

URL PDF HTML 收藏
2607.04438 2026-07-21 cs.CV cs.AI cs.HC cs.MA cs.MM 版本更新

ResearchStudio-Reel: Automate the Last Mile of Research from Paper to Poster, Video, and Blog

ResearchStudio-Reel:实现从论文到海报、视频和博客的研究最后一公里自动化

Lingao Xiao, Yalun Dai, Yangyu Huang, Qihao Zhao, Wenshan Wu, Hugo He, Ruishuo Chen, Jin Jiang, Qianli Ma, Jiahuan Zhang, Xin Zhang, Ying Xin, Yang Ou, Yan Xia, Scarlett Li, Longbo Huang, Zhipeng Zhang, Yang He, Yap Kim Hui, Yan Lu

机构 * Microsoft Research(微软研究院) National University of Singapore(新加坡国立大学) Nanyang Technological University(南洋理工大学) Tsinghua University(清华大学) Peking University(北京大学) Shanghai Jiao Tong University(上海交通大学) Westlake University(西湖大学) CFAR, A*STAR(计算科学与工程研究所,新加坡科技研究局)

AI总结 研究传播自动化困难,以往方法有局限。该研究提出将最后一公里构建为技能组合,实例化ResearchStudio-Reel,包括共享提取器、可编辑生成器和交互式收敛层,能产出多种可编辑工件,效果优于现有系统。

详情
AI中文摘要

研究传播,即将论文转化为海报、演讲视频和博客文章,仍然是手动的最后一公里。以前的自动化方法孤立地处理每个工件,每个都从头重新提取论文,通常提供单向渲染,作者无法在PowerPoint或Word中重新打开,并且根据软VLM偏好分数来评估质量,而在承载部分仍为空时分数会趋于平稳。我们认为这最后一公里最好构建为技能组合:瘦代理可读契约,共享一个上游提取器,并在测量填充循环中包装确定性原语,其出口是硬通过/失败渲染门。我们将其实例化为ResearchStudio-Reel,五个Claude代码和Codex技能组织成一个共享提取器(Paper2Assets)、三个可编辑生成器(Paper2Poster、Paper2Video、Paper2Blog)和一个交互式收敛层(Paper2Reel)。Paper2Assets将每篇论文提取一次到一个共享包中,供每个下游技能重用;三个生成器生成一个可打印的海报、一个同步的演讲视频和一个双语博客,它们在事实层面上保持一致,并能通过PowerPoint或Word进行往返;Paper2Reel然后将这三个绑定到一个独立的HTML查看器中,其部分级点击会使视频、幻灯片、字幕和博客跳转到匹配的内容。在Paper2Poster基准测试中,我们的海报在美学和信息子标准方面领先于先前的自动化系统和单镜头前沿语言模型,在两名外部VLM评委的评估下,在美学方面超过了作者自己的海报,并在84%至93%的论文中总体获胜;能力审计进一步表明,通过将与叙述对齐的幻灯片亮点与由布局感知DOCX修复控制的双语博客独特配对,ResearchStudio-Reel是唯一能够提供所有三个可编辑工件的管道。项目可在此https URL上获取

英文摘要

Despite growing automation, turning a paper into a coherent poster, talk video, and blog piece often remains a labor-intensive last mile. Recent systems increasingly generate multiple dissemination formats, but a practical workflow must also keep the outputs editable in native tools and bound into one navigable deliverable for revision and reuse. We present ResearchStudio-Reel, a native-editable dissemination workspace that binds its three artifacts into one interactive deliverable at the experience level, implemented as five skills executable in Claude Code and Codex: one shared extractor, three editable artifact generators, and one interactive convergence layer. A shared asset bundle feeds a PowerPoint poster and video deck, plus a bilingual Word blog; rather than re-rendering the paper into a fourth format, Paper2Reel converges these already-produced artifacts at the experience level, binding poster regions, video segments, and blog passages into one interactive viewer. Artifact-specific release checks make this delivery contract testable, and Paper2Poster additionally uses a measured-fill loop. On the Paper2Poster benchmark, our Claude Code configuration achieves the best scores among automated systems on all three aesthetic sub-criteria and the best or tied-best scores on two of three information sub-criteria. Under two VLMjudges, it exceeds the authors' posters in average aesthetics (3.56 vs. 3.03) and wins on overall quality on 74 and 95 of the 100 papers under the two judges. The full pipeline additionally packages the native-editable source artifacts and their aligned viewer. Project is available at https://aka.ms/ResearchStudio

URL PDF HTML 收藏
2606.19341 2026-07-21 cs.CV cs.CL cs.SD 版本更新

Native Active Perception as Reasoning for Omni-Modal Understanding

原生主动感知作为全模态理解的推理

Zhenghao Xing, Ruiyang Xu, Yuxuan Wang, Jinzheng He, Ziyang Ma, Qize Yang, Yunfei Chu, Jin Xu, Junyang Lin, Chi-Wing Fu, Pheng-Ann Heng

机构 * The Chinese University of Hong Kong(香港中文大学) Shanghai Jiao Tong University(上海交通大学) Nanyang Technological University(南洋理工大学) Qwen Team, Alibaba Group(阿里巴巴集团Qwen团队)

AI总结 提出OmniAgent,一种基于POMDP迭代观察-思考-行动循环的原生全模态智能体,通过主动感知将推理复杂度与视频时长解耦,在多个基准上达到开源模型最优性能。

Comments Accepted at ICML 2026. Code and models: https://github.com/harryhsing/omniagent

详情
AI中文摘要

用于长视频理解的被动模型通常依赖于“全看一遍”范式,无论查询难度如何都统一处理帧,导致计算成本随视频时长增长。尽管出现了交互式框架,但它们通常依赖于全局预扫描,其上下文成本仍随视频长度扩展。我们提出OmniAgent,第一个原生全模态智能体,将视频理解建模为基于POMDP的迭代观察-思考-行动循环。OmniAgent执行按需动作,选择性地将视听线索提炼到持久文本记忆中,有效将推理复杂度与原始视频时长解耦。为实现这一点,我们引入了(1)智能体监督微调,通过最佳N轨迹合成和双阶段质量控制在启动原生主动感知;(2)带TAURA(轮次感知自适应不确定性重缩放优势)的智能体强化学习,利用轮次级熵将信用分配引导至关键发现轮次。关键的是,OmniAgent表现出正向测试时缩放,性能随推理轮次增加而提升,验证了主动感知的有效性。在十个基准(如VideoMME、LVBench)上的实验结果表明,OmniAgent在开源模型中达到了最先进性能。值得注意的是,在LVBench上,我们的7B智能体优于10倍大的Qwen2.5-VL-72B(50.5% vs. 47.3%)。

英文摘要

Passive models for long video understanding typically rely on a "watch-it-all" paradigm, processing frames uniformly regardless of query difficulty, causing computational cost to grow with video duration. Although interactive frameworks have emerged, they often rely on global pre-scanning, and their context cost still scales with video length. We propose OmniAgent, the first native omni-modal agent that formulates video understanding as a POMDP-based iterative Observation-Thought-Action cycle. OmniAgent executes on-demand actions to selectively distill audio-visual cues into a persistent textual memory, effectively decoupling reasoning complexity from raw video duration. To operationalize this, we introduce (1) Agentic Supervised Fine-Tuning to bootstrap native active perception via best-of-N trajectory synthesis with dual-stage quality control, and (2) Agentic Reinforcement Learning with TAURA (Turn-aware Adaptive Uncertainty Rescaled Advantage), which leverages turn-level entropy to steer credit assignment toward pivotal discovery turns. Crucially, OmniAgent exhibits positive test-time scaling, where performance improves as the number of reasoning turns increases, validating the efficacy of active perception. Empirical results across ten benchmarks (e.g., VideoMME, LVBench) demonstrate that OmniAgent achieves state-of-the-art performance among open-source models. Notably, on LVBench, our 7B agent outperforms the 10$\times$ larger Qwen2.5-VL-72B (50.5% vs. 47.3%).

URL PDF HTML 收藏
2606.10135 2026-07-21 cs.CV cs.AI 版本更新

BiWM: Advancing Open-Source Interactive Video World Models with Bidirectional Autoregression

BiWM:利用双向自回归推进开源交互式视频世界模型

Shaohao Rui, Xiaofeng Mao, Zhanyu Zhang, Peijia Lin, Yansong Zhu, Yibo Zhang, Haibin Wan, Zhangrui Zhao, Weijie Ma

机构 * LynnReal AI Shanghai Innovation Institute(上海创新研究院) Shanghai Jiao Tong University(上海交通大学) Fudan University(复旦大学)

AI总结 提出BiWM框架,通过双向自回归范式将预训练视频骨干转化为交互式世界模型,仅需两阶段训练(微调+分布匹配蒸馏),支持多尺度模型和长程生成,优于现有因果流水线。

详情
AI中文摘要

将双向视频扩散模型过渡到自回归范式提高了视频世界模型的交互性,但现有的因果流水线需要多个阶段(控制微调、自回归训练、因果初始化、少步蒸馏),并且由于误差累积,质量仍落后于双向模型。最近的世界模型如Yume-1.5和Matrix-Game-3.0采用双向自回归方法,通过自我纠正误差传播获得保真度和稳定的长程展开,但开源框架(如minWM)仅支持因果模型。我们提出BiWM,这是首个在双向自回归范式下用于交互式视频世界模型的全栈框架,联合优化生成质量和推理速度。从预训练视频骨干开始,BiWM通过微调注入相机控制,然后运行几步分布匹配蒸馏(DMD)阶段,将骨干转化为动作/相机可控的世界模型:仅需两个训练阶段(而非minWM的四个),在8xH200 GPU上几百步内收敛。单一方案覆盖Wan2.1-1.3B、Wan2.2-5B、HunyuanVideo-1.5-8B和LTX-2.3-22B,并支持现有双向模型的二次微调。BiWM实现了minWM失去可控性的真实相机控制,集成了可插拔历史压缩(FramePack风格和PackForcing风格)用于长程展开,并提供可选的NVFP4 4位训练/推理流水线。为对抗DMD的模式寻求退化,我们添加了GAN和覆盖前向KL目标,以保留场景动态。我们开源BiWM,用于资源受限的研究和高保真环境模拟。

英文摘要

Interactive video world models commonly convert bidirectional video generators into causal autoregressive systems through control fine-tuning, autoregressive training, causal initialization, and few-step distillation. This pipeline is costly, while frozen causal histories accumulate errors that degrade long-horizon fidelity and controllability. We present BiWM, the first open-source full-stack training framework for bidirectional autoregressive video world models. BiWM retains full attention within each generated chunk and requires only two stages: camera/action-control fine-tuning and few-step Distribution Matching Distillation (DMD). Both stages converge within a few hundred optimizer steps on 8 H200 GPUs. The framework supports Wan2.1-T2V-1.3B, Wan2.2-TI2V-5B, HunyuanVideo-1.5-TI2V-8B, and LTX-2.3-22B, together with real-world camera control, pluggable long-history compression, and optional low-bit deployment. Supervised and forward-KL anchors mitigate DMD mode collapse and preserve scene dynamics. BiWM provides a compact, reproducible path from pretrained bidirectional video models to interactive, controllable, and efficient world models.

URL PDF HTML 收藏
2606.09249 2026-07-21 cs.CV 版本更新

DECIS: Dual-Evidence Corrective Verification for Interpretable Strabismus Diagnostic Decision-Making

MAGIS:基于证据的多智能体推理用于可解释的斜视临床决策

Xikai Tang, Yifan Wang, Jiafan Zhuang, Li Luo, Jinming Guo, Xiaoling Xie, Jiacheng Liu, Peiwei Wei, Lihao Zhong, Xiaoli Kang, Jie Cen, Guangqiang Yin, Kunliang Qiu, Ce Zheng, Zhun Fan

机构 * School of Information and Software Engineering, University of Electronic Science and Technology of China(电子科技大学信息与软件工程学院) Shenzhen Institute for Advanced Study, University of Electronic Science and Technology of China(电子科技大学深圳高等研究院) Joint Shantou International Eye Center of Shantou University and The Chinese University of Hong Kong(汕头大学·香港中文大学联合汕头国际眼科中心) School of Artificial Intelligence, Guangzhou City Polytechnic(广州城市职业学院人工智能学院) Medical College, Shantou University(汕头大学医学院) College of Engineering, Shantou University(汕头大学工学院) Department of Ophthalmology, Xinhua Hospital Affiliated to Shanghai Jiaotong University School of Medicine(上海交通大学医学院附属新华医院眼科) Shenzhen Loop Area Institute(深圳环路区域研究所)

AI总结 提出MAGIS框架,通过多智能体协作、双重证据约束上下文和基于证据的纠正验证机制,将斜视诊断从黑箱生成转变为结构化推理,在细粒度斜视基准上将加权F1分数从72.0%提升至91.3%,并显著提高诊断报告的临床可靠性。

详情
AI中文摘要

斜视是一种常见的眼部疾病,需要细粒度亚型诊断以制定个性化治疗方案。然而,现有的深度学习方法主要提供诊断预测,缺乏透明推理;而近期的大视觉语言模型(LVLMs)虽然在联合图像理解和报告生成方面有前景,但在这种对证据敏感且规则驱动的医学任务中极易产生幻觉。为解决这些问题,我们提出了MAGIS,一个基于证据的多智能体可解释斜视诊断推理框架。MAGIS将黑箱端到端生成转变为结构化的诊断过程,包括候选假设生成、双重证据约束上下文、基于证据的纠正验证和报告生成。具体而言,我们引入了双重证据约束上下文(DECC)机制,将来自九个注视方位照片的视觉证据和基于证据的临床诊断规则联合组织成约束上下文,以实现可靠的诊断推理。我们进一步开发了基于证据的纠正验证(EBCV)机制,验证当前诊断假设是否得到视觉证据、基于热图的视觉线索和基于证据的临床诊断规则的支持。当检测到不一致时,触发假设修正。在细粒度斜视基准上的实验表明,MAGIS不仅显著优于其他最先进的诊断系统,将加权F1分数从72.0%提高到91.3%,而且大幅提升了生成诊断报告的临床可靠性(一致性、对齐性和完整性)。这些结果表明,MAGIS为构建准确、基于证据且临床可解释的斜视诊断系统提供了有效解决方案。

英文摘要

Strabismus is a common ocular disorder that requires fine-grained subtype diagnosis for individualized treatment planning. However, existing deep learning methods mainly provide diagnostic predictions without transparent reasoning, while recent large vision-language models (LVLMs), although promising for joint image understanding and report generation, remain highly prone to hallucination in this evidence-sensitive and rule-driven medical task. To address these challenges, we propose DECIS, a Dual-Evidence Corrective verification for Interpretable Strabismus diagnostic decision-making framework. DECIS transforms black-box end-to-end generation into a structured diagnostic process consisting of candidate hypothesis generation, dual-evidence constrained context, evidence-based corrective verification, and report generation. Specifically, we introduce a Dual-Evidence Constrained Context (DECC) mechanism that jointly organizes visual evidence from the photograph of the nine cardinal positions of gaze and evidence-based clinical diagnostic rules into a constrained context for reliable diagnostic reasoning. We further develop an Evidence-Based Corrective Verification (EBCV) mechanism that verifies whether the current diagnostic hypothesis is supported by visual evidence, heatmap-based visual cues, and evidence-based clinical diagnostic rules. Hypothesis refinement is triggered when inconsistency is detected. Experiments on a fine-grained strabismus benchmark demonstrate that DECIS not only outperforms other state-of-the-art diagnostic systems, improving the weighted F1 score from 72.0% to 91.3%, but also improves the clinical reliability (consistency, alignment, and completeness) of generated diagnostic reports. These results demonstrate that DECIS provides an effective solution for building accurate, evidence-based, and clinically interpretable strabismus diagnosis systems.

URL PDF HTML 收藏
2606.08015 2026-07-21 cs.RO 版本更新

Q-VGM: Q-Value-Gradient Matching for Off-Policy Reinforcement Learning of Flow-Matching VLA

Q-VGM: 基于Q引导的值梯度匹配的流匹配VLA策略

Ziqian Wang, Yitian Liu, Xingjian Mao, Minqian Wang, Yao Mu

机构 * Shanghai Jiao Tong University(上海交通大学) University of Michigan, Ann Arbor(密歇根大学安娜堡分校) University of Electronic Science and Technology of China(电子科技大学)

AI总结 提出Q-VGM离线强化学习方法,通过将值梯度转化为去噪时间上的值梯度场,避免反向传播去噪链,高效微调流匹配VLA策略,在LIBERO等任务上显著提升成功率。

Comments 13 pages, 3 figures, 4 tables

详情
AI中文摘要

我们提出Q引导的值梯度匹配(Q-VGM),一种离线强化学习方法,解决了微调流匹配视觉-语言-动作(VLA)策略中长期存在的挑战:如何高效地根据学习到的Q函数改进一个表达力强的流匹配动作专家。有效的改进必须利用评论家的一阶(梯度)信息,但这对于流策略很困难,因为直接通过其多步去噪过程反向传播值函数在VLA规模下数值不稳定,而策略梯度方法所需的可处理动作似然在迭代去噪下不可用。现有的基于值的方法要么通过整个去噪链反向传播,要么仅在测试时使用评论家而不更新策略,要么将评论家改进的动作作为终端标签蒸馏而不监督速度场。Q-VGM通过利用VGG-Flow(一种生成建模中流对齐的值梯度视角)绕过了这些问题,它将值梯度转化为去噪时间上的值梯度场,而不是不稳定的端到端目标。这不需要动作似然,也不需要反向传播去噪链,并且在一个固定的重放缓冲区上操作。评论家是一个动作敏感的Cal-QL集成,基于紧凑的RLT特征和每层动作注入。Q-VGM实现了一种实用的少样本初始化然后从经验中学习的范式:从少样本SFT pi0.5 VLA开始,该方法利用自生成的rollout数据显著提升任务性能,无需额外的专家监督。在LIBERO上,Q-VGM将平均成功率从75.0%提升到92.5%;在RoboTwin 2.0上,从76.4%提升到87.2%;在两个真实机器人桌面任务上,从40.0%提升到67.5%,在所有三种设置中均优于所有相同骨干、相同评论家的基线。

英文摘要

We propose Q-Guided Value-Gradient Matching (Q-VGM), an off-policy reinforcement learning method for a central difficulty in fine-tuning flow-matching vision-language-action (VLA) policies: improving an expressive flow-matching action expert with a learned $Q$-function. Effective improvement must exploit the critic's first-order signal $\nabla_A Q$, yet flow policies make this hard: backpropagating values through the multi-step denoising chain is unstable at VLA scale, and the tractable action likelihoods required by policy gradients are unavailable under iterative denoising. Existing value-based methods therefore backpropagate through the full chain, use the critic only for test-time selection or guidance, or distill critic-improved actions as terminal labels that never supervise the velocity field. Q-VGM instead casts policy improvement as optimal control over the denoising dynamics, where the optimal residual velocity is the gradient of a denoising-time value function: clean-action estimates improved by iterative $Q$-gradient ascent with keep-best selection are converted into residual velocity targets that directly supervise the velocity field -- no action likelihoods, no backpropagation through the denoising chain, fully offline on a fixed replay buffer, and no critic at inference time. The critic is an action-sensitive stepwise IQL critic on compact latent states from the frozen VLA backbone. This enables a few-shot-initialization, learn-from-experience paradigm: starting from a few-shot-SFT $π_{0.5}$ policy, Q-VGM improves the policy from its own rollouts without additional expert supervision, raising the average LIBERO success rate from 79.0% to 92.5%, outperforming all same-backbone, same-critic baselines, and attaining high success rates on four real-robot manipulation tasks, including fine-grained plug insertion.

URL PDF HTML 收藏
2606.05558 2026-07-21 cs.LG 版本更新

Autoregressive Diffusion World Models for Off-Policy Evaluation of LLM Agents

自回归扩散世界模型用于LLM智能体的离线评估

Kaixuan Liu, Guojun Xiong, Weinan Zhang, Shengpu Tang

机构 * Department of Computer Science, Emory University(埃默里大学计算机科学系) School of Computer Science, Shanghai Jiao Tong University(上海交通大学计算机科学学院)

AI总结 提出ADWM框架,通过自回归扩散世界模型从预收集轨迹中模拟环境响应,实现无需在线交互的LLM智能体策略离线评估。

详情
AI中文摘要

在多轮交互环境中评估大语言模型(LLM)智能体成本高且风险大,因为它需要在线环境交互。我们提出ADWM(自回归扩散世界模型),一个仅从预收集轨迹中估计新LLM智能体策略性能的评估框架。核心思想是学习一个潜在扩散世界模型,模拟环境如何响应评估策略,而无需在真实环境中执行。现有的基于扩散的OPE方法通过联合扩散状态和动作,在单次传递中引导完整轨迹,这一假设对于动作是离散文本且必须在观察环境后从策略中采样的LLM智能体不成立。与遭受复合误差的自回归世界模型不同,ADWM将每个转移建模为独立的去噪过程,实现可靠的逐步展开,其中世界模型和智能体按因果顺序交替。关键的是,被评估的LLM智能体通过策略条件得分函数直接引导每一步的扩散生成,确保模拟轨迹准确反映其决策模式。实验上,ADWM在多种多轮智能体任务中实现了准确的价值估计和评估可靠性,展示了其作为离线LLM智能体评估实用框架的前景。

英文摘要

Evaluating large language model (LLM) agents in multi-turn interactive environments is expensive and risky, as it requires online environment interaction. We propose ADWM (Autoregressive Diffusion World Model), an evaluation framework that estimates the performance of a new LLM agent policy purely from pre-collected trajectories. The core idea is to learn a latent diffusion world model that simulates how the environment responds to the evaluation policy, without ever executing it in the real environment. Existing diffusion-based OPE methods guide full trajectories in a single pass by jointly diffusing states and actions, an assumption that breaks down for LLM agents whose actions are discrete text that must be sampled from the policy after observing the environment. Unlike autoregressive world models that suffer from compounding errors, ADWM models each transition as an independent denoising process, enabling reliable step-by-step rollouts where the world model and agent alternate in causal order. Crucially, the LLM agent under evaluation directly guides the diffusion generation at each step via a policy-conditioned score function, ensuring that simulated trajectories accurately reflect its decision-making patterns. Empirically, ADWM achieves accurate value estimates and evaluation reliability across diverse multi-turn agent tasks, demonstrating its promise as a practical framework for offline LLM agent evaluation.

URL PDF HTML 收藏
2606.04165 2026-07-21 hep-ex cs.LG hep-ph physics.ins-det 版本更新

CaloTrilogy: Toward a Breakthrough in One-Step, End-to-End, Physics-Guided Shower Generation for Modern Calorimeters

CaloTrilogy:迈向现代量热器一步式端到端物理引导簇射生成的突破

Cheng Jiang, Sitian Qian, Kevin Pedro, Oz Amram, Huilin Qu, Maggie Voetberg

机构 * School of Physics and Astronomy, University of Edinburgh(爱丁堡大学物理与天文学学院) Department of Physics, University of Wisconsin-Madison(威斯康星大学麦迪逊分校物理系) Fermi National Accelerator Laboratory(费米国家加速器实验室) State Key Laboratory of Dark Matter Physics, Tsung-Dao Lee Institute & School of Physics and Astronomy, Shanghai Jiao Tong University(上海交通大学暗物质物理国家重点实验室、李政道研究所及物理与天文学学院) Key Laboratory for Particle Astrophysics and Cosmology (MOE) & Shanghai Key Laboratory for Particle Physics and Cosmology, Shanghai Jiao Tong University(教育部粒子天体物理与宇宙学重点实验室及上海粒子物理与宇宙学重点实验室,上海交通大学)

AI总结 提出一种结合平均速度场积分器、学习生成先验和物理引导损失项的框架,实现一步或少量评估步骤的高质量簇射生成,性能与最先进的流和扩散模型相当。

详情
AI中文摘要

当前和未来对撞机的高精度量热器模拟对计算资源的需求快速增长,促使开发机器学习替代传统蒙特卡洛工具(如Geant4)。流匹配和基于扩散的生成模型因其样本质量而成为高维快速模拟的主流方法,但通常在推理时需要${\cal O}(100)$次函数评估,并常依赖辅助网络约束全局可观测量,损害了简化的端到端生成。我们引入了一个统一框架,改进了速度、簇射质量和物理保真度之间的平衡。该方法结合了:(i)平均速度场积分器,实现一步或少量评估的采样;(ii)从数据而非随机噪声构建的簇射空间学习生成先验;(iii)训练期间对关键可观测量施加归纳偏置的物理引导损失项。这些元素是训练时的正则化器,保持了端到端推理且无额外成本。仅需一步或少量评估步骤,该模型在多个公开的高粒度量热器数据集上达到了与最先进的流和扩散模型竞争的簇射质量。结果表明层间簇射结构与底层物理一致,为未来的快速模拟工作流提供了有力候选。

英文摘要

High-precision calorimeter simulation at current and future colliders imposes rapidly growing computational demands, motivating the development of machine-learning surrogates for traditional Monte Carlo tools such as Geant4. Flow matching and diffusion-based generative models have become leading approaches for high-dimensional fast simulation because of their sample quality, but typically require ${\cal O}(100)$ function evaluations at inference and often rely on auxiliary networks to constrain global observables, compromising streamlined end-to-end generation. We introduce a unified framework that improves the balance between speed, shower quality, and physics fidelity. The method combines: (i) an average velocity field integrator that enables sampling in one or a few evaluations; (ii) a learned generative prior in shower space, constructed from data rather than random noise; and (iii) physics-guided loss terms that impose inductive biases on key observables during training. These elements are training time regularizers, preserving end-to-end inference with no additional cost. With only one or a few evaluation steps, the model achieves shower quality competitive with state-of-the-art flow and diffusion approaches, tested on several public high granularity calorimeter datasets. The results demonstrate inter-layer shower structure consistent with the underlying physics, providing a strong candidate for future fast simulation workflows.

URL PDF HTML 收藏
2511.08423 2026-07-21 cs.CV 版本更新

OmniAID: Decoupling Semantics and Artifacts for Universal AI-Generated Image Detection in the Wild

OmniAID: 解耦语义与伪影以实现通用AI生成图像野外检测

Yuncheng Guo, Junyan Ye, Chenjue Zhang, Hengrui Kang, Haohuan Fu, Conghui He, Weijia Li

机构 * Shanghai Artificial Intelligence Laboratory(上海人工智能实验室) Sun Yat-Sen University(中山大学) Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院,清华大学) Shanghai Jiao Tong University(上海交通大学)

AI总结 提出OmniAID框架,通过解耦混合专家架构分离语义缺陷和通用伪影,结合两阶段训练策略和Mirage数据集,实现跨生成模型和语义内容的鲁棒AI生成图像检测。

Comments Accepted by ICML 2026

详情
AI中文摘要

一个真正通用的AI生成图像(AIGI)检测器必须同时泛化到多种生成模型和不同的语义内容。当前方法学习单一的、纠缠的伪造表示,混淆了内容相关的缺陷与内容无关的伪影,并进一步受到过时基准的限制。我们提出OmniAID,一种以解耦混合专家(MoE)架构为核心的新框架,该架构分离了:(1)通过可路由的专门语义专家在不同内容领域中的语义缺陷,以及(2)通过固定的通用伪影专家从内容相关缺陷中分离出内容无关的通用伪影。两阶段训练策略首先通过领域特定的困难采样独立专门化专家,然后训练一个轻量级门控网络以实现有效的输入路由。通过明确解耦“生成了什么”(内容特定缺陷)与“如何生成”(通用伪影),OmniAID实现了鲁棒的泛化。我们还引入了Mirage,一个大规模、当代的数据集,包含现代训练集和具有挑战性的测试集。大量实验表明,OmniAID超越了现有检测器,为针对现代野外威胁的AIGI检测建立了新标准。代码可在https://github.com/yunncheng/OmniAID获取。

英文摘要

A truly universal AI-Generated Image (AIGI) detector must simultaneously generalize across diverse generative models and varied semantic content. Current methods learn a single, entangled forgery representation, conflating content-dependent flaws with content-agnostic artifacts, and are further constrained by outdated benchmarks. We propose OmniAID, a novel framework centered on a decoupled Mixture-of-Experts (MoE) architecture that separates: (1) semantic flaws across distinct content domains via Routable Specialized Semantic Experts, and (2) content-agnostic universal artifacts from content-dependent flaws via a Fixed Universal Artifact Expert. A two-stage training strategy first specializes experts independently with domain-specific hard-sampling, then trains a lightweight gating network for effective input routing. By explicitly decoupling "what is generated" (content-specific flaws) from "how it is generated" (universal artifacts), OmniAID achieves robust generalization. We also introduce Mirage, a large-scale, contemporary dataset comprising a modern training set and a challenging test set. Extensive experiments demonstrate that OmniAID surpasses existing detectors, establishing a new standard for AIGI detection against modern, in-the-wild threats. Code is available at https://github.com/yunncheng/OmniAID.

URL PDF HTML 收藏
2604.08523 2026-07-21 cs.CL cs.AI 版本更新

ClawBench: Can AI Agents Complete Everyday Online Tasks?

ClawBench:AI代理能否完成日常在线任务?

Yuxuan Zhang, Yubo Wang, Yipeng Zhu, Penghui Du, Junwen Miao, Xuan Lu, Zhuofeng Li, Xingwei Qu, Zhengkang Guo, Yuanzhe Shen, Dingjie Song, Han Zhou, Tuney Zheng, Xian Wu, Hao Yu, Songcheng Cai, Yi Lu, Yunzhuo Hao, Minyi Lei, Liang Chen, Kai Zou, Huifeng Yin, Wendong Xu, Dongfu Jiang, Ping Nie, Jiaheng Liu, Wenhu Chen, Kelsey R. Allen

机构 * University of British Columbia(不列颠哥伦比亚大学) Vector Institute(向量研究所) Etude AI Carnegie Mellon University(卡内基梅隆大学) University of Waterloo(滑铁卢大学) Shanghai Jiao Tong University(上海交通大学) UniPat AI Zhejiang University(浙江大学) HKUST(香港科技大学) Tsinghua University(清华大学)

AI总结 ClawBench通过153个日常任务测试AI代理能力,涵盖15类144个平台,挑战多步骤流程和复杂操作,揭示现有模型在真实环境中的局限性。

Comments Project page: https://claw-bench.com

详情
AI中文摘要

AI代理可能能自动化邮箱,但能否自动化其他日常任务?日常在线任务为评估下一代AI代理提供了现实且未解的测试环境。我们引入ClawBench,一个包含153个简单任务的评估框架,涵盖144个活跃平台,从完成购买和预约到提交工作申请。这些任务需要超越现有基准的能力,如从用户提供的文档中获取信息、跨不同平台的多步骤流程导航以及填写大量详细表单。不同于现有在离线沙盒中评估的基准,ClawBench在生产网站上运行,保留了真实世界网页交互的全部复杂性、动态性和挑战。轻量级拦截层仅捕获并阻止最终提交请求,确保安全评估无实际影响。对7个前沿模型的评估显示,无论是专有还是开源模型,只能完成少量任务。例如,Claude Sonnet 4.6仅完成33.3%。ClawBench的进步使我们更接近于能够作为可靠通用助手的AI代理。

英文摘要

AI agents may be able to assist with emails and documents, but can they reliably complete everyday online workflows on real websites? Everyday online tasks offer a realistic yet unsolved testbed for evaluating the next generation of AI agents. To this end, we introduce ClawBench, an evaluation framework comprising 153 everyday online tasks that people need to accomplish regularly in their lives and work, spanning 144 platforms across 15 categories, from completing purchases and booking appointments to submitting job applications. These tasks require capabilities beyond existing benchmarks, such as obtaining relevant information from user-provided documents, navigating multi-step workflows across diverse platforms, and write-heavy operations like filling in many detailed forms correctly. Unlike existing benchmarks that evaluate agents in offline sandboxes with static pages, ClawBench operates on production websites, preserving the full complexity, dynamic nature, and interaction challenges of real-world web environments. An interception layer captures and blocks the final submission request, ensuring safe evaluation without real-world side effects. Our evaluations of 8 frontier models show that both proprietary and open-source models complete only a small portion of these tasks. For example, Claude Sonnet 4.6 achieves only 33.3%, which exposes gaps in current AI agents. Progress on ClawBench brings us closer to AI agents that can function as general-purpose assistants.

URL PDF HTML 收藏
2603.21876 2026-07-21 cs.CV 版本更新

Thermal Topology Collapse: Universal Physical Patch Attacks on Infrared Vision Systems

热拓扑坍缩:针对红外视觉系统的通用物理贴片攻击

Chengyin Hu, Yikun Guo, Yuxian Dong, Qike Zhang, Kalibinuer Tiliwalidi, Yiwei Wei, Haitao Shi, Jiujiang Guo, Jiahuan Long, Xiang Chen

机构 * China University of Petroleum-Beijing at Karamay(中国石油大学(北京)克拉玛依校区) University of Electronic Science and Technology of China(电子科技大学) College of Intelligence and Computing, Tianjin University(天津大学智能与计算学部) School of Software, Shandong University(山东大学软件学院) Shanghai Jiaotong University(上海交通大学)

AI总结 本文提出通用物理贴片攻击方法UPPA,通过几何约束参数化贝塞尔块建模扰动,利用粒子群优化算法实现全局数据分布统一优化,实现动态变形下的拓扑稳定性,实验表明其在无在线计算开销下具有高攻击成功率和强跨域泛化能力。

详情
AI中文摘要

尽管红外行人检测器已被广泛应用于视觉感知任务,但其对物理对抗攻击的易受攻击性正变得越来越明显。现有物理攻击方法主要依赖实例特定的在线优化和刚性图案设计,导致部署成本高且物理鲁棒性不足。为解决这些限制,本文提出了通用物理贴片攻击(UPPA),这是首个在红外领域中的通用物理攻击方法。该方法利用几何约束参数化贝塞尔块来建模扰动,并利用粒子群优化(PSO)算法在全局数据分布上进行统一优化,从而在动态变形下保持拓扑稳定性。在物理部署阶段,我们将优化的数字扰动转化为物理冷贴片,实现连续且平滑的低温分布,自然符合红外成像的热辐射特性。大量实验表明,UPPA在无任何在线计算开销的情况下实现了出色的物理攻击成功率,同时表现出强跨域泛化能力和可靠的黑盒转移性。

英文摘要

Infrared pedestrian detectors are increasingly deployed in all-weather perception systems, but their robustness against physical adversarial attacks remains insufficiently understood. Existing infrared physical attacks are mostly instance-specific, requiring perturbations to be optimized for particular samples, poses, or scenes, which limits scalability under changing deployment conditions. This paper proposes Universal Physical Patch Attack (UPPA), a universal cold-patch framework for infrared pedestrian detection. UPPA is built on the observation that infrared physical perturbations should exploit smooth, low-frequency thermal structures rather than visible-light texture patterns. It represents the attack carrier as topology-constrained Bézier Curved-Blocks, providing a compact and manufacturable geometric parameterization, and optimizes one shared perturbation with Particle Swarm Optimization (PSO) under Thin Plate Spline (TPS) deformation and Expectation over Transformation (EOT) imaging transformations. The optimized pattern is then deployed as wearable cold patches without sample-specific re-optimization during deployment. Experiments on five infrared datasets and nine pedestrian detectors show consistent digital attack performance, strong cross-dataset and cross-model transferability, and a 92.59\% attack success rate in real-world physical experiments. Ablation, visualization, and defense analyses show that Curved-Blocks disrupt pedestrian thermal feature aggregation and remain difficult for image-restoration-style defenses to remove. These results reveal a practical universal physical vulnerability in current infrared pedestrian detection systems and provide a benchmark for robustness evaluation.

URL PDF HTML 收藏
2602.03061 2026-07-21 cs.LG cs.AI math.ST stat.ME stat.ML stat.TH 版本更新

Evaluating LLMs When They Do Not Know the Answer: Statistical Evaluation of Mathematical Reasoning via Comparative Signals

评估LLMs在不知道答案时的性能:通过比较信号进行数学推理的统计评估

Zihan Dong, Zhixian Zhang, Yang Zhou, Can Jin, Ruijia Wu, Linjun Zhang

机构 * Rutgers University(罗格斯大学) Shanghai Jiao Tong University(上海交通大学)

AI总结 本文提出一种统计高效的方法,通过结合标准结果与成对比较信号,提升LLM数学推理能力评估的准确性和稳定性。

详情
AI中文摘要

评估LLM中的数学推理能力受到基准数据量有限和模型固有随机性的影响,导致准确性估计高方差和跨平台排名不稳定。在困难问题上,LLM可能无法生成正确最终答案,但仍然能提供可靠的成对比较信号,表明两个候选解决方案中哪个更好。我们利用这一观察,设计了一个统计上高效的评估框架,结合标准标记结果与通过让模型判断辅助推理链获得的成对比较信号。将这些比较信号视为控制变量,我们开发了一种基于高效影响函数(EIF)的半参数估计器,适用于辅助推理链被观察的场景。这产生了一个一步估计器,实现了半参数效率界,保证了严格方差减少,且在原理上允许渐近正态性以进行不确定性量化。在模拟中,我们的一步估计器显著提高了排名准确性,随着模型输出噪声增长,收益增加。在GPQA Diamond、AIME 2025和GSM8K上的实验进一步展示了更精确的性能估计和更可靠的模型排名,特别是在小样本情况下,传统评估相当不稳定。

英文摘要

Evaluating mathematical reasoning in LLMs is constrained by limited benchmark sizes and inherent model stochasticity, yielding high-variance accuracy estimates and unstable rankings across platforms. On difficult problems, an LLM may fail to produce a correct final answer, yet still provide reliable pairwise comparison signals indicating which of two candidate solutions is better. We leverage this observation to design a statistically efficient evaluation framework that combines standard labeled outcomes with pairwise comparison signals obtained by having models judge auxiliary reasoning chains. Treating these comparison signals as control variates, we develop a semiparametric estimator based on the efficient influence function (EIF) for the setting where auxiliary reasoning chains are observed. This yields a one-step estimator that achieves the semiparametric efficiency bound, guarantees strict variance reduction over naive sample averaging, and admits asymptotic normality for principled uncertainty quantification. Across simulations, our one-step estimator substantially improves ranking accuracy, with gains increasing as model output noise grows. Experiments on GPQA Diamond, AIME 2025, and GSM8K further demonstrate more precise performance estimation and more reliable model rankings, especially in small-sample regimes where conventional evaluation is pretty unstable.

URL PDF HTML 收藏
2602.02402 2026-07-21 cs.RO cs.AI cs.CV physics.app-ph 版本更新

SoMA: A Real-to-Sim Neural Simulator for Robotic Soft-body Manipulation

SoMA:一种用于机器人软体操作的实-仿真神经模拟器

Mu Huang, Hui Wang, Kerui Ren, Linning Xu, Yunsong Zhou, Mulin Yu, Bo Dai, Jiangmiao Pang

机构 * Fudan University, China(复旦大学) Shanghai Artificial Intelligence Laboratory, China(上海人工智能实验室) Shanghai Jiao Tong University, China(上海交通大学) The Chinese University of Hong Kong, China(香港中文大学) The University of Hong Kong, China(香港大学)

AI总结 SoMA是一种用于机器人软体操作的实-仿真神经模拟器,通过统一的潜在神经空间实现可控、稳定的长周期操作和泛化能力。

Comments Project page: https://city-super.github.io/SoMA/

详情
AI中文摘要

在丰富的相互作用下模拟可变形物体仍然是实-仿真机器人操作中的基本挑战,其动力学由环境效应和机器人动作共同驱动。现有模拟器依赖于预定义的物理或数据驱动的动力学,而没有机器人条件控制,限制了精度、稳定性和泛化能力。本文提出了SoMA,一种3D高斯点模拟器,用于软体操作。SoMA在统一的潜在神经空间中耦合可变形动力学、环境力和机器人关节动作,实现端到端的实-仿真模拟。通过学习的高斯点建模相互作用,实现了可控、稳定的长周期操作和泛化能力,超越观察轨迹而无需预定义的物理模型。SoMA通过提高20%的重演精度和泛化能力,在现实世界机器人操作中实现了复杂任务如长周期布料折叠的稳定模拟。

英文摘要

Simulating deformable objects under rich interactions remains a fundamental challenge for real-to-sim robot manipulation, with dynamics jointly driven by environmental effects and robot actions. Existing simulators rely on predefined physics or data-driven dynamics without robot-conditioned control, limiting accuracy, stability, and generalization. This paper presents SoMA, a 3D Gaussian Splat simulator for soft-body manipulation. SoMA couples deformable dynamics, environmental forces, and robot joint actions in a unified latent neural space for end-to-end real-to-sim simulation. Modeling interactions over learned Gaussian splats enables controllable, stable long-horizon manipulation and generalization beyond observed trajectories without predefined physical models. SoMA improves resimulation accuracy and generalization on real-world robot manipulation by 20%, enabling stable simulation of complex tasks such as long-horizon cloth folding.

URL PDF HTML 收藏
2512.24679 2026-07-21 cs.AI eess.SP 版本更新

Multi-modal cross-domain mixed fusion model with dual disentanglement for fault diagnosis under unseen working conditions

用于未知工况下故障诊断的具有双重解缠的多模态跨域混合融合模型

Pengcheng Xia, Yixiang Huang, Chengjin Qin, Chengliang Liu

机构 * State Key Laboratory of Mechanical System and Vibration(机械系统与振动国家重点实验室) Shanghai Jiao Tong University(上海交通大学)

AI总结 针对未知工况下故障诊断问题,提出具有双重解缠的多模态跨域混合融合模型,通过双重解缠框架、跨域混合融合策略和三模态融合机制,实现多模态表示学习与域泛化,实验验证该方法优于先进方法及各组件有效性。

Comments Accepted for publication in Mechanical Systems and Signal Processing

Journal ref Mechanical Systems and Signal Processing, Volume 258, 2026, 114693

详情
AI中文摘要

智能故障诊断已成为确保机械可靠性不可或缺的技术。然而,现有方法在模型于未知工况下测试的实际场景中性能显著下降,且域适应方法依赖目标域样本。此外,多数现有研究依赖单模态传感信号,忽视多模态信息互补性。本文提出用于故障诊断的具有双重解缠的多模态跨域混合融合模型。开发双重解缠框架解耦模态不变与特定模态特征以及域不变与特定域表示,设计跨域混合融合策略增强模态和域多样性,引入三模态融合机制整合多模态异构信息。在未知恒定和时变工况下对感应电机故障诊断进行大量实验,结果表明该方法优于先进方法,消融研究验证各组件及多模态融合有效性。代码可通过链接获取。

英文摘要

Intelligent fault diagnosis has become an indispensable technique for ensuring machinery reliability. However, existing methods suffer significant performance decline in real-world scenarios where models are tested under unseen working conditions, while domain adaptation approaches are limited to their reliance on target domain samples. Moreover, most existing studies rely on single-modal sensing signals, overlooking the complementary nature of multi-modal information for improving model generalization. To address these limitations, this paper proposes a multi-modal cross-domain mixed fusion model with dual disentanglement for fault diagnosis. A dual disentanglement framework is developed to decouple modality-invariant and modality-specific features, as well as domain-invariant and domain-specific representations, enabling both comprehensive multi-modal representation learning and robust domain generalization. A cross-domain mixed fusion strategy is designed to randomly mix modality information across domains for modality and domain diversity augmentation. Furthermore, a triple-modal fusion mechanism is introduced to adaptively integrate multi-modal heterogeneous information. Extensive experiments are conducted on induction motor fault diagnosis under both unseen constant and time-varying working conditions. The results demonstrate that the proposed method consistently outperforms advanced methods and comprehensive ablation studies further verify the effectiveness of each proposed component and multi-modal fusion. The code is available at: https://github.com/xiapc1996/MMDG.

URL PDF HTML 收藏
2607.16019 2026-07-20 cs.LG 新提交

Presentation, Not Mechanism: A Render Confound in Deprecation-Aware Memory Evaluation

呈现,而非机制:可感知弃用的内存评估中的渲染混淆

Zhaoyang Jiang, Zhizhong Fu, Zicheng Li, Yunsoo Kim, Jiacong Mi, Xuanqi Peng, Fei Teng, Honghan Wu

机构 * School of Health & Wellbeing, University of Glasgow(格拉斯哥大学健康与福祉学院) Department of Respiratory and Critical Care Medicine, Shanghai Sixth People’s Hospital, Shanghai Jiao Tong University School of Medicine(上海交通大学医学院附属第六人民医院呼吸与危重症医学科) School of Life Science and Technology, University of Electronic Science and Technology of China(电子科技大学生命科学与技术学院) Institute of Health Informatics, University College London(伦敦大学学院健康信息学研究所)

AI总结 研究人工智能系统从自我修订记录检索信息时面临的证据状态修订问题,通过比较多种方法,发现渲染对评估有混淆影响,提出内存评估应固定渲染,弃用感知系统应采用覆盖查询的最粗保留状态。

详情
AI中文摘要

人工智能系统越来越多地从自我修订的记录中检索信息,如问题线程、百科历史、政策日志和长对话等。挑战不仅在于找到相关证据,还在于确定哪些声明仍然有效、哪些已被取代以及何时弃权。结构化内存有望通过类型化边、时间更新和冲突状态来解决此问题,但评估通常会同时改变机制和提示呈现。我们将此作为证据状态修订进行研究,在来自GitHub、多仓库问题历史、维基百科和DyKnow风格时间流的2907个高度一致的问题上比较了平面检索、粗粒度边失效和细粒度修订账本。一个渲染匹配的对照组(相同布局,禁用弃用)揭示了核心混淆:当一个值被更改并随后恢复时,修订账本似乎比平面基线高出0.182,但几乎所有增益都来自更简单的呈现;细粒度机制残余与零无显著差异(两个评判组中为+0.021至+...

英文摘要

AI systems increasingly retrieve from records that revise themselves: issue threads, encyclopedic histories, policy logs, and long conversations. The challenge is not only finding relevant evidence, but deciding which claims remain in force, which were superseded, and when to abstain. Structured memories promise to solve this with typed edges, temporal updates, and conflict status, yet evaluations often change mechanism and prompt presentation together. We study this as Evidence-State Revision, comparing flat retrieval, coarse edge invalidation, and fine-grained RevisionLedger on 2,907 high-agreement questions from GitHub, multi-repo issue histories, Wikipedia, and DyKnow-style temporal streams. A render-matched control (same layout, deprecation disabled) reveals the central confound: when a value is changed and later restored, RevisionLedger appears to beat a flat baseline by +0.182, but almost all the gain comes from easier presentation; the fine-grained mechanism residual is indistinguishable from zero (+0.021 to +0.025 across two judge families). After presentation is controlled, coarse invalidation is the only mechanism that pays for current-state queries, beating the fine ledger by 0.084; the same query-sufficiency principle says provenance mainly needs retained invalidated evidence, not richer typing. Memory evaluations should hold render fixed, and deprecation-aware systems should deploy the coarsest retained state that covers their queries.

URL PDF HTML 收藏
2607.15650 2026-07-20 cs.CV 新提交

DiTango: Cost-Effective Parallel Diffusion Generation with Selective Attention State Reuse

DiTango:基于选择性注意力状态重用的经济高效并行扩散生成

Yuyang Chen, Runxin Zhong, Zan Zong, Hengjie Li, Yuyang Jin, Jidong Zhai

机构 * Shanghai Jiao Tong University(上海交通大学) Shanghai AI Laboratory(上海人工智能实验室) Tsinghua University(清华大学) University of Science and Technology Beijing(北京科技大学) PJlab(PJ实验室)

AI总结 研究针对扩散变换器并行化技术在多节点环境下通信开销大的问题,提出DiTango框架,通过选择性注意力状态机制及锚点引导的状态选择规划器等,在多节点设置中实现加速并保持生成质量。

Journal ref The 35th International Symposium on High-Performance Parallel and Distributed Computing (HPDC'26), July 13--16, 2026, Cleveland, OH, USA

详情
AI中文摘要

人工智能生成内容的最新进展推动了扩散变换器(DiTs)在高分辨率、长时间内容生成中的广泛应用。虽然并行化技术加速了扩散推理,但由于多节点环境中的过度通信开销,它们面临着重大的可扩展性挑战。我们观察到上下文并行(CP)中的序列分区表现出明显的异质性:空间上相邻的分区对注意力计算结果的贡献更大。通过将这种异构模式映射到分层通信拓扑,我们可以以降低的通信成本访问高贡献分区。这一见解激发了我们新颖的选择性注意力状态机制,该机制在去噪步骤中战略性地平衡部分注意力计算和历史结果重用。我们提出了DiTango,一种用于DiT生成的高效并行框架。DiTango具有一个锚点引导的状态选择规划器,用于优化每个分区的计算重用决策,并辅以一个运行时,用于编排高效的以状态为中心的操作。这种设计在保持生成质量的同时实现了卓越的系统效率。对流行扩散模型的实验评估表明,DiTango在多节点设置中实现了高达1.9倍的端到端和3.2倍的注意力加速,且具有近线性缩放,同时保持与最先进方法相当的生成质量。

英文摘要

Recent advances in AI-generated content have driven widespread adoption of Diffusion Transformers (DiTs) for high-resolution, long-duration content generation. While parallelization techniques accelerate diffusion inference, they face significant scalability challenges due to excessive communication overhead in multi-node environments. We observe that sequence partitions in Context Parallelism (CP) exhibit distinct heterogeneity: spatially proximate partitions contribute more significantly to attention computation results. By mapping this heterogeneous pattern to hierarchical communication topology, we can access high-contribution partitions with reduced communication cost. This insight motivates our novel selective attention state mechanism that strategically balances partial attention computation and historical result reuse across denoising steps. We present DiTango, an efficient parallel framework for DiT generation. DiTango features an anchor-guided state selection planner that optimizes computation-reuse decisions for each partition, complemented by a runtime that orchestrates efficient state-centric operations. This design achieves superior system efficiency while preserving generation quality. Experimental evaluation on popular diffusion models demonstrates that DiTango achieves up to 1.9x end-to-end and 3.2x attention speedup with near-linear scaling in multi-node settings, while maintaining generation quality comparable to state-of-the-art approaches.

URL PDF HTML 收藏
2607.11436 2026-07-20 cs.AI 版本更新

The Ebb and Flow of Multimodal Focus: Scheduling Visual Relay Windows for Grounded VLM Reasoning

多模态焦点的潮起潮落:为基于视觉的语言模型推理调度视觉中继窗口

Wencheng Ye, Yi Bin, Yujuan Ding, Hongye Fang, Zheng Wang, Xing Xu, Jingkuan Song, Yun Zhang, Sirui Da, Heng Tao Shen

机构 * School of Computer Science and Technology, Tongji University(同济大学计算机科学与技术学院) School of Fashion and Textiles, The Hong Kong Polytechnic University(香港理工大学纺织及制衣学院) School of Artificial Intelligence, Shanghai Jiao Tong University(上海交通大学人工智能学院)

AI总结 研究视觉语言模型视觉证据不稳定问题,通过剖析其内部多模态注意力焦点的三阶段分布,提出TRACE框架,该框架能自适应控制推理,在多模型和多基准测试中显著提升基于证据的多模态推理能力。

详情
AI中文摘要

视觉语言模型在多模态推理基准测试中越来越成功,但其视觉证据一旦进入语言堆栈往往变得不稳定,削弱了基于证据的推理。为理解这种脆弱性,我们通过机制视角研究视觉语言模型的内部动态,发现多模态注意力焦点在深度上有稳定的三阶段重新分布。我们将中间阶段作为视觉中继窗口(VRW),并表明其几何形状随任务需求变化,与基于基础的生成有因果关系,能区分无支持的答案和更强的推理轨迹。在此基础上,我们提出TRACE,一个具有轻量级训练模块的任务自适应推理时间控制框架。它在预填充期间重塑中继分配,并在解码期间交接后保留组装的视觉支持。在四个开放权重的视觉语言模型主干和七个基准测试中,TRACE在对基础敏感的设置上有显著提升,平均提高4.33分,最高提高6.6分,同时也改善了推理繁重的任务。这些结果表明,明确控制跨深度的多模态焦点为加强基于证据的多模态推理提供了一种统一且有效的机制。

英文摘要

Vision-language models increasingly succeed on multimodal reasoning benchmarks, yet their visual evidence often becomes unstable once it enters the language stack, weakening evidence-grounded reasoning. To understand this fragility, we examine the internal dynamics of VLMs through a mechanistic lens and uncover a stable three-stage redistribution of multimodal attention focus across depth: an early question-conditioned organization, a critical middle visual-dominant relay, and a late return to answer formation. We operationalize the middle phase as the Visual Relay Window (VRW), and show that its geometry varies with task demand, is causally tied to grounded generation, and distinguishes unsupported answers from stronger reasoning trajectories. Guided by this internal rhythm, we propose TRACE, a task-adaptive inference-time control framework with lightweight trained modules. It reshapes relay allocation during prefill and preserves assembled visual support after handoff during decoding. Across four open-weight VLM backbones and seven benchmarks, TRACE delivers large gains on grounding-sensitive settings, improving them by 4.33 points on average and by up to 6.6 points, while also improving reasoning-heavy tasks. These results show that explicitly controlling multimodal focus across depth offers a unified and effective mechanism for strengthening evidence-grounded multimodal reasoning.

URL PDF HTML 收藏