arXivDaily arXiv每日学术速递 周一至周五更新

高校专区

Huazhong University of Science and Technology(华中科技大学)

至 收录 690
2607.17849 2026-07-21 cs.HC cs.CL cs.CV 新提交

AlphaOracle: Oracle bone script decipherment via human-workflow-inspired deep learning

AlphaOracle:通过受人工工作流程启发的深度学习进行甲骨文破译

Yuliang Liu, Haisu Guan, Pengjie Wang, Xinyu Wang, Jinpeng Wan, Kaile Zhang, Handong Zheng, Xingchen Liu, Zhebin Kuang, Huanxin Yang, Bang Li, Yongge Liu, Lianwen Jin, Xiang Bai

机构 * School of Software Engineering, Huazhong University of Science and Technology(华中科技大学软件学院) School of Electronic Information Engineering, South China University of Technology(华南理工大学电子信息学院) Key Laboratory of Oracle Bone Inscriptions Information Processing, Anyang Normal University(安阳师范学院甲骨文信息处理重点实验室)

AI总结 针对约3000个未破译甲骨文字符的问题,提出受人工工作流程启发的AlphaOracle框架,经多阶段管道进行甲骨文破译,与专家解读高度一致,还能减少分析时间,为甲骨文及其他未破译文字研究提供参考。

Comments Accepted by The Innovation 2026

Journal ref The Innovation 7(11), 101462, 2026

详情
AI中文摘要

由于铭文残缺和证据稀少,4500个甲骨文字符中约有3000个仍未被破译。当前的人工智能方法无法复制整合形式分析、上下文语义和文献学推理的专家工作流程。我们引入了AlphaOracle,这是一个受人工工作流程启发的框架,它使用迄今为止最大的数字化语料库将甲骨文破译系统化。其多阶段管道包括:(i)拓片解析;(ii)基于部首的形态分析和历时建模;(iii)上下文检索与语义对齐;(iv)对照经典文献进行文献学验证。每个阶段都会生成明确的、带有置信度权重的证据链,最终生成可供学术验证的可解释报告。在多个测试字符上,AlphaOracle的解读与专家解读高度一致。在对86位领域专家的研究中,它将分析时间减少了64%,79%的参与者认为它非常有用。值得注意的是,AlphaOracle将字符“老”解析为地名或氏族名称,为商代行政和社会解读提供了具体修订。这些结果表明,与文献学实践相结合的计算方法可以促进甲骨文研究,并为其他未破译文字的研究提供概念参考。

英文摘要

Approximately 3,000 of the 4,500 oracle bone script (OBS) characters remain undeciphered due to fragmentary inscriptions and sparse evidence. Current AI approaches fail to replicate expert workflows that integrate form analysis, contextual semantics, and philological reasoning. We introduce AlphaOracle, a human-workflow-inspired framework that systematizes OBS decipherment using the largest digitized corpus to date. Its multi-stage pipeline comprises: (i) rubbing parsing; (ii) radical-based morphological analysis with diachronic modeling; (iii) contextual retrieval with semantic alignment; and (iv) philological validation against classical sources. Each stage generates explicit, confidence-weighted evidence chains, culminating in interpretable reports for scholarly verification. Across multiple test characters, AlphaOracle's readings strongly agreed with expert interpretations. In a study of 86 domain specialists, it reduced analysis time by 64% and 79% of participants rated it highly useful. Notably, AlphaOracle resolves the character "Lao" as a toponymic or clan designation, offering concrete revisions to Shang administrative and social interpretations. These results suggest that computational methods aligned with philological practice can facilitate OBS research and provide a conceptual reference for studies of other undeciphered scripts.

URL PDF HTML 收藏
2607.17332 2026-07-21 cs.RO 新提交

Multi-Resolution Voxelized Map-Based Stereo Visual-Inertial Odometry

基于多分辨率体素化地图的立体视觉惯性里程计

Shuyi Pan, Hangtian Wang, Zhaoxing Zhang, Chengliang Zhang, Zikang Yuan, Xin Yang

机构 * Huazhong University of Science and Technology(华中科技大学) AI Chip Center for Emerging Smart Systems (AC-CESS)(新兴智能系统人工智能芯片中心)

AI总结 研究视觉惯性里程计中姿态估计问题,提出多分辨率先验地图构建方法及基于地图的VIO系统,通过体素化地图、锥形索引策略和DDA算法,减少数据传输量与计算负载,实验证明该系统能在少数据传输下实现准确姿态估计。

Comments 9 pages, 4 figures, 6 tables. Accepted to appear in the 2026 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2026)

详情
AI中文摘要

结合先验地图可显著提高视觉惯性里程计(VIO)中姿态估计的准确性和鲁棒性。然而,地图数据量庞大且传输带宽有限,使得在边缘设备上持续加载本地地图不切实际。本文提出一种多分辨率先验地图构建方法及相应的基于地图的VIO系统。先验地图在多个分辨率下进行体素化,每个体素仅保留一个地图点。在线VIO操作期间,采用锥形索引策略将边缘设备上的二维特征与三维地图点相关联。通过三维数字微分分析器(DDA)算法,根据当前位置到三维点的距离确定锥体截距,从而选择合适的分辨率级别并检索相应体素内的唯一地图点。该方法最小化了传输所需的数据量和数据关联期间的计算负载。在两个公共数据集上进行的大量实验表明,我们的系统在需要最少数据传输的情况下实现了准确的姿态估计。

英文摘要

Incorporating prior maps significantly enhances the accuracy and robustness of pose estimation in visual-inertial odometry (VIO). However, the large data volume of such maps, combined with limited transmission bandwidth, makes it impractical to continuously load local maps onto an edge device. In this paper, we propose a multi-resolution prior map construction method and a corresponding map-based VIO system. The prior map is voxelized at multiple resolutions, with each voxel retaining only a single map point. During online VIO operation, a cone-shaped indexing strategy associates 2D features on the edge device with 3D map points. The cone's intercept is determined by the distance from the current position to the 3D points, enabling the selection of the appropriate resolution level and the retrieval of the unique map point within the corresponding voxel via a 3D digital differential analyzer (DDA) algorithm. This approach minimizes both the volume of data required for transmission and the computational load during data association. Extensive experiments on two public datasets demonstrate that our system achieves accurate pose estimation while requiring minimal data transmission.

URL PDF HTML 收藏
2607.17250 2026-07-21 cs.CL 新提交

EvolvingWorld: An Open-Schema Framework for Co-Evolving Role-Play Agents and World Model in Interactive Literary World

EvolvingWorld:用于交互式文学世界中角色与世界模型协同进化的开放架构框架

Qing Zong, Yue Guo, Mengxin Yang, Yiwen Guo, Yangqiu Song

机构 * Hong Kong University of Science and Technology(香港科技大学) LIGHTSPEED(光速) Huazhong University of Science and Technology(华中科技大学)

AI总结 研究交互式文学世界中角色与世界协同进化问题,提出EvolvingWorld框架,含角色智能体和世界模型两个耦合模块,制定7个可训练任务,构建数据集并引入评估协议,实验证明其能有效改进长期模拟。

详情
AI中文摘要

本文介绍了EvolvingWorld,一个用于交互式文学世界中角色与世界协同进化的框架和基准。现有系统要么将交互式文学模拟视为静态角色模仿,要么视为孤立场景生成,无法捕捉角色和世界如何随时间共同进化。为解决此问题,EvolvingWorld将文学模拟建模为一个长期过程,其中角色互动、场景推进,且角色和世界状态持续更新。与依赖固定架构的先前系统不同,EvolvingWorld采用开放架构框架来支持跨不同文学世界的模拟。该框架由两个耦合模块组成:一个用于多角色角色扮演和持续角色档案进化的角色智能体,以及一个基于大语言模型的用于全局和位置/实体级状态维护及场景推进的世界模型。基于此架构,制定了7个可训练任务用于场景初始化、互动生成和状态更新。从57本书构建数据集,产生138596个监督训练样本和222个测试快照。还引入了涵盖10个维度和20个指标的轨迹级大语言模型评判评估协议。实验表明,EvolvingWorld能通过有效维持持续、连贯的角色和世界发展来改进长期模拟。

英文摘要

This paper introduces EvolvingWorld, a framework and benchmark for character and world co-evolution in interactive literary worlds. Existing systems either treat interactive literary simulation as static persona imitation or isolated scene generation, failing to capture how characters and worlds evolve together over time. To address this, EvolvingWorld models literary simulation as a long-horizon process where characters interact, scenes progress, and character and world states are persistently updated. Unlike prior systems relying on fixed schemas, EvolvingWorld adopts an open-schema framework to support simulation across diverse literary worlds. The framework consists of two coupled modules: a Character Agent for multi-character role-play and persistent profile evolution, and an LLM-based World Model for global and location/entity-level state maintenance and scene progression. Based on this architecture, we formulate 7 trainable tasks for scene initialization, interaction generation, and state update. We construct a dataset from 57 books, producing 138,596 supervised training samples and 222 snapshots for testing. Furthermore, we introduce a trajectory-level LLM-as-Judge evaluation protocol spanning 10 dimensions and 20 metrics. Experiments show that EvolvingWorld can improve long-horizon simulation by effectively maintaining persistent, coherent character and world development.

URL PDF HTML 收藏
2607.16308 2026-07-21 cs.CV cs.AI 新提交

DAUPNet: Domain-Aware Uncertainty Modeling for Reliable Prototype Discrimination in Cross-Domain Few-Shot Semantic Segmentation

DAUPNet:跨域少样本语义分割中用于可靠原型区分的域感知不确定性建模

Lei Yuan, Zhongxu Hu, Jingyi Wen, Pengxing Yi

机构 * Huazhong University of Science and Technology(华中科技大学)

AI总结 针对跨域少样本语义分割中因域偏移致原型匹配不可靠的问题,提出DAUPNet框架,通过协调特征、概率表示原型及用不确定性调节优化,在四个标准目标域取得较好平均mIoU,为跨域少样本语义分割提供了可靠方法。

详情
AI中文摘要

跨域少样本语义分割主要被表述为学习域不变表示或改善支持-查询对应关系。然而,大的域偏移仍使原型匹配不可靠。我们提出DAUPNet,一个将跨域原型匹配重新表述为不确定性感知原型区分的统一框架。它先协调层次化支持-查询特征,再概率性表示前景和背景原型,最后用估计的不确定性调节对比优化。在四个标准目标域上,DAUPNet在单样本和五样本设置下分别取得72.6%和76.7%的平均mIoU,证明了该方法的有效性。

英文摘要

Cross-domain few-shot semantic segmentation (CD-FSS) has predominantly been formulated as learning domain-invariant representations or improving support-query correspondence. Nevertheless, large domain shifts still make prototype matching unreliable: inconsistent hierarchical responses corrupt the support representation, deterministic prototypes cannot express boundary and appearance ambiguity, and treating prototypes with different reliability equally during optimization weakens foreground-background separation. We therefore propose DAUPNet, a unified framework that reformulates cross-domain prototype matching as uncertainty-aware prototype discrimination. DAUPNet first harmonizes hierarchical support-query features to provide stable evidence, then represents foreground and background prototypes probabilistically, and finally uses their estimated uncertainty to regulate contrastive optimization. On four standard target domains, DAUPNet achieves 72.6% and 76.7% average mIoU in the 1-shot and 5-shot settings, respectively, including substantial gains on the two medical domains. These results demonstrate that modeling prototype uncertainty and incorporating it into optimization provides a robust and interpretable approach to CD-FSS under severe domain shift. The code is available at https://github.com/madness-Lei/DAUPNet

URL PDF HTML 收藏
2607.16247 2026-07-21 cs.LG cs.CV 新提交

Self-Evolving Just-In-Time Memory for Proactive Embodied Safety

用于主动具身安全的自进化即时记忆

Bingrui Sima, Lizhong Wang, Xiaoya Lu, Kun He, Xiao Yang

机构 * Huazhong University of Science and Technology(华中科技大学) Tsinghua University(清华大学) Shanghai Jiao Tong University(上海交通大学)

AI总结 研究视觉语言模型在闭环交互中应对动态危险的问题,提出自进化即时记忆框架,含RSG、事实记忆和经验记忆,并通过自动测试-验证-写入循环完善元技能,实验证明该框架大幅提升安全成功率且不阻碍任务进展。

详情
AI中文摘要

虽然视觉语言模型(VLMs)使具身智能体能够执行复杂的家庭任务,但它们在闭环交互中难以主动应对动态出现的危险。现有安全方法常依赖运行时护栏来阻止不安全行为或导致过度谨慎,严重阻碍任务进展。为打破安全与进展的权衡,我们引入自进化即时记忆框架,将具身安全从阻碍进展的护栏转变为主动减轻危险。该框架由用于部分可观测下持续安全相关状态跟踪的风险充足拓扑信念图(RSG)、用于精确危险预测的基于智能体的事实记忆以及注入程序元技能以指导可执行、保持进展的减轻危险的经验记忆组成。此外,我们提出自动测试-验证-写入循环,让智能体在测试时能从执行轨迹中不断完善减轻危险的元技能。在IS-Bench上的实验表明,我们的框架大幅提高了多个VLM主干的安全成功率(如在Qwen3-VL-8B上提高30.3%),使智能体能够主动减轻危险而不阻碍任务进展。

英文摘要

While Vision-Language Models (VLMs) have empowered embodied agents to execute complex household tasks, they struggle to proactively handle dynamically emerging hazards during closed-loop interactions. Existing safety approaches often rely on runtime guardrails to block unsafe actions or induce excessive caution, which severely stalls task progress instead of actively resolving the underlying risks. To break this safety-progress trade-off, we introduce the Self-Evolving Just-In-Time Memory framework, which reframes embodied safety from progress-stalling guardrails to proactive hazard mitigation. The framework consists of a Risk-Sufficient Topological Belief Graph (RSG) for persistent safety-relevant state tracking under partial observability, an Agency-Grounded Factual Memory for precise hazard anticipation, and an Experience Memory that injects procedural Meta-Skills to guide executable, progress-preserving mitigation. Furthermore, we propose an automated Test-Verify-Write loop, allowing agents to continually refine their mitigation Meta-Skills from execution traces at test time. Experiments on IS-Bench demonstrate that our framework substantially boosts the Safe-Success rate across multiple VLM backbones (e.g., +30.3% on Qwen3-VL-8B), enabling agents to proactively mitigate hazards without stalling task progress. Code is available at https://github.com/DyMessi/JIT-Memory.

URL PDF HTML 收藏
2607.10623 2026-07-21 cs.GR cs.CV 版本更新

LATO.2: Factorized 3D Mesh Generation with Vertex and Topology Flow

LATO.2:基于顶点和拓扑流的因式分解3D网格生成

Hang Long, Tianhao Zhao, Junkai Lin, Youjia Zhang, Huipeng Guo, Rendong Liang, Jiale Xu, Jozef Hladký, Matthias Nießner, Yuanming Hu, Wei Yang

机构 * Huazhong University of Science and Technology(华中科技大学) Meshy AI Technical University of Munich(慕尼黑技术大学)

AI总结 研究针对现有拓扑感知网格生成方法的不足,提出LATO.2框架,将网格生成分解为顶点流和连通性流,由专用变分自编码器支持,具有逐部分生成和拓扑自适应编辑优势,实验证明其超越现有方法。

详情
AI中文摘要

最近,在精心设计的潜在表示上进行流匹配已成为拓扑感知网格生成的强大范例。然而,现有方法在联合潜在空间中联合对顶点和连通性进行建模,将连续的顶点几何与离散的组合结构纠缠在一起,这使流学习变得复杂,并表现为顶点漂移和表面破裂。我们提出了LATO.2,这是一个因式分解流匹配框架,它将网格生成分解为一个顶点流,然后是一个基于已实现顶点的连通性流,两个阶段都锚定在一个共享的粗体素支架上。专用变分自编码器支持这两个阶段,以亚体素精度恢复顶点,并将离散连通性嵌入连续潜在空间。我们展示了这种因式分解的两个独特优势:(i)逐部分生成,其中支架被分割,每个部分以全潜在容量合成,产生比整体潜在更高分辨率的网格;(ii)拓扑自适应编辑,其中操纵第一阶段顶点会诱导相应的连通性,而无需重新优化。实验表明,LATO.2在几何保真度和连通性质量方面超过了现有的拓扑感知网格生成器。

英文摘要

Flow matching over carefully designed latent representations has recently emerged as a powerful paradigm for topology-aware mesh generation. Existing approaches, however, model vertices and connectivity jointly in a joint latent space, entangling continuous vertex geometry with discrete combinatorial structure; this complicates flow learning and manifests as drifting vertices and broken surfaces. We present LATO.2, a factorized flow matching framework that decomposes mesh generation into a vertex flow followed by a connectivity flow conditioned on the realized vertices, with both stages anchored to a shared coarse voxel scaffold. Dedicated VAEs underpin the two stages, recovering vertices at sub-voxel precision and embedding discrete connectivity into a continuous latent space. We demonstrate two advantages unique to this factorization: (i) part-wise generation, in which the scaffold is partitioned and each part synthesized at full latent capacity, yielding substantially higher-resolution meshes than a monolithic latent permits; and (ii) topology-adaptive editing, in which manipulating first-stage vertices induces the corresponding connectivity without re-optimization. Experiments show that LATO.2 surpasses state-of-the-art topology-aware mesh generators in geometric fidelity and connectivity quality.

URL PDF HTML 收藏
2606.27088 2026-07-21 cs.CV 版本更新

SubdivAR: Autoregressive Next-Scale Prediction for Neural Mesh Subdivision

SubdivAR: 用于神经网格细分的自回归下一尺度预测

Huipeng Guo, Zikai Song, Hang Long, Jielei Zhang, Wenbing Li, Junkai Lin, Tianhao Zhao, Jinshen Zhang, Tianle Guo, Wei Yang

机构 * Huazhong University of Science and Technology(华中科技大学)

AI总结 提出SubdivAR框架,将网格细分建模为自回归下一尺度预测,利用混合拓扑感知Transformer结合全局语义与局部拓扑特征,在FII-40K数据集上显著降低Hausdorff距离和Chamfer距离。

详情
AI中文摘要

网格细分是将粗糙、可编辑网格转换为高分辨率表面的基本操作,广泛应用于数字资产创建。经典的基于规则的方案依赖于固定的局部细化规则,通常产生过度平滑的表面。最近的神经细分方法改善了细节合成,但仍受限于局部建模且泛化能力有限。我们提出SubdivAR,一种基于我们提出的网格自回归表示(MAR)的神经网格细分框架。MAR将不同细分级别的网格排列成有序的尺度序列,将细分重新表述为自回归下一尺度预测。为支持这一表述,我们引入了一种混合拓扑感知Transformer,它将全局语义注意力与拓扑约束的局部特征聚合相结合。SubdivAR采用下一尺度坐标预测范式,在每个细化阶段回归顶点偏移,以保留细分拓扑同时恢复细粒度几何细节。为实现可靠学习,我们构建了FII-40K,一个包含近40,000个高质量网格并具有多级细分监督的精选数据集。实验表明,SubdivAR优于最先进的基线,将Hausdorff距离和Chamfer距离分别降低了18.8%和14.2%,并在复杂的开放曲面几何上展现出强大的鲁棒性。

英文摘要

Mesh subdivision is a fundamental operation for converting coarse, editable meshes into high-resolution surfaces, with broad applications in digital asset creation. Classical rule-based schemes rely on fixed local refinement rules and often produce over-smoothed surfaces. Recent neural subdivision methods improve detail synthesis, but remain constrained by local modeling and exhibit limited generalizability. We present SubdivAR, a neural mesh subdivision framework based on our proposed Mesh Autoregressive Representation (MAR). MAR arranges meshes at different subdivision levels into an ordered scale sequence, reformulating subdivision as autoregressive next-scale prediction. To support this formulation, we introduce a Hybrid Topology-Aware Transformer that combines global semantic attention with topology-constrained local feature aggregation. SubdivAR adopts a next-scale coordinate prediction paradigm, regressing vertex offsets at each refinement stage to preserve subdivision topology while recovering fine-grained geometric details. To enable reliable learning, we construct FII-40K, a curated dataset of nearly 40,000 high-quality meshes with multi-level subdivision supervision. Experiments show that SubdivAR outperforms state-of-the-art baselines, reducing Hausdorff Distance and Chamfer Distance by 18.8% and 14.2%, respectively, and demonstrates strong robustness on complex open-surface geometries.

URL PDF HTML 收藏
2607.15321 2026-07-20 cs.CV 新提交

Rethinking the Readout: Unlocking Video Backbones for AI-Generated Video Detection

重新思考读出方式:解锁用于人工智能生成视频检测的视频主干网络

Manni Cui, Ziheng Qin, ZiAn Wang, Ruiqi Liu, Dianyuan Zou, Jianglan Wei, Han Zhou, Yu Liu, Jingrui Xu, Wenhao Wang, Zhenyu Zhang

机构 * Huazhong University of Science and Technology(华中科技大学) Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所) Jilin University(吉林大学) Vast Intelligence Lab(旷视智能实验室)

AI总结 研究针对AI生成视频检测中视频主干网络表现不佳的问题,提出速度门控补丁速度剖析(V-PVP)方法,通过替换聚合层激活冻结主干网络时间潜力,提升检测性能,在AIGVDBench上达95.28的AUC。

详情
AI中文摘要

人工智能生成的视频(AIGV)通常包含由帧间不一致而非单个帧内产生的细微时间伪影。因此,捕获此类伪影的检测器应受益于视频预训练主干网络而非仅图像预训练的探测器。然而,在实践中,具有标准全局读出的视频主干网络在AIGV基准测试中往往无法超越强大的图像预训练探测器。我们将这种差距归因于读出中的过度时空聚合。视频预训练主干网络倾向于将每个帧压缩成单个全局描述符。这种压缩抑制了局部补丁级别的时间动态并丢弃了补丁间关系,而这些正是AIGV检测最可靠依赖的线索。基于此,我们提出了速度门控补丁速度剖析(V-PVP),一种轻量级读出方式,仅用补丁速度场上的两个并行流替换聚合层,仅增加约0.5M可训练参数。V-PVP作为一个通用的即插即用模块,在端到端微调及线性探测设置下,能持续提升不同视频主干网络的性能。我们的方法在AIGVDBench上达到了95.28的AUC,同时保持主干网络完全冻结。结果表明,简单地替换聚合层就能重新激活冻结视频主干网络的时间潜力,恢复其在AIGV检测上的优势。代码可在该https链接获取。

英文摘要

AI-generated videos (AIGVs) typically contain subtle temporal artifacts that arise from inter-frame inconsistencies rather than within individual frames. A detector that captures such artifacts should therefore benefit from video pretrained backbones over image only ones. In practice, however, video backbones with standard global readouts often fail to outperform strong image pretrained probes on AIGV benchmarks. We attribute this gap to excessive spatiotemporal aggregation in the readout. Video pretrained backbones tend to compress each frame into a single global descriptor. This compression suppresses local patch level temporal dynamics and discards inter patch relations, which are precisely the cues that AIGV detection most reliably depends on. Based on this, we propose Velocity Gated Patch Velocity Profiling (V-PVP), a lightweight readout that replaces only the aggregation layer with two parallel streams over the patch velocity field, adding only about $0.5$M trainable parameters. V-PVP serves as a general plug-and-play module that consistently improves performance across diverse video backbones under both end-to-end fine-tuning and linear probing settings. Our method reaches \textbf{95.28} AUC on AIGVDBench while keeping the backbone fully frozen. The results show that simply replacing the aggregation layer reactivates the temporal potential of frozen video backbones, restoring their advantage on AIGV detection. Code is available at https://anonymous.4open.science/r/PVP-81B3/.

URL PDF HTML 收藏
2603.19235 2026-07-20 cs.CV cs.RO 版本更新

Generation Models Know Space: Unleashing Implicit 3D Priors for Scene Understanding

生成模型了解空间:解锁隐式3D先验用于场景理解

Xianjin Wu, Dingkang Liang, Tianrui Feng, Kui Xia, Yumeng Zhang, Xiaofan Li, Xiao Tan, Xiang Bai

机构 * Huazhong University of Science and Technology(华中科技大学) Baidu Inc.(百度公司)

AI总结 本文提出通过大规模视频生成模型的隐式空间先验提升场景理解,引入VEGA-3D框架,利用预训练视频扩散模型作为潜在世界模拟器,通过时空特征提取与语义融合增强大语言模型的几何信息,实验验证其在3D场景理解、空间推理和具身操控中的优越性。

Comments Accepted by ECCV 2026

详情
AI中文摘要

尽管多模态大语言模型展现了出色的语义能力,但它们常常面临空间盲区,难以进行细粒度几何推理和物理动态。现有解决方案通常依赖显式的3D模态或复杂的几何架构,受限于数据稀缺性和泛化挑战。在本文中,我们提出了一种范式转变,通过利用大规模视频生成模型中的隐式空间先验。我们假设为了合成时间一致的视频,这些模型本质上学习了稳健的3D结构先验和物理定律。我们引入VEGA-3D(Video Extracted Generative Awareness),一个即插即用的框架,通过重新利用预训练的视频扩散模型作为潜在世界模拟器。通过从中间噪声水平提取时空特征,并通过令牌级自适应门控融合机制将其与语义表示结合,我们增强了大语言模型的密集几何提示,而无需显式的3D监督。在3D场景理解、空间推理和具身操控基准上的广泛实验表明,我们的方法优于最先进的基线,验证了生成先验为物理世界理解提供了可扩展的基础。代码可在https://github.com/H-EmbodVis/VEGA-3D上公开获取。

英文摘要

While Multimodal Large Language Models demonstrate impressive semantic capabilities, they often suffer from spatial blindness, struggling with fine-grained geometric reasoning and physical dynamics. Existing solutions typically rely on explicit 3D modalities or complex geometric scaffolding, which are limited by data scarcity and generalization challenges. In this work, we propose a paradigm shift by leveraging the implicit spatial prior within large-scale video generation models. We posit that to synthesize temporally coherent videos, these models inherently learn robust 3D structural priors and physical laws. We introduce VEGA-3D (Video Extracted Generative Awareness), a plug-and-play framework that repurposes a pre-trained video diffusion model as a Latent World Simulator. By extracting spatiotemporal features from intermediate noise levels and integrating them with semantic representations via a token-level adaptive gated fusion mechanism, we enrich MLLMs with dense geometric cues without explicit 3D supervision. Extensive experiments across 3D scene understanding, spatial reasoning, and embodied manipulation benchmarks demonstrate that our method outperforms state-of-the-art baselines, validating that generative priors provide a scalable foundation for physical-world understanding. Code is publicly available at https://github.com/H-EmbodVis/VEGA-3D.

URL PDF HTML 收藏
2603.12262 2026-07-20 cs.CV 版本更新

Video Streaming Thinking: VideoLLMs Can Watch and Think Simultaneously

视频流思考:VideoLLMs可以同时观看和思考

Yiran Guan, Liang Yin, Dingkang Liang, Jianzhong Ju, Zhenbo Luo, Jian Luan, Yuliang Liu, Xiang Bai

机构 * Huazhong University of Science and Technology(华中科技大学) MiLM Plus Xiaomi Inc.(小米公司)

AI总结 VST提出了一种视频流理解的新范式,通过在流式过程中激活推理,提高实时响应和理解能力,同时在多个基准测试中表现出更高的效率和泛化能力。

Comments Accepted by ECCV 2026, project page https://1ranguan.github.io/VST/

详情
AI中文摘要

在线视频大语言模型(VideoLLMs)在支持响应式、实时交互中发挥关键作用。现有方法专注于流式感知,缺乏同步的逻辑推理流。然而,直接应用测试时缩放方法会导致不可接受的响应延迟。为了解决这一权衡,我们提出了视频流思考(VST),一种新的流式视频理解范式。它支持一边观看一边思考的机制,在流式过程中激活对输入视频片段的推理。这种设计通过在视频播放过程中摊销LLM推理延迟,提高了及时理解和连贯认知,同时保持实时响应能力。此外,我们引入了一个全面的训练后流水线,该流水线整合了VST-SFT,通过结构上适应离线VideoLLM到因果流式推理,以及VST-RL,通过在多轮视频交互环境中进行自我探索提供端到端改进。此外,我们设计了一个自动化训练数据合成流水线,利用视频知识图谱生成高质量的流式问答对,使用实体-关系基础的流式推理链来强制多证据推理和持续关注视频流。广泛评估显示,VST-7B在在线基准测试中表现强劲,例如在StreamingBench上达到79.5%的准确率,在OVO-Bench上达到59.3%的准确率。同时,VST在离线长格式或推理基准测试中仍具竞争力。与Video-R1相比,VST响应速度提高了15.7倍,并在VideoHolmes上实现了+5.4%的改进,证明了其在多样化视频理解任务中的更高效率和强大泛化能力。代码、数据和模型将在https://github.com/1ranGuan/VST上发布。

英文摘要

Online Video Large Language Models (VideoLLMs) play a critical role in supporting responsive, real-time interaction. Existing methods focus on streaming perception, lacking a synchronized logical reasoning stream. However, directly applying test-time scaling methods incurs unacceptable response latency. To address this trade-off, we propose Video Streaming Thinking (VST), a novel paradigm for streaming video understanding. It supports a thinking while watching mechanism, which activates reasoning over incoming video clips during streaming. This design improves timely comprehension and coherent cognition while preserving real-time responsiveness by amortizing LLM reasoning latency over video playback. Furthermore, we introduce a comprehensive post-training pipeline that integrates VST-SFT, which structurally adapts the offline VideoLLM to causal streaming reasoning, and VST-RL, which provides end-to-end improvement through self-exploration in a multi-turn video interaction environment. Additionally, we devise an automated training-data synthesis pipeline that uses video knowledge graphs to generate high-quality streaming QA pairs, with an entity-relation grounded streaming Chain-of-Thought to enforce multi-evidence reasoning and sustained attention to the video stream. Extensive evaluations show that VST-7B performs strongly on online benchmarks, e.g. 79.5% on StreamingBench and 59.3% on OVO-Bench. Meanwhile, VST remains competitive on offline long-form or reasoning benchmarks. Compared with Video-R1, VST responds 15.7 times faster and achieves +5.4% improvement on VideoHolmes, demonstrating higher efficiency and strong generalization across diverse video understanding tasks. Code, data, and models will be released at https://github.com/1ranGuan/VST.

URL PDF HTML 收藏
2512.13636 2026-07-20 cs.CV cs.RO 版本更新

MindDrive: A Vision-Language-Action Model for Autonomous Driving via Online Reinforcement Learning

MindDrive: 一种通过在线强化学习实现自动驾驶的视觉-语言-动作模型

Haoyu Fu, Diankun Zhang, Zongchuang Zhao, Jianfeng Cui, Hongwei Xie, Bing Wang, Guang Chen, Hangjun Ye, Dingkang Liang, Xiang Bai

机构 * Huazhong University of Science and Technology(华中科技大学) Xiaomi EV(小米电动车)

AI总结 MindDrive通过在线强化学习实现自动驾驶中的视觉-语言-动作模型,结合决策专家和动作专家,提升复杂场景下的驾驶决策与探索效率。

Comments Accepted by ECCV 2026; Project Page: https://xiaomi-mlab.github.io/MindDrive/

详情
AI中文摘要

当前自动驾驶中的视觉-语言-动作(VLA)范式主要依赖于模仿学习(IL),这引入了固有的挑战,如分布偏移和因果混淆。在线强化学习通过试错学习提供了一条解决这些问题的有希望的途径。然而,将在线强化学习应用于自动驾驶中的VLA模型受到连续动作空间中探索效率低的阻碍。为克服这一限制,我们提出了MindDrive,一种由大型语言模型(LLM)和两组不同的LoRA参数组成的VLA框架。一个LLM用作决策专家进行场景推理和驾驶决策,而另一个则作为动作专家,动态地将语言决策映射到可行的轨迹中。通过将轨迹级奖励反馈到推理空间中,MindDrive能够在有限的离散语言驾驶决策集上实现试错学习,而不是直接在连续动作空间中操作。这种方法有效平衡了复杂场景中的最优决策、人类驾驶行为和在线强化学习中的高效探索。使用轻量级的Qwen-0.5B LLM,MindDrive在具有挑战性的Bench2Drive基准上实现了78.04的驾驶得分(DS)和55.09%的成功率(SR)。据我们所知,这是首次展示在线强化学习在自动驾驶VLA模型中的有效性的工作。

英文摘要

Current Vision-Language-Action (VLA) paradigms in autonomous driving primarily rely on Imitation Learning (IL), which introduces inherent challenges such as distribution shift and causal confusion. Online Reinforcement Learning offers a promising pathway to address these issues through trial-and-error learning. However, applying online reinforcement learning to VLA models in autonomous driving is hindered by inefficient exploration in continuous action spaces. To overcome this limitation, we propose MindDrive, a VLA framework comprising a large language model (LLM) with two distinct sets of LoRA parameters. The one LLM serves as a Decision Expert for scenario reasoning and driving decision-making, while the other acts as an Action Expert that dynamically maps linguistic decisions into feasible trajectories. By feeding trajectory-level rewards back into the reasoning space, MindDrive enables trial-and-error learning over a finite set of discrete linguistic driving decisions, instead of operating directly in a continuous action space. This approach effectively balances optimal decision-making in complex scenarios, human-like driving behavior, and efficient exploration in online reinforcement learning. Using the lightweight Qwen-0.5B LLM, MindDrive achieves Driving Score (DS) of 78.04 and Success Rate (SR) of 55.09% on the challenging Bench2Drive benchmark. To the best of our knowledge, this is the first work to demonstrate the effectiveness of online reinforcement learning for the VLA model in autonomous driving.

URL PDF HTML 收藏
2607.14684 2026-07-17 cs.CV 新提交

GlobalForge: Towards Robust AI-Generated Image Detection

GlobalForge:迈向强大的人工智能生成图像检测

Manni Cui, Ruiqi Liu, Dianyuan Zou, Ziheng Qin, Jingrui Xu, ZiAn Wang, Jianglan Wei, Han Zhou, Yu Liu, Yan Wang, Shu Wu

机构 * Huazhong University of Science and Technology(华中科技大学) Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所) Jilin University(吉林大学) Tsinghua University(清华大学)

AI总结 研究针对AI生成图像检测器在现实世界通道传播后性能下降问题,提出GlobalForge框架,通过局部信息瓶颈和全局结构推理模块,基于退化对比结构损失联合训练,提升了检测器在多种基准上的性能。

详情
AI中文摘要

人工智能生成图像(AIGI)检测器在干净基准上有很高准确率,但图像经现实世界通道传播后性能大幅下降。研究发现其脆弱性源于过度拟合生成器在小空间邻域留下的局部伪像,易被常见传播退化破坏。为此提出GlobalForge框架,含局部信息瓶颈(LIB)和全局结构推理(GSR)模块,二者基于退化的对比结构损失联合训练以保持特征稳定。还引入RealDeg - Bench支持细粒度鲁棒性评估。GlobalForge在8个野外基准组上平均BAcc比之前最先进方法提高5.89%,在RealDeg - Bench的单退化和复合退化下均领先于代表性基线。

英文摘要

AI-generated image (AIGI) detectors achieve strong accuracy on clean benchmarks, but their performance drops sharply after images are propagated through real-world channels. We trace this fragility to what these detectors actually learn: they overfit to local artifacts left by generators in small spatial neighborhoods, which are easily destroyed by common propagation degradations such as JPEG compression and blur. Instead, we shift the discriminative cue from fragile local artifacts to more robust global structure. Building on this, we propose GlobalForge, a framework with two complementary modules. The Local Information Bottleneck (LIB) suppresses local components to block shortcut learning, while the Global Structural Reasoning (GSR) module forces every token to gather evidence from distant regions. Both modules are trained jointly under a contrastive structural loss based on degradation that keeps the resulting features stable under degradation. To support fine-grained robustness evaluation, we further introduce RealDeg-Bench, covering 7 common degradation operators and multi-step compound chains. GlobalForge improves average BAcc on 8 in-the-wild benchmark groups by $\mathbf{5.89\%}$ over the previous state-of-the-art, and is clearly ahead of representative baselines on RealDeg-Bench under both single and compound degradations. Code is available at https://anonymous.4open.science/r/GlobalForge-BE0F/.

URL PDF HTML 收藏
2607.14088 2026-07-16 cs.CV 新提交

VideoRAE: Taming Video Foundation Models for Generative Modeling via Representation Autoencoders

VideoRAE:通过表示自动编码器驯服用于生成建模的视频基础模型

Zhihao Xie, Junfeng Wu, Xinting Hu, Junchao Huang, Li Jiang

机构 * The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳)) Huazhong University of Science and Technology(华中科技大学) Shenzhen Loop Area Institute(深圳环宇研究院) University of Science and Technology of China(中国科学技术大学)

AI总结 研究视频生成模型潜在空间问题,提出VideoRAE,利用冻结视频基础编码器特征经1D自注意力投影仪压缩,支持多种潜在空间,通过多码本高维量化等实现强大重建,收敛快,验证了冻结VFM表示的有效性。

Comments Home page: https://zhxie0117.github.io/VideoRAE

详情
AI中文摘要

视频生成模型通常依赖于由3D变分自动编码器(3D-VAE)学习的潜在空间。然而,传统的3D-VAE主要针对像素级重建进行优化,这可能会限制其潜在空间捕获的语义和时空结构。同时,诸如V-JEPA 2和VideoMAEv2等视频基础模型(VFM)显示出强大的视频理解能力,但其冻结表示能否转换为紧凑、具有重建能力且对生成友好的视频潜在空间在很大程度上仍未得到探索。我们通过VideoRAE回答了这个问题,它是一种表示自动编码器,利用冻结视频基础编码器的多尺度分层特征,并通过轻量级1D自注意力投影仪对其进行压缩。VideoRAE通过多码本高维量化支持扩散变压器的连续潜在空间和自回归模型的离散令牌。在解码过程中,表示自动编码器通过冻结的VFM教师改进语义保留并实现无KL正则化的训练。实验表明,VideoRAE在连续和离散模式下均实现了强大的重建。在UCF-101上,它分别使用AR和DiT生成器获得了40和93的最新类到视频gFVD,同时收敛速度比竞争的自动编码器基线快约5倍。在受控的2B规模文本到视频研究中,在可比设置下,用VideoRAE替换LTX-VAE会导致更快的收敛。这些结果验证了冻结的VFM表示作为通用且对生成友好的视频潜在空间。模型和代码将在这个https URL上发布。

英文摘要

Video generative models commonly rely on latent spaces learned by 3D Variational Autoencoders (3D-VAEs). However, conventional 3D-VAEs are mainly optimized for pixel-level reconstruction, which can limit the semantic and spatio-temporal structure captured by their latents. Meanwhile, Video Foundation Models (VFMs) such as V-JEPA 2 and VideoMAEv2 show strong video understanding capabilities, yet whether their frozen representations can be transformed into compact, reconstruction-capable, and generation-friendly video latents remains largely unexplored. We answer this question with VideoRAE, a representation autoencoder that leverages multi-scale hierarchical features from a frozen video foundation encoder and compresses them with a lightweight 1D self-attention projector. VideoRAE supports both continuous latents for Diffusion Transformers and discrete tokens for autoregressive models via multi-codebook high-dimensional quantization. During decoding, a local-and-global representation alignment objective with the frozen VFM teacher improves semantic preservation and enables training without KL regularization. Experiments show that VideoRAE achieves strong reconstruction in both continuous and discrete regimes. On UCF-101, it obtains state-of-the-art class-to-video gFVDs of 40 and 93 with AR and DiT generators, respectively, while converging approximately 5x faster than competing autoencoder baselines. In a controlled 2B-scale text-to-video study, replacing LTX-VAE with VideoRAE leads to faster convergence under comparable settings. These results validate frozen VFM representations as versatile and generation-friendly video latents. The model and code will be released on https://zhxie0117.github.io/VideoRAE.

URL PDF HTML 收藏
2607.13940 2026-07-16 cs.AI 新提交

A Self-Evolving Agent for Longitudinal Personal Health Management

一种用于纵向个人健康管理的自我进化智能体

Haoran Li, Jiebi Deng, Tong Jin, Jinghong Han, Yuxin Wang, Zexin Wang, Qingyi Si, Weikang Gong, Xiahai Zhuang, Jia You, Wei Cheng, Jianfeng Feng, Hongcheng Guo

机构 * School of Data Science, Fudan University(复旦大学数据科学学院) School of Life Sciences, Beijing University of Chinese Medicine(北京中医药大学生命科学学院) Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University(复旦大学脑科学与智能技术研究院) School of Computer Science and Technology, Huazhong University of Science and Technology(华中科技大学计算机科学与技术学院) JD.com, Inc.(京东公司) Key Laboratory of Computational Neuroscience and Brain-Inspired Intelligence, Fudan University, Ministry of Education(复旦大学计算神经科学与类脑智能教育部重点实验室) Department of Neurology, Huashan Hospital, Fudan University(复旦大学附属华山医院神经内科)

AI总结 研究针对多数健康AI系统孤立处理请求的问题,开发开源智能体架构HealthClaw,通过自我进化更新支持,经合成基准和生物医学任务评估,在准确率、隐私性及任务指标增益上表现出色,支持纵向个人健康智能体的自我进化记忆。

Comments 20 pages, 4 figures, 6 supplementary tables. Code: https://github.com/HC-Guo/HealthClaw

详情
AI中文摘要

个人健康管理是在反复接触中展开的,但大多数健康人工智能系统孤立地处理每个请求。我们开发了HealthClaw,这是一种开源智能体架构,它会随着个人日常、偏好、测量和风险的变化更新支持。它将共享安全规则和医学知识与包含个人资料事实、可重复使用程序和情景痕迹的私人纵向记忆分开。每次事件后,归纳法决定应更新个人资料、修订程序、保留情景还是排除。我们用合成的一年期基准和九个200例生物医学任务评估了HealthClaw。在900次纵向支持探测中,答案准确率从当前查询提示的0.2%提高到HealthClaw的45.7%,同时提示侧上下文暴露比全历史提示低71.7%。在100次隐私探测中,HealthClaw产生了更高的隐私感知答案质量和更少的不安全披露。在生物医学任务中,特定任务主要指标的平均绝对增益为27.0个百分点,经过错误发现率校正后,七个增益仍然显著。这些离线基准支持纵向个人健康智能体的受治理、自我进化记忆,尽管临床有效性需要前瞻性评估。HealthClaw可在这个https网址公开获取。

英文摘要

Personal health management unfolds over repeated encounters, yet most health AI systems treat each request in isolation. We developed HealthClaw, an open-source agent architecture that updates support as a person's routines, preferences, measurements and risks change. It separates shared safety rules and medical knowledge from private longitudinal memory containing profile facts, reusable procedures and episodic traces. After each episode, induction determines what should update the profile, revise a procedure, remain episodic or be excluded. We evaluated HealthClaw with a synthetic year-long benchmark and nine 200-case biomedical tasks. Across 900 longitudinal support probes, answer accuracy increased from 0.2% with current-query prompting to 45.7% with HealthClaw, while prompt-side context exposure was 71.7% lower than with full-history prompting. In 100 privacy probes, HealthClaw produced higher privacy-aware answer quality and fewer unsafe disclosures than both baselines. Across the biomedical tasks, the mean absolute gain in the task-specific primary metric was 27.0 percentage points, and seven gains remained significant after false-discovery-rate correction. These offline benchmarks support governed, self-evolving memory for longitudinal personal health agents, although clinical effectiveness requires prospective evaluation. HealthClaw is publicly available at https://github.com/HC-Guo/HealthClaw.

URL PDF HTML 收藏
2607.12753 2026-07-15 cs.CV 新提交

RFMSR: Residual Flow Matching for Image Super-Resolution

RFMSR:用于图像超分辨率的残差流匹配

Shuwei Huang, Tianyao Luo, Jicheng Liu, Daizong Liu, Pan Zhou

机构 * Huazhong University of Science and Technology(华中科技大学) Wuhan University(武汉大学)

AI总结 研究针对图像超分辨率,提出残差流匹配框架RFMSR,将源分布集中于LQ潜变量以减少传输距离并保留结构先验,采用两阶段训练策略,实验证明该方法相比SOTA实现了相当甚至更优的感知质量。

详情
AI中文摘要

图像超分辨率(ISR)借助扩散模型和流匹配取得了显著进展。基于文本到图像(T2I)的主流方法利用大规模基础模型作为生成先验,虽实现了令人印象深刻的感知质量,但模型规模庞大且训练成本高昂。近期基于流匹配的纯视觉方法有显著进步,然而它们采用从纯高斯先验到数据分布的标准流公式,丢弃了低质量(LQ)输入中已有的丰富结构信息。此外,现有的单步加速技术往往丧失了模型的多步推理能力。本文提出用于图像超分辨率的残差流匹配(RFMSR),这是一个纯视觉框架,将源分布集中在LQ潜变量上,减少传输距离并在整个流轨迹中保留结构先验。还引入了两阶段训练策略:第一阶段通过条件流匹配预训练速度场,第二阶段对单步预测应用端到端监督,同时保留所有时间步的速度损失,在不牺牲多步细化的情况下实现高质量单步生成。大量实验表明,RFMSR与现有最先进(SOTA)方法相比,实现了相当甚至更优的感知质量。

英文摘要

Image super-resolution (ISR) has witnessed remarkable progress with diffusion models and flow matching. The dominant text-to-image (T2I) based approaches leverage large-scale foundation models as generative priors, achieving impressive perceptual quality but at the cost of massive model sizes and prohibitive training expenses. Recent flow-matching-based vision-only approaches have made significant strides; however, they adopt standard flow formulations that transport from a pure Gaussian prior to the data distribution, discarding the rich structural information already present in the low-quality (LQ) input. Furthermore, existing single-step acceleration techniques often forfeit the model's multi-step inference capability. In this paper, we propose Residual Flow Matching for Image Super-Resolution (RFMSR), a vision-only framework that centers the source distribution at the LQ latent, reducing transport distance and preserving structural priors throughout the flow trajectory. We further introduce a two-phase training strategy: Phase I pretrains the velocity field via conditional flow matching, while Phase II applies end-to-end supervision to the single-step prediction while retaining the velocity loss across all timesteps, achieving high-quality single-step generation without sacrificing multi-step refinement. Extensive experiments demonstrate that RFMSR achieves comparable or even superior perceptual quality compared to state-of-the-art (SOTA) methods. The source code is available at https://github.com/Faze-Hsw/RFMSR.

URL PDF HTML 收藏
2606.30347 2026-07-15 cs.CV cs.AI 版本更新

FFAvatar: Feed-Forward 4D Head Avatar Reconstruction from Sparse Portrait Images

FFAvatar: 从稀疏肖像图像进行前馈4D头部化身重建

Jianjiang Yao, Ke Xian, Renxiang Dai, Robert Caiming Qiu

机构 * School of Electronic Information and Communications, Huazhong University of Science and Technology, Wuhan 430074, China(华中科技大学电子信息与通信学院,武汉430074,中国)

AI总结 提出基于Transformer的3D高斯框架FFAvatar,通过交替注意力机制和稀疏到密集学习范式,从一张或多张参考图像高效构建高质量、可动画的4D头部化身,支持增量重建和动态个性化。

Comments Accepted to ECCV 2026. Project page: https://jj-yao.github.io/ffavatar/

详情
AI中文摘要

我们提出FFAvatar,一种基于Transformer的3D高斯框架,用于从一张或多张参考肖像图像快速构建高质量且可动画的4D头部化身。与需要固定数量输入视图的现有前馈方法不同,FFAvatar支持增量重建,随着更多参考图像的可用逐步细化化身表示。我们方法的核心是交替注意力机制,它将身份外观与表情和视角变化解耦,从而重建一个在姿势和面部表情间保持一致的规范3D外观。为了平衡视觉保真度和计算效率,我们引入了一种稀疏到密集的学习范式。首先使用锚定在FLAME顶点级别的稀疏基元学习粗外观特征,随后在UV域中对其进行密集化以捕获细粒度的几何和纹理细节。我们进一步提出了一个即插即用的运动细化模块,通过对参数变形之外的残差运动进行建模,实现特定主体的动态个性化。大量实验表明,FFAvatar高效地生成高保真且可控的4D头部化身,在不同表情和视角下实现了卓越的灵活性、驱动效率和身份一致的渲染。

英文摘要

We present FFAvatar, a Transformer-based 3D Gaussian framework for fast construction of high-quality and animatable 4D head avatars from one or more reference portrait images. Unlike existing feed-forward approaches that require a fixed number of input views, FFAvatar supports incremental reconstruction, progressively refining the avatar representation as additional reference images become available. At the core of our method is an alternating attention mechanism that disentangles identity appearance from expression and viewpoint variations, enabling the reconstruction of a canonical 3D appearance that remains consistent across poses and facial expressions. To balance visual fidelity and computational efficiency, we introduce a sparse-to-dense learning paradigm. Coarse appearance features are first learned using sparse primitives anchored to the FLAME vertex level and are subsequently densified in the UV domain to capture fine-grained geometric and texture details. We further propose a plug-and-play motion refinement module that enables subject-specific dynamic personalization by modeling residual motion beyond parametric deformation. Extensive experiments demonstrate that FFAvatar efficiently produces high-fidelity and controllable 4D head avatars, achieving superior flexibility, driving efficiency, and identity-consistent rendering across diverse expressions and viewpoints.

URL PDF HTML 收藏
2602.12783 2026-07-15 cs.IR cs.AI 版本更新

SQuTR: A Robustness Benchmark for Spoken Query to Text Retrieval under Acoustic Noise

SQuTR:一种在语音噪声下 spoken query 到文本检索的鲁棒性基准

Yuejie Li, Ke Yang, Yueying Hua, Berlin Chen, Jianhao Nie, Yueping He, Caixin Kang

机构 * Huazhong University of Science and Technology(华中科技大学) The University of Hong Kong(香港大学) Soochow University(苏州大学) University of Science and Technology of China(中国科学技术大学) Wuhan University(武汉大学) Tsinghua University(清华大学) The University of Tokyo(东京大学)

AI总结 SQuTR通过大规模数据集和统一评估协议,评估语音检索系统在复杂噪声环境下的鲁棒性,揭示了极端噪声下检索性能显著下降的问题。

Comments Accepted by SIGIR 2026

Journal ref Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '26), July 20--24, 2026, Melbourne, VIC, Australia

详情
AI中文摘要

语音查询检索是现代信息检索中的重要交互模式。然而,现有评估数据集通常局限于简单查询和受限噪声条件,无法有效评估语音查询检索系统在复杂声学扰动下的鲁棒性。为解决这一限制,我们提出了SQuTR,一个用于语音查询检索的鲁棒性基准,包含大规模数据集和统一评估协议。SQuTR聚合了来自六个常用英文和中文文本检索数据集的37,317个唯一查询,涵盖多个领域和多样化的查询类型。我们使用200名真实说话人的语音轮廓合成语音,并在受控的SNR水平下混合17种真实世界环境噪声,使从安静到高度噪声条件的可重复鲁棒性评估成为可能。在统一协议下,我们对代表性 cascaded 和 end-to-end 检索系统进行了大规模评估。实验结果表明,随着噪声增加,检索性能下降,不同系统下降幅度差异显著。即使大规模检索模型在极端噪声下也难以应对,表明鲁棒性仍然是关键瓶颈。总体而言,SQuTR提供了一个可重复的测试平台用于基准测试和诊断分析,并促进了未来关于语音查询到文本检索鲁棒性的研究。

英文摘要

Spoken query retrieval is an important interaction mode in modern information retrieval. However, existing evaluation datasets are often limited to simple queries under constrained noise conditions, making them inadequate for assessing the robustness of spoken query retrieval systems under complex acoustic perturbations. To address this limitation, we present SQuTR, a robustness benchmark for spoken query retrieval that includes a large-scale dataset and a unified evaluation protocol. SQuTR aggregates 37,317 unique queries from six commonly used English and Chinese text retrieval datasets, spanning multiple domains and diverse query types. We synthesize speech using voice profiles from 200 real speakers and mix 17 categories of real-world environmental noise under controlled SNR levels, enabling reproducible robustness evaluation from quiet to highly noisy conditions. Under the unified protocol, we conduct large-scale evaluations on representative cascaded and end-to-end retrieval systems. Experimental results show that retrieval performance decreases as noise increases, with substantially different drops across systems. Even large-scale retrieval models struggle under extreme noise, indicating that robustness remains a critical bottleneck. Overall, SQuTR provides a reproducible testbed for benchmarking and diagnostic analysis, and facilitates future research on robustness in spoken query to text retrieval.

URL PDF HTML 收藏
2607.11562 2026-07-14 cs.CV 新提交

MonkeyOCRv2: A Visual-Text Foundation Model for Document AI

MonkeyOCRv2:用于文档人工智能的视觉-文本基础模型

Yuliang Liu, Zhang Li, Ziyang Zhang, Shuo Zhang, Qiang Liu, Jiajun Song, Zidun Guo, Xinhan Wang, Handong Zheng, Yang Liu, Dongliang Luo, Zhiyin Ma, Jiarui Zhang, Xiang Bai

机构 * Huazhong University of Science and Technology(华中科技大学) Kingsoft Office(金山办公软件)

AI总结 研究针对主流视觉编码器难以应用于文档图像的问题,提出MonkeyOCRv2模型。通过构建大型文档图像预训练语料库及采用联合预训练策略,在多个文档分析任务中提升性能,并验证其作为视觉编码器在文档解析和理解任务中的有效性。

详情
AI中文摘要

主流视觉编码器在自然图像上预训练,因文档图像中密集文本和精细字符笔画需字符级视觉感知,无法直接应用于文档图像。本文提出用于文档人工智能的视觉-文本预训练模型MonkeyOCRv2。首先构建了包含11300万张17种语言图像的MonkeyDoc v2文档图像预训练语料库。其次提出联合学习图像到文本生成和像素级文档重建的预训练策略。在五个文档分析任务上进行大量实验,结果表明MonkeyOCRv2在所有任务中均提升了性能,还验证了其作为多模态大语言模型视觉编码器在文档解析和理解任务中的有效性。

英文摘要

Mainstream visual encoders are pretrained on natural images and cannot be effectively applied to document images without document-oriented adaptation, as dense text and fine-grained character strokes demand character-level visual perception. We present MonkeyOCRv2, a visual-text pretrained model for document AI. First, we construct MonkeyDoc v2, to our knowledge the largest document-image pretraining corpus, comprising 113 million images spanning 17 languages. Second, we propose a pretraining strategy that jointly learns image-to-text generation and pixel-level document reconstruction: the former aligns visual representations with textual content, while the latter preserves character strokes and layout details. Extensive experiments are conducted on five representative document analysis tasks, including text recognition, formula recognition, text detection, document tampering detection, and overlapping text segmentation. Replacing the original encoders with MonkeyOCRv2 consistently improves performance across all five tasks. Finally, we validate its effectiveness as the vision encoder of multimodal large language models on the more challenging tasks of document parsing and document understanding. Kept frozen and paired with a lightweight language model, it yields a 0.7B document parsing model that sets a new open-source state-of-the-art on MDPBench, a recent benchmark spanning digital-born and photographed documents across 17 languages, surpassing the previous best 3B dots.mocr by 2.8% absolute with a vision encoder roughly 11$\times$ smaller. The frozen encoder also powers a document understanding model that outperforms counterparts built on CLIP, DINO, and SAM across eight benchmarks under identical training settings. These results suggest that document-oriented visual pretraining can serve as a foundation for document intelligence in its own right.

URL PDF HTML 收藏
2607.11347 2026-07-14 cs.AI 新提交

From Neural Network Decisions to Training Cases: An Exact Account via Case-Based Decision Theory

从神经网络决策到训练案例:基于案例决策理论的精确阐述

Manli Yan, Yuebin Lin, Yaowen Yu, Yong Zhao

机构 * Huazhong University of Science and Technology(华中科技大学)

AI总结 研究神经网络决策,基于案例决策理论,通过OLS行动读出的精确案例分解,能追溯分数到训练案例、衡量行动一致性等,在多任务中恢复案例级偏好结构,实现高一致性且审计简单,探测器保真度由分数重建衡量。

Comments Preprint. 15 pages, 9 figures, 4 tables. Includes appendix

详情
AI中文摘要

神经网络在医疗诊断、信贷审批和能源投标等高风险领域越来越多地指导决策。这些场景中的审计需要案例级证据,即哪些训练案例支持某个行动以及它们带来了什么结果。基于案例的决策理论(CBDT)通过汇总记忆案例的结果支持来形式化这种推理。我们表明,在固定神经表示上拟合的OLS行动读出允许精确的基于案例的分解。每个行动分数是训练案例回报的加权和,系数由经验Gram几何确定。我们确定了CBDT相似性语义的充分条件;在其之外,系数通常应被视为带符号的Gram几何影响。这种分解产生了审计信号,可将分数追溯到训练案例、衡量行动一致性并识别薄弱支持。在合成CBDT、PJM、成人收入和违约信贷任务中,该方法恢复了案例级偏好结构,并在比较的归因基线中实现了最高的平均前30一致性,同时在支持重建方面保持竞争力。审计只需要拟合一个OLS顶层探测器,而无需重新训练表示或访问原始优化轨迹;探测器保真度通过分数重建来衡量。

英文摘要

Neural networks increasingly guide decisions in high-stakes domains such as medical diagnosis, credit approval, and energy bidding. Audit in these settings requires case-level evidence: which training cases support an action and what outcomes they carried. Case-based decision theory (CBDT) formalizes this reasoning by aggregating outcome support from remembered cases. We show that an OLS action readout fitted on a fixed neural representation admits an exact case-based decomposition. Each action score is a weighted sum of training-case returns, with coefficients determined by empirical Gram geometry. We identify a sufficient regime for CBDT similarity semantics; outside it, the coefficients should generally be treated as signed Gram-geometric influence. The decomposition yields audit signals that trace scores to training cases, measure action coherence, and identify weak support. Across synthetic CBDT, PJM, Adult Income, and Default Credit tasks, the method recovers case-level preference structure and achieves the highest mean Top-30 consistency among compared attribution baselines, while remaining competitive on support reconstruction. The audit requires only fitting an OLS top-layer probe, without retraining the representation or accessing the original optimization trajectory; probe fidelity is measured by score reconstruction.

URL PDF HTML 收藏
2607.11338 2026-07-14 cs.AI 新提交

AutoVSR: Automatic Visual-to-Symbolic Reasoning for Symbolic Expression Generation from Circuit Schematic

AutoVSR:用于从电路原理图生成符号表达式的自动视觉到符号推理

Zhe Xiao, Longfei Li, Xu He, Haoying Wu, Zixing Zhang, Mingyu Liu

机构 * Hunan University, Changsha, China(湖南大学) Wuhan University of Technology, Wuhan, China(武汉理工大学) Huazhong University of Science and Technology, Wuhan, China(华中科技大学)

AI总结 研究旨在解决从电路原理图生成符号表达式的难题,提出AutoVSR框架,利用视觉语言模型,通过重建电路图为可执行中间表示并借助符号求解器推理,引入两项创新提升准确率,在任务中表现优于其他方法,还降低了推理成本和提高了计算效率。

详情
AI中文摘要

符号表达式能有效表征和预测电路行为,但直接从电路原理图推导具有挑战性,该过程需准确的视觉到符号的电路结构构建及正确的多步符号推导。本文提出AutoVSR,一个使用视觉语言模型进行视觉到符号生成电路表达式的自动化框架。通过将电路图重建为可执行中间表示并利用符号求解器推理,显著提高了符号表达式生成的准确性。它引入了两项关键创新:基于组件规则检索和验证反馈的IR构建方法,以及作为配备符号工具库的规划代理实现的符号求解器以进行可靠的多步推导。与端到端VLM方法和专门方法相比,AutoVSR在主要符号表达式生成任务上分别实现了30.01 - 59.45%和41.96 - 51.84%的准确率提升,且在推理成本和计算效率上超越了闭源的最先进VLM。代码可在指定网址获取。

英文摘要

Symbolic expressions can effectively characterize and predict circuit behavior, but deriving them directly from circuit schematics is challenging. This process requires accurate visual-to-symbolic construction of circuit structure from images and correct multi-step symbolic derivation, both of which impose strict correctness requirements. This work proposes AutoVSR, an automated framework for visual-to-symbolic generation of circuit expressions using Vision Language Models (VLMs). By reconstructing circuit diagrams into an executable intermediate representation (Executable IR) and leveraging a symbolic solver for reasoning, AutoVSR significantly improves the accuracy of symbolic expression generation. AutoVSR introduces two key innovations: an IR construction method guided by component rule retrieval and verification-based feedback, and a symbolic solver implemented as a planning agent equipped with a symbolic tool library for reliable multi-step derivation. Compared with end-to-end VLM approaches and specialized methods on the main symbolic expression generation task, AutoVSR achieves accuracy improvements of 30.01--59.45% and 41.96--51.84%, respectively. Moreover, AutoVSR surpasses closed-source state-of-the-art VLMs in inference cost and computational efficiency. Code is available at https://github.com/LongfeiLi1/AutoVSR.

URL PDF HTML 收藏
2607.11233 2026-07-14 cs.CV 新提交

Structure-Detail Decoupled Autoregressive Generation for Fast and High-Fidelity Virtual Try-On

用于快速和高保真虚拟试穿的结构-细节解耦自回归生成

Lu Yang, Xiaonan Hu, Yanan Li, Daqi Liu, Xiang Bai, Hao Lu

机构 * Huazhong University of Science and Technology(华中科技大学) Wuhan Institute of Technology(武汉工程大学) Xiaomi EV(小米汽车)

AI总结 研究虚拟试穿问题,提出STAR-VTON框架,通过解耦潜在空间结构合成与像素空间细节恢复,结合匹配信息细化器,实现高效高保真虚拟试穿,其中VAR-VTON效率高,像素空间细化器能有效恢复细节。

详情
AI中文摘要

虚拟试穿(VTON)是一个双条件图像生成问题,需要准确保留人物并忠实进行服装变形和细节合成。基于扩散的VTON方法在压缩潜在空间中联合建模这些因素,但因固有潜在压缩存在高频细节损失。近期视觉自回归(VAR)模型为高质量生成提供了有前景的替代方案,但因缺乏有效双条件机制未用于VTON。为此引入VAR-VTON,虽有效但在保留细粒度服装细节上仍有困难。进而提出STAR-VTON,通过解耦潜在空间结构合成与像素空间细节恢复,借助匹配信息细化器建立对应关系来恢复细节。实验表明STAR-VTON实现了效率与保真度的良好权衡,VAR-VTON比基于扩散的方法快至少4倍且不降低质量,像素空间细化器能有效恢复细节并可作为即插即用模块惠及现有VTON方法。

英文摘要

Virtual try-on (VTON) is a bi-conditional image generation problem that requires not only accurate person preservation but also faithful garment deformation and detail synthesis. Diffusion-based VTON methods can jointly model these factors in a compressed latent space, but suffer from high-frequency detail loss due to inherent latent compression, even with costly multi-step denoising. Recent visual autoregressive (VAR) models offer a promising alternative for high-quality generation with faster inference, yet remain unexplored for VTON due to the lack of effective bi-conditioning mechanisms. To bridge this gap, we first introduce VAR-VTON, a VAR-based VTON model that incorporates garment conditioning and structural guidance for efficient latent-space VTON. Despite its efficacy, latent-space generation still struggles to preserve fine-grained garment details. We argue that different VTON sub-tasks should be addressed in different representation spaces: structural synthesis such as garment warping and person layout is suited to the latent space, whereas fine-grained detail recovery should be tackled in the pixel space. Motivated by this insight, we further propose STAR-VTON, a Two-Stage AutoRegressive framework that builds upon VAR-VTON by decoupling latent-space structural synthesis from pixel-space detail recovery. Our idea is to resort to a matching-informed refiner to establish dense correspondences between the stage-one generation and the source garment to directly map fine-grained pixel-space details. Extensive experiments show that STAR-VTON achieves an impressive efficiency-fidelity trade-off: VAR-VTON runs at least $4\times$ faster than diffusion-based counterparts without degrading quality, and the pixel-space refiner effectively restores fine details and acts as a plug-and-play module that can benefit existing VTON approaches.

URL PDF HTML 收藏
2607.11166 2026-07-14 cs.CL 新提交

Query-Focused Event Summarization: A Dataset and Benchmark

查询聚焦事件摘要:一个数据集和基准

Chenyu Hu, Bang Wang

机构 * School of Electronic Information and Communications, Huazhong University of Science and Technology(华中科技大学电子信息与通信学院)

AI总结 针对现有查询聚焦摘要(QFS)数据集缺乏面向事件的摘要及方法在大规模语料库表现不佳的问题,提出查询聚焦事件摘要(QFES)任务,构建QFESum数据集,引入含RAT和SHC的两阶段框架,实验证明该框架有效。

Comments 22 pages, 9 figures, and 13 tables. Dataset and code are available at https://github.com/sarcasm-hcy02/QFES-QFESum

详情
AI中文摘要

主题语料库是语义连贯文档的集合,共同描述共享主题事件的不同方面,通常包含成百上千个文档。虽然用户对主题事件的兴趣往往涉及多个维度,但查询聚焦摘要(QFS)旨在生成适合用户查询的摘要。然而,现有QFS数据集缺乏面向事件的摘要,且大多数QFS方法在大规模语料库上表现不佳。为应对这些挑战,我们提出查询聚焦事件摘要(QFES)任务并构建QFESum数据集,包含8个主题事件、16684个文档和104个查询。此外,我们引入了一个两阶段的QFES框架,由带自适应阈值的查询聚焦检索(RAT)和基于层次聚类的查询聚焦摘要(SHC)组成。在QFESum上的实验结果表明,RAT和SHC始终优于基线,证明了它们对QFES的有效性。数据集和代码可通过此https URL公开获取。

英文摘要

A thematic corpus is a collection of semantically coherent documents that collectively describe different aspects of a shared thematic event. Such a corpus typically contains hundreds or even thousands of documents. While users' interests in a thematic event often span multiple dimensions, Query-Focused Summarization (QFS) aims to generate summaries tailored to users' queries. However, existing QFS datasets lack event-oriented summarization, and most QFS methods struggle with large-scale corpora. To address these challenges, we propose the Query-Focused Event Summarization (QFES) task and construct the QFESum dataset, which contains 8 thematic events, 16,684 documents, and 104 queries. Furthermore, we introduce a two-stage QFES framework consisting of Query-Focused Retrieval with Adaptive Thresholding (RAT) and Query-Focused Summarization based on Hierarchical Clustering (SHC). Experimental results on QFESum show that RAT and SHC consistently outperform the baselines, demonstrating their effectiveness for QFES. The dataset and code are publicly available at https://github.com/sarcasm-hcy02/QFES-QFESum.

URL PDF HTML 收藏
2607.10625 2026-07-14 cs.RO 新提交

Dual-Process Atomic Skill Learning: Decoupling Semantic Reasoning and Real-Time Control

双过程原子技能学习:解耦语义推理与实时控制

Jun Chen, Erdent Bao, Wenlong Dong, Jierui Liu, Qi Cai, Hao Wan, Shaopeng Li, Weijun Qin, Jing Liang, Huiping Zhuang

机构 * University of Electronic Science and Technology of China(电子科技大学) Huazhong University of Science and Technology(华中科技大学) Southern University of Science and Technology(南方科技大学) South China University of Technology(华南理工大学) EbTech Co. Ltd.(依比特科技有限公司)

AI总结 研究针对语言条件模仿学习推广到多步组合任务的挑战,提出双过程原子技能学习框架DASL,通过解耦语义推理与实时控制,含低频和高频策略,有效减轻技能码本干扰问题,性能显著优于现有基线。

Comments 28 pages,20 figures,21 tables

详情
AI中文摘要

语言条件模仿学习对使机器人按自然语言指令执行复杂任务至关重要,但推广到多步组合任务仍是重大挑战。现有分层方法因联合训练中高层技能推理与低层动作生成紧密耦合,常面临训练不稳定和码本崩溃问题。受认知双过程理论启发,提出双过程原子技能学习(DASL),一种新颖的异步分层模仿学习框架,将缓慢的语义推理与快速实时运动控制解耦。DASL包括通过矢量量化预测可解释离散技能的低频策略,以及利用潜在扩散模型和决策变换器根据这些潜在技能生成精确动作的高频策略。通过异步协调这些模块并利用扩散构建潜在空间,减轻了联合训练范式中常见的技能码本干扰问题。跨模拟基准和实验评估表明,DASL显著优于现有基线,在技能获取和对未见指令的组合泛化方面表现出色。

英文摘要

Language-conditioned Imitation Learning (IL) is essential for enabling robots to perform complex tasks following natural language instructions. However, generalizing to multi-step compositional tasks remains a significant challenge. While hierarchical approaches attempt to address this by decomposing tasks into atomic skills, existing methods often suffer from training instability and codebook collapse due to the tight coupling between high-level skill reasoning and low-level action generation in joint training paradigms. Inspired by the Dual-Process Theory of cognition, we propose Dual-Process Atomic Skill Learning (DASL), a novel asynchronous hierarchical imitation learning framework that decouples slow semantic reasoning from fast, real-time motion control. DASL comprises a Slow-Frequency Policy that predicts interpretable, discrete skills via Vector Quantization, and a High-Frequency Policy that leverages a latent diffusion model and a Decision Transformer to generate precise actions conditioned on these latent skills. By asynchronously coordinating these modules and utilizing diffusion to structure the latent space, our framework mitigates the skill codebook interference problem common in joint training paradigms. Evaluations across simulation benchmarks and experiment demonstrate that DASL significantly outperforms state-of-the-art baselines, excelling in skill acquisition and compositional generalization to unseen instructions. GitHub page: https://github.com/Hatakekaka/DASL

URL PDF HTML 收藏
2607.10395 2026-07-14 cs.CV 新提交

Self-supervised Automatic Matting

自监督自动抠图

Xiaonan Hu, Zhiyuan Lu, Jingdong Zhao, Hao Lu

机构 * Huazhong University of Science and Technology(华中科技大学) Beijing Normal University(北京师范大学) Waseda University(早稻田大学)

AI总结 研究能否仅用RGB图像训练自动抠图模型。提出自监督框架SSMatte,将问题分解为语义锚定和细节抠图,通过特定损失生成提示并优化网络。该方法性能优,推动自动抠图进入完全无标注范式。

详情
AI中文摘要

高质量的alpha遮罩标注成本高昂,这为深度图像抠图造成了根本性的数据瓶颈。尽管先前工作试图使用诸如三通道图或掩码等更粗糙的标签来降低标注成本,但它们仍依赖昂贵的逐像素监督,限制了可扩展性和泛化能力。在本文中,我们进一步拓展边界并提出问题:能否仅使用RGB图像训练自动抠图模型,完全无需人工标注?我们通过展示SSMatte来回答这一问题,它是一个自监督框架,首次实现了与全监督自动抠图相当的性能。我们的关键见解是将问题分解为语义锚定和细节抠图。SSMatte首先通过基于广义瑞利商的新颖且训练高效的语义锚定损失传播类令牌种子,从冻结的自监督ViT特征生成语义抠图提示。然后该提示锚定一个细节抠图网络,通过基于定点的损失进行优化以强制alpha-RGB一致性。大量实验表明,SSMatte优于先前的弱监督方法,在人像基准测试中与全监督模型的性能相匹配,并在增加数据时展现出良好的可扩展性和泛化行为。我们的工作将自动抠图推向了全新的、完全无需标注的范式。代码将公开。

英文摘要

High-quality alpha mattes are notoriously expensive to annotate, creating a fundamental data bottleneck for deep image matting. While prior work attempts to reduce annotation cost using coarser labels like trimaps or masks, they remain reliant on costly per-pixel supervision, limiting scalability and generalization. In this work, we push the boundary further and ask: can we train an automatic matting model using only RGB images, with no manual annotation at all? We answer this by presenting SSMatte, a self-supervised framework that for the first time achieves performance on par with fully-supervised automatic matting. Our key insight is to decompose the problem into semantic anchoring and detail matting. SSMatte first generates a semantic matting prompt from frozen self-supervised ViT features by propagating class-token seeds via a novel, training-efficient semantic anchoring loss based on a generalized Rayleigh quotient. This prompt then anchors a detail matting network, which is optimized via a fixed-point-based loss that enforces alpha-RGB consistency. Extensive experiments show SSMatte outperforms prior weakly-supervised methods, matches the performance of fully-supervised models on portrait benchmarks, and demonstrates favorable scaling and generalization behaviors with additional data. Our work pushes automatic matting to an fresh, fully annotation-free paradigm. Code will be available.

URL PDF HTML 收藏
2607.10329 2026-07-14 cs.CV 新提交

Imperceptible and Reversible Adversarial Examples against Vision-Language Models for Privacy Protection

用于隐私保护的针对视觉语言模型的不可察觉且可逆的对抗性示例

Qi Lu, Ziqi Zhou, Yufei Song, Zijing Li, Lulu Xue, Minghui Li, Shengshan Hu, Leo Yu Zhang

机构 * School of Cyber Science and Engineering, Huazhong University of Science and Technology(华中科技大学网络空间安全学院) College of Computer Science, Chongqing University(重庆大学计算机科学学院) School of Software and engineering, Huazhong University of Science and Technology(华中科技大学软件工程学院) School of Information and Communication Technology, Griffith University(格里菲斯大学信息与通信技术学院)

AI总结 研究针对视觉语言模型基于文本的隐私攻击问题,提出CloakDiff框架,结合扩散对抗编辑与可逆网络生成不可察觉的对抗性示例,设计EDM启发式采样提高保真度,实验证明该框架能实现多模态隐私保护且具高视觉质量和可逆性。

Comments Accepted by ACM MM 2026

详情
AI中文摘要

视觉语言模型(VLMs)提供强大的多模态能力,但也使用户面临基于文本的隐私攻击,对手通过抓取在线照片并查询VLMs来提取敏感属性。现有的可逆对抗性示例(RAE)方法在纯视觉任务中保护图像,但在多模态设置中失败,当前针对VLMs的对抗性示例依赖于严重降低视觉质量的高频噪声。我们提出了CloakDiff,这是第一个针对VLMs中基于文本的查询攻击进行可逆、高保真隐私保护的框架。CloakDiff通过将基于扩散的对抗性编辑与用于无损恢复的嵌入原始图像的可逆网络相结合来生成不可察觉的对抗性示例。它扰动像素空间嵌入并操纵潜在的交叉注意力图,以确保在保留全局视觉结构的同时具有强大的跨模型和跨提示可转移性。为了进一步提高保真度,我们设计了EDM启发式采样,这是一种用于对抗性指导的有原则的扩散调度。在多个数据集和VLMs上的实验表明,CloakDiff提供了具有高视觉质量和可逆性的多模态隐私保护。

英文摘要

Vision Language Models (VLMs) offer powerful multimodal ability but also expose users to text-based privacy attacks where adversaries crawl online photos and query VLMs to extract sensitive attributes. Existing reversible adversarial example (RAE) methods protect images in purely visual tasks but fail in multimodal settings, and current adversarial examples on VLMs rely on high frequency noise that severely degrades visual quality. We propose CloakDiff, the first framework for reversible, high fidelity privacy protection against text-based query attacks in VLMs. CloakDiff produces imperceptible adversarial examples by combining diffusion based adversarial editing with an invertible network that embeds the original image for lossless recovery. It perturbs both pixel space embeddings and manipulates latent cross attention maps to ensure strong cross-model and cross-prompt transferability while preserving global visual structure. To further enhance fidelity, we design EDM Heuristic Sampling, a principled diffusion schedule for adversarial guidance. Experiments on multiple datasets and VLMs demonstrate that CloakDiff delivers multimodal privacy preservation with high visual quality and reversibility.

URL PDF HTML 收藏
2607.04597 2026-07-14 cs.LG stat.ML 版本更新

Minimum Block Width for Universal Approximation by Residual Neural Networks with Inner Width One

具有内部宽度为一的残差神经网络通用逼近的最小块宽度

Qi Zhou, Xuan Zhou, Xiao-Song Yang

机构 * School of Mathematics and Statistics, Huazhong University of Science and Technology(华中科技大学数学与统计学院) Hubei Key Laboratory of Engineering Modeling and Scientific Computing, Huazhong University of Science and Technology(湖北省工程建模与科学计算重点实验室)

AI总结 研究残差神经网络通用逼近性质,针对特定激活函数建立块宽度上下界,证明内部宽度为1时在紧致域上实现\(L^p\)逼近的最小块宽度,还给出特定块宽度网络能实现一致逼近及宽度小于某值不能逼近所有目标函数的结论。

详情
AI中文摘要

在本文中,我们研究了残差神经网络的通用逼近性质,并获得了一些新结果。对于输入和输出维度\(d_x\)和\(d_y\),以及LeakyReLU、ReLU、类ReLU激活函数,建立了块宽度的上下界。为了在任何紧致域上实现\(L^p\)逼近\((1\leq p <+\infty)\),我们表明当内部宽度为1时,精确的最小块宽度是\(\max\{d_x,d_y\}\)。此外,我们表明在每个残差分支的内部宽度为1的约束下,具有块宽度\(\min\{d_x + d_y, \max\{2d_x + 1, d_y\}\}\)的残差神经网络可以在任何紧致域上实现一致逼近。此外,对于任何激活函数族,我们证明无论内部宽度如何,块宽度小于\(\max\{d_x, d_y\}\)的残差神经网络在\(L^p\)意义和一致意义上都不能逼近所有目标函数。

英文摘要

In this paper, we study the universal approximation property of residual neural networks, and obtain some new results. For input and output dimensions $d_x$ and $d_y$, and LeakyReLU, ReLU, ReLU-like activation functions, the upper and lower bounds of the minimum block width are established. To achieve $L^p$ approximation $(1\leq p <+\infty)$ on any compact domain, we show that the exact minimum block width is $\max\{d_x,d_y\}$ when each residual branch has inner width 1. Furthermore, we show that residual neural networks with block width $\min\{d_x+d_y, \max\{2d_x+1,d_y\}\}$ can achieve uniform approximation on any compact domain under the constraint that each residual branch has inner width 1. Besides, for any activation function family, we prove that there exist functions that cannot be approximated by residual neural networks with block width less than $\max\{d_x, d_y\}$, both in the $L^p$ sense and the uniform sense, regardless of inner width.

URL PDF HTML 收藏
2605.10332 2026-07-14 cs.AI 版本更新

EmbodiSkill: Skill-Aware Reflection for Self-Evolving Embodied Agents

EmbodiSkill:基于技能的反思以实现自我进化的具身智能体

Ruofei Ju, Xinrui Wang, Xin Ding, Yifan Yang, Hao Wu, Shiqi Jiang, Qianxi Zhang, Hao Wen, Xiangyu Li, Weijun Wang, Kun Li, Yunxin Liu, Haipeng Dai, Wei Wang, Ting Cao

机构 * Nanjing University(南京大学) Huazhong University of Science and Technology(华中科技大学) University of Science and Technology of China(中国科学技术大学) Microsoft Research(微软研究院) Institute for AI Industry Research (AIR) Tsinghua University(清华大学人工智能产业研究院)

AI总结 本文提出EmbodiSkill,一种无需训练的具身技能自我进化框架,通过技能感知反思和针对性修订提升任务成功率。

详情
AI中文摘要

具身智能体可通过指导对象搜索、动作执行和状态变化的技能在多样环境中获益。由于具身环境在布局、对象状态和其他执行因素上各不相同,这些技能必须从任务执行过程中生成的轨迹中自我进化。然而,现有技能自我进化方法主要在数字环境中开发,通常将轨迹转换为粗略的技能更新。直接将此范式应用于具身设置存在问题,因为任务失败不仅可能反映不正确的技能内容,还可能表明智能体未能遵循有效指导的执行失误。我们提出EmbodiSkill,一种通过技能感知反思和针对性修订实现具身技能自我进化的无训练框架。EmbodiSkill通过当前技能解读轨迹,使用技能变化证据更新技能体,并使用执行失误证据保存和强调有效指导。在ALFWorld和EmbodiedBench上的实验表明,EmbodiSkill在具身任务成功率方面表现一致。在ALFWorld中,EmbodiSkill使冻结的Qwen3.5-27B执行器达到93.28%的任务成功率,优于作为直接智能体使用的GPT-5.2,后者无技能。这些结果表明,技能感知的自我进化有助于具身智能体从自身轨迹中积累可重用的程序性知识。

英文摘要

Embodied agents can benefit from skills that guide object search, action execution, and state changes across diverse environments. Since embodied environments vary across layouts, object states, and other execution factors, these skills must self-evolve from trajectories generated during task execution. However, existing skill self-evolution methods are mainly developed in digital environments and often convert trajectories into coarse skill updates. Directly applying this paradigm to embodied settings is problematic, because a failed task execution may reflect not only incorrect skill content, but also an execution lapse in which the agent fails to follow valid guidance. We propose EmbodiSkill, a training-free framework for embodied skill self-evolution through skill-aware reflection and targeted revision. EmbodiSkill interprets each trajectory with respect to the current skill, uses skill-changing evidence to update the skill body, and uses execution-lapse evidence to preserve and emphasize valid guidance. Experiments on ALFWorld and EmbodiedBench show that EmbodiSkill consistently improves embodied task success. On ALFWorld, EmbodiSkill enables a frozen Qwen3.5-27B executor to reach 93.28% task success, outperforming GPT-5.2 used as a direct agent without skills by 31.58%. These results show that skill-aware self-evolution helps embodied agents accumulate reusable procedural knowledge from their own trajectories.

URL PDF HTML 收藏
2603.12939 2026-07-13 cs.RO 版本更新

RoboStream: Weaving Spatio-Temporal Reasoning with Memory in Vision-Language Models for Robotics

RoboStream: 用记忆融合时空推理提升视觉语言模型在机器人中的应用

Yuzhi Huang, Jie Wu, Weijue Bu, Ziyi Xiong, Gaoyang Jiang, Ye Li, Kangye Ji, Shuzhao Xie, Yue Huang, Chenglei Wu, Jingyan Jiang, Zhi Wang

机构 * Shenzhen International Graduate School, Tsinghua University(深圳国际研究生院,清华大学) YXGN Robotics(YXGN机器人) China University of Mining and Technology(中国矿业大学) Shenzhen Technology University(深圳技术大学) Huazhong University of Science and Technology(华中科技大学) Xiamen University(厦门大学)

AI总结 RoboStream通过时空融合令牌和因果时空图实现持久记忆,解决机器人长时间任务中的几何锚定和状态追踪问题,提升长期任务表现。

Comments Accepted by ECCV 2026

详情
AI中文摘要

使可靠的长周期机器人操作成为开放世界具身智能的关键步骤。然而,基于视觉语言模型的规划器将每一步视为孤立的观测到动作映射,迫使它们在每个决策点从原始像素重新推断场景几何,同时缺乏对先前动作如何改变环境的认识。尽管在短周期表现强劲,这些系统缺乏持续的时空推理能力,无法实现持久的几何锚定和动作触发的状态转换记忆。没有持续的状态追踪,感知误差会在执行周期内累积,被暂时遮挡的对象会被灾难性遗忘,这些叠加的失败会导致先决条件违反,进而影响后续步骤。相比之下,人类保持持续的内心模型,持续跟踪空间关系和动作后果,而不是在每个瞬间重新构建。受这种人类因果时空推理能力的启发,我们提出了RoboStream,一种无需训练的框架,通过时空融合令牌实现几何锚定,将视觉证据绑定到3D几何属性以实现持久的对象定位,并通过因果时空图记录动作触发的状态转换。这种设计使规划器能够在不额外训练或微调的情况下追踪因果链并保留遮挡下的对象永久性。RoboStream在长周期RLBench上达到90.5%,在具有挑战性的现实世界积木搭建任务中达到44.4%,而SoFar和VoxPoser分别得分为11.1%,证明了时空推理和因果记忆是可靠长周期操作的关键缺失组件。

英文摘要

Enabling reliable long-horizon robotic manipulation is a crucial step toward open-world embodied intelligence. However, VLM-based planners treat each step as an isolated observation-to-action mapping, forcing them to reinfer scene geometry from raw pixels at every decision point while remaining unaware of how prior actions have reshaped the environment. Despite strong short-horizon performance, these systems lack the spatio-temporal reasoning required for persistent geometric anchoring and memory of action-triggered state transitions. Without persistent state tracking, perceptual errors accumulate across the execution horizon, temporarily occluded objects are catastrophically forgotten, and these compounding failures lead to precondition violations that cascade through subsequent steps. In contrast, humans maintain a persistent mental model that continuously tracks spatial relations and action consequences across interactions rather than reconstructing them at each instant. Inspired by this human capacity for causal spatio-temporal reasoning with persistent memory, we propose RoboStream, a training-free framework that achieves geometric anchoring through Spatio-Temporal Fusion Tokens (STF-Tokens), which bind visual evidence to 3D geometric attributes for persistent object grounding, and maintains causal continuity via a Causal Spatio-Temporal Graph (CSTG) that records action-triggered state transitions across steps. This design enables the planner to trace causal chains and preserve object permanence under occlusion without additional training or fine-tuning. RoboStream achieves 90.5% on long-horizon RLBench and 44.4% on challenging real-world block-building tasks, where both SoFar and VoxPoser score 11.1%, demonstrating that spatio-temporal reasoning and causal memory are critical missing components for reliable long-horizon manipulation.

URL PDF HTML 收藏
2607.08772 2026-07-10 cs.CV 新提交

Wat3R: Underwater 3D Geometry Learning without Annotations

Wat3R:无需标注的水下3D几何学习

Jiangwei Ren, Xingyu Jiang, Zijie Song, Wei Xu, Hongkai Lin, Dingkang Liang, Xiang Bai

机构 * Huazhong University of Science and Technology(华中科技大学)

AI总结 针对水下3D几何估计难题,提出跨域半监督学习框架Wat3R,基于师生架构,无需标注水下数据,利用未标注视频学习,设计跨视图一致性损失,构建Water3D数据集,实验证明其在水下多视图深度估计和点云重建上性能优于现有方法。

Comments Accepted to ECCV 2026. The dataset and code are available at https://github.com/LSXI7/Wat3R

详情
AI中文摘要

由于光衰减、散射以及缺乏大规模高质量3D标注,水下环境中的3D几何估计面临独特挑战。开创性方法依赖大量密集标注,在水下环境不实用。本文提出Wat3R,一种跨域半监督学习框架,能将前馈3D重建模型从空中场景适配到水下。该方法基于师生架构,无需任何标注水下数据,仅通过大量未标注真实水下视频片段学习稳健几何表示。还设计了跨视图一致性损失,利用其他视图几何线索补偿当前视图因水衰减和散射造成的信息退化。鉴于缺乏综合评估基准,构建了Water3D数据集用于几何任务评估。实验结果表明Wat3R在水下多视图深度估计和点云重建方面优于当前最先进方法。

英文摘要

Estimating 3D geometry in underwater environments presents unique challenges due to light attenuation, scattering, and the absence of large-scale, high-quality 3D annotations. Pioneering methods rely on massive dense annotations that are impractical in underwater settings. In this paper, we propose Wat3R, a cross-domain semi-supervised learning framework designed to adapt feed-forward 3D reconstruction models from air to underwater scenes. Uniquely, our method eliminates the need for any annotated underwater data following a teacher-student architecture, that learns robust geometry representations merely on abundant unlabeled real underwater video footage. We also design a cross-view consistency loss that leverages geometric cues from other views to compensate for the information degradation in the current view caused by water attenuation and scattering. Furthermore, considering the lack of comprehensive evaluation benchmarks, we construct Water3D, a diverse dataset covering various water bodies and underwater scenarios, designed for geometric task evaluation. Experimental results demonstrate that Wat3R outperforms current state-of-the-art methods in underwater multi-view depth estimation and point cloud reconstruction. The dataset and code are available at https://github.com/LSXI7/Wat3R .

URL PDF HTML 收藏
2607.08434 2026-07-10 cs.CV 新提交

DeltaV: Thinking with Visual State Updates in Unified Large Multimodal Models

DeltaV:在统一大型多模态模型中通过视觉状态更新进行思考

Pengjie Wang, Linger Deng, Zujia Zhang, Shaojie Zhang, Zhenbo Luo, Pei Fu, Jian Luan, Xiang Bai, Yuliang Liu

机构 * Huazhong University of Science and Technology(华中科技大学) Xiaomi Inc.(小米公司)

AI总结 研究针对统一大型多模态模型全图像生成范式的问题,提出DeltaV模型,通过视觉更新避免冗余,引入TSIM路由器匹配令牌预算,构建StructCoT数据集,实验证明该范式能减少视觉令牌、提升推理能力,DeltaV - 2B性能优异。

详情
AI中文摘要

当前统一大型多模态模型(ULMMs)通过文本推理和中间视觉状态支持交错多模态推理,但通常将每个视觉状态生成为完整图像。这种全图像生成范式引入了大量视觉令牌冗余,并稀释了对稀疏但推理关键状态转换的监督。我们提出了DeltaV,一种用视觉更新取代全图像生成的ULMM。基于历史视觉状态,DeltaV增量预测紧凑的更新令牌,以捕获推理步骤中的视觉变化,避免对不变内容的重复建模。为了使每次更新的令牌预算与视觉变化的幅度相匹配,DeltaV引入了时间相似性(TSIM)路由器,一旦边际重建增益低于阈值,就停止分配令牌。为了支持更多样化和可推广的推理,我们进一步构建了StructCoT,一个包含105万个样本、跨越44个任务领域的大规模交错多模态推理数据集。实验表明,视觉更新范式在不影响重建保真度的情况下,平均减少了55.6%的新生成视觉令牌,并比全图像生成提高了3.3%的多模态推理能力。在StructCoT和大规模多模态数据上训练的DeltaV - 2B在域内多模态推理评估中比更大的开源模型高出8.4%,在外部多模态推理和理解基准上比可比规模的Qwen3 - VL - 2B高出5.9%。代码、模型和StructCoT将在这个https URL上发布。

英文摘要

Current Unified Large Multimodal Models (ULMMs) support interleaved multimodal reasoning through textual reasoning and intermediate visual states, but typically generate each visual state as a full image. This full-image generation paradigm introduces substantial visual-token redundancy and dilutes supervision on sparse yet reasoning-critical state transitions. We propose DeltaV, a ULMM that replaces full-image generation with visual updates. Conditioned on historical visual states, DeltaV incrementally predicts compact update tokens that capture the visual changes across reasoning steps, avoiding repeated modeling of unchanged content. To align the token budget of each update with the magnitude of visual change, DeltaV introduces a temporal similarity (TSIM) Router, which stops allocating tokens once the marginal reconstruction gain falls below a threshold. To support more diverse and generalizable reasoning, we further construct StructCoT, a large-scale interleaved multimodal reasoning dataset with 1.05M samples spanning 44 task domains. Experiments show that the visual-update paradigm reduces newly generated visual tokens by 55.6\% on average without compromising reconstruction fidelity, and improves multimodal reasoning by 3.3\% over full-image generation. Trained with StructCoT and large-scale multimodal data, DeltaV-2B further outperforms substantially larger open-source models by 8.4\% on in-domain multimodal reasoning evaluations and surpasses the comparable-scale Qwen3-VL-2B by 5.9\% on external multimodal reasoning and understanding benchmarks. Code, models, and StructCoT will be released at https://github.com/Pengjie-W/DeltaV.

URL PDF HTML 收藏