arXivDaily arXiv每日学术速递 周一至周五更新

高校专区

University of Maryland, College Park(马里兰大学帕克分校)

至 收录 493
2607.08940 2026-07-21 cs.LG 版本更新

TSRouter: Dynamic Modality-Model Selection for Time Series Reasoning

TSRouter:用于时间序列推理的动态模态-模型选择

Fangxu Yu, Tao Feng, Dehai Min, Lu Cheng, Ge Liu, Tianyi Zhou

机构 * University of Maryland, College Park(马里兰大学帕克分校) University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校) University of Illinois Chicago(伊利诺伊大学芝加哥分校) MBZUAI(Mohamed Bin Zayed University of Artificial Intelligence)

AI总结 研究针对时间序列推理中不同模型能力互补及特性差异问题,提出基于图的TSRouter动态路由框架,通过构建异构图并将路由设为候选评分问题来选择最优模态-模型对,在多任务评估中性能显著提升,还具备零样本泛化等优势。

Comments Accepted to COLM 2026

详情
AI中文摘要

时间序列推理对解决现实世界问题至关重要。大语言模型(LLMs)和视觉-语言模型(VLMs)在处理时间序列数据时能力互补。LLMs将时间序列作为文本序列处理,保留精确数值理解,但难以处理全局模式;VLMs通过可视化时间序列有效捕捉模式,但可能丢失细粒度细节。此外,模型在特定任务专业知识和推理成本上差异显著。为此,引入TSRouter,一个基于图的动态路由框架。它构建任务、查询、模态和模型节点的异构图来关联查询特征、模态属性和模型能力。将路由制定为候选评分问题,根据用户定义的性能-成本偏好评估每个模态-模型对以选择最优候选。在4个不同时间序列推理任务上的综合评估表明,TSRouter显著优于各种基线,相对提升16%至46%。此外,TSRouter展示了对未见模型和新任务的强大零样本即插即用泛化能力,并通过成本感知优化在降低计算开销的同时保持高性能。

英文摘要

Time series reasoning is essential for real-world problem-solving. While both Large Language Models (LLMs) and Vision-Language Models (VLMs) can reason about time-series data, their capabilities are complementary: LLMs process time series as text sequences and thus preserve exact numerical understanding, but struggle with global patterns, whereas VLMs efficiently capture these patterns by visualizing time series but may lose fine-grained details. Moreover, models vary significantly in task-specific expertise and inference costs. Dynamically selecting the most suitable modality and model for each query is therefore crucial, yet challenging because it requires modeling the complex interactions among tasks, queries, modalities, and models, which carry rich contextual signals. To this end, we introduce TSRouter, a graph-based dynamic routing framework. TSRouter constructs a heterogeneous graph of task, query, modality, and model nodes to contextualize the interactions among query characteristics, modality attributes, and model capabilities. TSRouter formulates routing as a candidate scoring problem, where each modality-model pair is evaluated based on user-defined performance-cost preferences to select the optimal candidate. Comprehensive evaluations on 4 distinct time series reasoning tasks reveal that TSRouter substantially outperforms diverse baselines with 16\% to 46\% relative improvements. Furthermore, TSRouter demonstrates robust zero-shot plug-and-play generalization to unseen models and novel tasks and preserves high performance while reducing computational overhead through cost-aware optimization. Our code is available at https://github.com/tianyi-lab/TSRouter.

URL PDF HTML 收藏
2601.17950 2026-07-21 cs.CV 版本更新

UPLiFT: Efficient Pixel-Dense Feature Upsampling with Local Attenders

UPLiFT:高效像素密集特征上采样与局部关注者

Matthew Walmer, Saksham Suri, Anirud Aggarwal, Abhinav Shrivastava

机构 * University of Maryland, College Park(马里兰大学学院公园分校) Meta(Meta公司)

AI总结 UPLiFT通过高效局部关注者操作符实现低成本的像素密集特征上采样,优于现有方法。

Comments Accepted to CVPR 2026 as conference paper. Website: see https://www.cs.umd.edu/~mwalmer/uplift/. Code and Models: see https://github.com/mwalmer-umd/UPLiFT

详情
AI中文摘要

任务无关特征上采样的空间已逐渐成为研究的有前景领域,旨在高效地从预训练的视觉主干网络中生成更密集的特征。这些方法通过学习将低分辨率特征映射到高分辨率版本,以较低的成本实现密集特征的生成。尽管早期的工作使用了迭代上采样方法,但更近期的工作则转向了基于交叉注意力的方法,这可能会陷入它们所上采样的主干网络相同的效率扩展问题。在本工作中,我们证明了迭代上采样方法仍能与基于交叉注意力的方法竞争;而且,它们可以以更低的推理成本实现最先进的性能。我们提出了UPLiFT,一种通用的像素密集轻量级特征转换架构。我们还提出了一种高效的局部关注者操作符,以克服先前迭代特征上采样方法的限制。该操作符使用了一种完全局部定义的替代注意力池化公式。我们证明了我们的局部关注者使UPLiFT在整个上采样过程中能够保持稳定的特征,从而以比现有像素密集特征上采样器更低的推理成本实现最先进的性能。此外,我们还将UPLiFT应用于生成性下游任务,并展示了其在VAE特征上采样中的Coupled Flow Matching模型上实现了具有竞争力的性能。总而言之,UPLiFT提供了一种通用且高效的生成更密集特征的方法。

英文摘要

The space of task-agnostic feature upsampling has emerged as a promising area of research to efficiently create denser features from pre-trained visual backbones. These methods act as a shortcut to achieve dense features for a fraction of the cost by learning to map low-resolution features to high-resolution versions. While early works in this space used iterative upsampling approaches, more recent works have switched to cross-attention-based methods, which risk falling into the same efficiency scaling problems of the backbones they are upsampling. In this work, we demonstrate that iterative upsampling methods can still compete with cross-attention-based methods; moreover, they can achieve state-of-the-art performance with lower inference costs. We propose UPLiFT, an architecture for Universal Pixel-dense Lightweight Feature Transforms. We also propose an efficient Local Attender operator to overcome the limitations of prior iterative feature upsampling methods. This operator uses an alternative attentional pooling formulation defined fully locally. We show that our Local Attender allows UPLiFT to maintain stable features throughout upsampling, enabling state-of-the-art performance with lower inference costs than existing pixel-dense feature upsamplers. In addition, we apply UPLiFT to generative downstream tasks and show that it achieves competitive performance with state-of-the-art Coupled Flow Matching models for VAE feature upsampling. Altogether, UPLiFT offers a versatile and efficient approach to creating denser features.

URL PDF HTML 收藏
2505.11491 2026-07-21 cs.LG physics.comp-ph 版本更新

Potential failures of physics-informed machine learning in traffic flow modeling: theoretical and experimental analysis

交通流建模中物理信息机器学习的潜在失败:理论与实验分析

Yuan-Zheng Lei, Yaobang Gong, Dianwei Chen, Yao Cheng, Xianfeng Terry Yang

机构 * University of Maryland, College Park, MD 20742, United States(马里兰大学学院公园分校) Florida Atlantic University, Boca Raton, FL 33431, United States(佛罗里达 Atlantic 大学)

AI总结 研究探讨物理信息机器学习在交通流建模中失败的原因,定义失败情形,指出有效更新条件难满足致PIML失败,通过理论分析建立物理残差MSE下界,解释基于LWR的PIML能优于ARZ的原因。

Comments Some errors about the error bound proofness

详情
AI中文摘要

本研究探讨了物理信息机器学习(PIML)在宏观交通流建模中失败的原因。我们将失败定义为PIML模型在性能上比纯数据驱动和纯物理基线低给定阈值的情况。与其他领域不同,物理残差本身并不妨碍优化。有效更新要求数据和物理梯度与真实梯度形成锐角,低分辨率循环数据难以满足这一条件。神经网络无法准确逼近密度和速度,构建的物理残差因离散采样和时间平均而退化,失去捕捉PDE动力学的能力,直接导致PIML失败。理论上,虽然LWR和ARZ解是弱解,但在温和条件下,对于分段$C^k$初始数据,它们在冲击集之外仍为$C^k$,冲击集的勒贝格测度为零。因此,几乎所有探测器或配置点都位于残差有效的光滑区域,MLP无法精确表示不连续性并不重要。最后,我们建立了物理残差的MSE下界:在温和条件下,ARZ等高阶模型的一致性误差界比LWR严格更大。这解释了为什么基于LWR的PIML即使在高分辨率数据下也能优于基于ARZ的PIML,且随着分辨率增加差距缩小,与先前的实证结果一致。

英文摘要

This study investigates why physics-informed machine learning (PIML) can fail in macroscopic traffic flow modeling. We define failure as cases where a PIML model underperforms both purely data-driven and purely physics-based baselines by a given threshold. Unlike in other fields, physics residuals themselves do not hinder optimization in this setting. Instead, effective updates require both data and physics gradients to form acute angles with the true gradient, a condition difficult to satisfy with low-resolution loop data. In such cases, neural networks cannot accurately approximate density and speed, and the constructed physics residuals, already degraded by discrete sampling and temporal averaging, lose their ability to capture PDE dynamics, which directly leads to PIML failure. Theoretically, although LWR and ARZ solutions are weak solutions, for piecewise $C^k$ initial data they remain $C^k$ off the shock set under mild conditions, which has Lebesgue measure zero. Thus, almost all detector or collocation points lie in smooth regions where residuals are valid, and the MLP's inability to exactly represent discontinuities is immaterial. Finally, we establish MSE lower bounds of physics residuals: higher-order models such as ARZ have strictly larger consistency error bounds than LWR under mild conditions. This explains why LWR-based PIML can outperform ARZ-based PIML even with high-resolution data, with the gap shrinking as resolution increases, consistent with prior empirical findings.

URL PDF HTML 收藏
2607.16090 2026-07-20 cs.LG cs.AI 新提交

DADiff: Diffusion-Driven Cross-Domain Policy Adaptation for Reinforcement Learning

DADiff:用于强化学习的扩散驱动跨域策略适应

Hanyang Chen, Anirudh Satheesh, Longchao Da, Hua Wei

机构 * Arizona State University(亚利桑那州立大学) University of Maryland, College Park(马里兰大学帕克分校)

AI总结 研究强化学习中跨域策略适应问题,提出基于扩散的DADiff框架,利用源域和目标域生成轨迹差异估计动态不匹配,开发奖励修改和数据选择变体,实验表明该方法性能优于现有方法,有效解决动态不匹配。

Comments Accepted by IROS 2026

详情
AI中文摘要

在强化学习中,跨域转移策略面临着重大挑战,因为源域和目标域之间存在动态不匹配。本文考虑在线动态适应设置,即在源域使用足够数据训练策略,同时仅允许与目标域进行有限交互。现有一些工作通过使用域分类器、值引导数据过滤或表示学习来解决动态不匹配问题。相反,我们从生成建模角度研究域适应问题。具体来说,我们引入DADiff,这是一个基于扩散的框架,它利用下一状态生成过程中源域和目标域生成轨迹之间的差异来估计动态不匹配。我们开发了奖励修改和数据选择变体来使策略适应目标域。我们还进行了理论分析,表明给定策略在两个域之间的性能差异受生成轨迹偏差的限制。我们在具有各种转移的环境中进行了广泛实验,结果表明我们的方法比现有方法具有更好的性能,有效解决了动态不匹配问题。

英文摘要

Transferring policies across domains poses a vital challenge in reinforcement learning, due to the dynamics mismatch between the source and target domains. In this paper, we consider the setting of online dynamics adaptation, where policies are trained in the source domain with sufficient data, while only limited interactions with the target domain are allowed. There are a few existing works that address the dynamics mismatch by employing domain classifiers, value-guided data filtering, or representation learning. Instead, we study the domain adaptation problem from a generative modeling perspective. Specifically, we introduce DADiff, a diffusion-based framework that leverages the discrepancy between source and target domain generative trajectories in the generation process of the next state to estimate the dynamics mismatch. Both reward modification and data selection variants are developed to adapt the policy to the target domain. We also provide a theoretical analysis to show that the performance difference of a given policy between the two domains is bounded by the generative trajectory deviation. More discussions on the applicability of the variants and the connection between our theoretical analysis and the prior work are further provided. We conduct extensive experiments in environments with various shifts to validate the effectiveness of our method. The results demonstrate that our method provides superior performance compared to existing approaches, effectively addressing the dynamics mismatch. We provide the code of our method at https://github.com/hanyang-chen/DADiff-release

URL PDF HTML 收藏
2510.26792 2026-07-17 cs.LG cond-mat.dis-nn cs.CR

Learning Pseudorandom Numbers with Transformers: Permuted Congruential Generators, Curricula, and Interpretability

利用Transformer学习伪随机数:排列同余生成器、课程学习与可解释性

Tao Tao, Maissam Barkeshli

机构 * Department of Physics, University of Maryland, College Park, USA(大学物理系,马里兰大学,College Park, USA) Meta FAIR Joint Quantum Institute, University of Maryland(Meta FAIR联合量子研究所,马里兰大学)

AI总结 本研究探讨了Transformer模型学习排列同余生成器生成的序列能力,揭示了课程学习在处理大模数时的重要性,并发现嵌入层中存在位旋转不变的聚类现象。

Comments 10+13 pages, 8+21 figures

Journal ref International Conference on Learning Representations (ICLR) 2026

详情
AI中文摘要

我们研究了Transformer模型学习由排列同余生成器(PCG)生成的序列的能力,这是一种广泛使用的伪随机数生成器(PRNG)家族。PCG通过在隐藏状态上应用一系列位移、异或、旋转和截断操作,显著增加了线性同余生成器(LCG)的难度。我们证明,尽管如此,Transformer仍能成功地对未见过的序列进行上下文预测,这些序列来自多种PCG变体,并且这些任务超出了已发表的经典攻击范围。在我们的实验中,我们使用高达2^22的模数,模型参数多达5000万,数据集多达50亿个标记。令人惊讶的是,即使输出被截断为单个位,模型也能可靠地进行预测。当多个不同的PRNGs在训练期间一起呈现时,模型可以联合学习它们,识别不同排列的结构。我们展示了模数m的缩放定律:近完美预测所需的上下文序列元素数量随sqrt(m)增长。对于更大的模数,优化进入扩展停滞阶段;在我们的实验中,学习模数m≥2^20需要结合来自较小模数的训练数据,这表明课程学习是至关重要的。最后,我们分析了嵌入层,并揭示了一种新的聚类现象:顶部主成分会将整数输入自动分组到位旋转不变的聚类中,揭示了如何将表示从较小的模数转移到较大的模数。

英文摘要

We study the ability of Transformer models to learn sequences generated by Permuted Congruential Generators (PCGs), a widely used family of pseudo-random number generators (PRNGs). PCGs introduce substantial additional difficulty over linear congruential generators (LCGs) by applying a series of bit-wise shifts, XORs, rotations and truncations to the hidden state. We show that Transformers can nevertheless successfully perform in-context prediction on unseen sequences from diverse PCG variants, in tasks that are beyond published classical attacks. In our experiments we scale moduli up to $2^{22}$ using up to $50$ million model parameters and datasets with up to $5$ billion tokens. Surprisingly, we find even when the output is truncated to a single bit, it can be reliably predicted by the model. When multiple distinct PRNGs are presented together during training, the model can jointly learn them, identifying structures from different permutations. We demonstrate a scaling law with modulus $m$: the number of in-context sequence elements required for near-perfect prediction grows as $\sqrt{m}$. For larger moduli, optimization enters extended stagnation phases; in our experiments, learning moduli $m \geq 2^{20}$ requires incorporating training data from smaller moduli, demonstrating a critical necessity for curriculum learning. Finally, we analyze embedding layers and uncover a novel clustering phenomenon: the top principal components spontaneously group the integer inputs into bitwise rotationally-invariant clusters, revealing how representations can transfer from smaller to larger moduli.

URL PDF HTML 收藏
2607.13285 2026-07-16 cs.AI cs.SE 新提交

Harness Handbook: Making Evolving Agent Harnesses Readable,Navigable, and Editable

Harness手册:使不断演进的智能体框架具有可读性、可导航性和可编辑性

Ruhan Wang, Yucheng Shi, Zongxia Li, Zhongzhi Li, Yue Yu, Junyao Yang, Kishan Panaganti, Haitao Mi, Dongruo Zhou, Leoweiliang

机构 * Tencent(腾讯) Indiana University(印第安纳大学) University of Maryland, College Park(马里兰大学帕克分校) University of Georgia(佐治亚大学) National University of Singapore(新加坡国立大学)

AI总结 研究智能体框架演进中行为定位难的问题,提出通过Harness手册和行为引导的渐进式披露,以行为为中心自动合成框架表示并辅助规划,提高行为定位和编辑计划质量,助力复杂智能体系统发展。

Comments 29 pages, 6 figures. Project page: https://ruhan-wang.github.io/Harness-Handbook/

详情
AI中文摘要

现代人工智能智能体的能力不仅取决于其基础模型,还取决于其框架,框架用于构建提示、管理状态、调用工具和协调执行。随着模型、API、环境和需求的发展,框架必须不断修改。在进行此类更改之前,开发人员或编码智能体必须识别实现目标行为的所有代码位置。这很困难,因为生产框架庞大、紧密耦合且行为分散,而修改请求描述系统应做什么,存储库按文件和模块组织。代码搜索、存储库索引和长上下文处理便于检查,但仍需手动恢复行为到代码的映射。行为定位因此是框架演进的核心瓶颈。我们引入了Harness手册,这是一种以行为为中心的表示,通过静态分析和LLM辅助结构化从框架代码库自动合成,将每个行为与其相应源链接起来。我们还引入了行为引导的渐进式披露(BGPD),它引导智能体从高级行为到相关实现细节,并根据当前源验证候选位置。在来自两个开源框架的各种修改请求上,手册辅助规划提高了行为定位和编辑计划质量,同时使用更少的规划器令牌,在分散站点、很少执行的路径和跨模块交互方面收益最大。因此,不断发展复杂的智能体系统不仅取决于生成编辑,还取决于确定这些编辑应在何处进行。

英文摘要

The capability of a modern AI agent depends not only on its foundation model but also on its harness, which constructs prompts, manages state, invokes tools, and coordinates execution. As models, APIs, environments, and requirements evolve, the harness must be continually modified. Before such a change can be made, a developer or coding agent must identify all code locations that implement the target behavior. This is difficult because production harnesses are large, tightly coupled, and behaviorally distributed, while modification requests describe what the system should do and repositories are organized by files and modules. Code search, repository indexing, and long-context processing ease inspection, but still leave this behavior-to-code mapping to be recovered by hand. Behavior localization is therefore a central bottleneck in harness evolution. We introduce the Harness Handbook, a behavior-centric representation synthesized automatically from a harness codebase via static analysis and LLM-assisted structuring, linking each behavior to its corresponding source. We also introduce Behavior-Guided Progressive Disclosure (BGPD), which guides agents from high-level behaviors to relevant implementation details and verifies candidate locations against the current source. On diverse modification requests from two open-source harnesses, Handbook-Assisted planning improves behavior localization and edit-plan quality while using fewer planner tokens, with the largest gains on scattered sites, rarely executed paths, and cross-module interactions. Evolving complex agentic systems thus depends not only on generating edits, but also on determining where those edits should be made.

URL PDF HTML 收藏
2405.17366 2026-07-16 cs.LG eess.SP 版本更新

EM-GANSim: Real-time and Accurate EM Simulation Using Conditional GANs for 3D Indoor Scenes

EM-GANSim:使用条件生成对抗网络进行3D室内场景的实时准确电磁仿真

Ruichen Wang, Dinesh Manocha

机构 * University of Maryland, College Park(马里兰大学College Park分校)

AI总结 研究针对3D室内无线通信电磁仿真问题,提出EM-GANSim方法,用改进条件GAN结合电磁理论预测功率分布,经多场景评估具有高准确率和低计算时间,是首个此类实时算法,还将发布代码和数据集。

Comments 12 pages, 9 figures, 5 tables

详情
AI中文摘要

我们提出了一种用于实时电磁传播的新型机器学习(ML)方法(EM-GANSim),用于3D室内环境中的无线通信仿真。我们的方法使用了一种改进的条件生成对抗网络(GAN),它结合了编码几何和发射机位置,同时遵循电磁传播理论。整体受物理启发的学习能够预测3D场景中的功率分布,用热图表示。我们在15个复杂的3D室内环境中评估了我们的方法,结果中还包括4个额外场景,展示了模型在不同条件下的通用性。我们的整体准确率与基于光线追踪的电磁仿真相当,平均平方误差值更低证明了这一点。此外,我们基于GAN的方法大幅减少了计算时间,在复杂基准测试中实现了5倍的加速。在实践中,它可以在几毫秒内计算出3D室内环境中任何位置的信号强度。我们还展示了一个包含3D模型和电磁光线追踪模拟热图的大型数据集。据我们所知,EM-GANSim是第一个用于复杂3D室内环境电磁仿真的实时算法。我们计划发布代码和数据集。

英文摘要

We present a novel machine-learning (ML) approach (EM-GANSim) for real-time electromagnetic (EM) propagation that is used for wireless communication simulation in 3D indoor environments. Our approach uses a modified conditional Generative Adversarial Network (GAN) that incorporates encoded geometry and transmitter location while adhering to the electromagnetic propagation theory. The overall physically-inspired learning is able to predict the power distribution in 3D scenes, which is represented using heatmaps. We evaluated our method on 15 complex 3D indoor environments, with 4 additional scenarios later included in the results, showcasing the generalizability of the model across diverse conditions. Our overall accuracy is comparable to ray tracing-based EM simulation, as evidenced by lower mean squared error values. Furthermore, our GAN-based method drastically reduces the computation time, achieving a 5X speedup on complex benchmarks. In practice, it can compute the signal strength in a few milliseconds on any location in 3D indoor environments. We also present a large dataset of 3D models and EM ray tracing-simulated heatmaps. To the best of our knowledge, EM-GANSim is the first real-time algorithm for EM simulation in complex 3D indoor environments. We plan to release the code and the dataset.

URL PDF HTML 收藏
2606.13460 2026-07-15 cs.CV 版本更新

VISA: VLM-Guided Instance Semantic Auditing for 3D Occupancy World Models

VISA: VLM引导的实例语义审计用于3D占据世界模型

Ruiqi Xian, Yuehan Xian, Jing Liang, Xuewei Qi, Dinesh Manocha

机构 * University of Maryland College Park(马里兰大学帕克分校) Nanjing University of Posts and Telecommunications(南京邮电大学) Stanford University(斯坦福大学) Motional AD Inc.(Motional AD公司)

AI总结 提出VISA方法,利用离线VLM对每个物理对象实例进行结构化语义审计,并通过可靠性加权损失蒸馏到3D占据模型中,无需VLM推理即可提升封闭集占据mIoU。

详情
AI中文摘要

语义3D占据为自动驾驶和机器人决策提供体素化世界状态,但对象和稀有类错误会影响自由空间解释、碰撞检测和时间状态传播。我们表明,常见的VLM策略(将3D体素或对象特征与裁剪-标题嵌入对齐)提高了文本-空间相似性,但未能可靠地改善封闭集占据mIoU。受此不匹配启发,我们提出VISA,一种针对现有占据世界模型的训练时语义审计方法。VISA对每个物理对象实例的代表性裁剪查询离线VLM,获得包含类别假设、可能混淆、可靠性、属性和证据的结构化审计,并将其沿对象轨迹传播。审计被关联到匹配的3D对象体素,并通过可靠性加权分类、属性因子和场景级审计图损失蒸馏到语义logits中,而推理保持不变且无需VLM。在nuScenes上,三次运行平均,VISA将OccWorld从19.06提升到20.05 mIoU,GaussianWorld从21.36提升到21.91 mIoU;在GaussianWorld上,对象mIoU从18.18提升到19.16,稀有类mIoU从15.60提升到16.79。这些结果表明,VLM更适合作为可靠性感知的语义审计器而非通用标题嵌入目标用于封闭集占据。

英文摘要

Semantic 3D occupancy provides a voxelized world state for autonomous driving and robot decision making, but object and rare-class errors can affect free-space interpretation, collision checking, and temporal state propagation. We show that a common VLM strategy, aligning 3D voxel or object features with crop-caption embeddings, improves text-space similarity without reliably improving closed-set occupancy mIoU. Motivated by this mismatch, we propose VISA, a training-time semantic auditing approach for existing occupancy world models. VISA queries an offline VLM on a representative crop of each physical object instance, obtains a structured audit with class hypotheses, plausible confusions, reliability, attributes, and evidence, and propagates it along the object track. The audit is grounded to matched 3D object voxels and distilled into semantic logits through reliability-weighted taxonomy, attribute-factor, and scene-level audit graph losses, while inference remains unchanged and requires no VLM. On nuScenes, averaged across three runs, VISA improves OccWorld from 19.06 to 20.05 mIoU and GaussianWorld from 21.36 to 21.91 mIoU; on GaussianWorld, object mIoU improves from 18.18 to 19.16 and rare-class mIoU from 15.60 to 16.79. These results suggest that VLMs are better suited to closed-set occupancy as reliability-aware semantic auditors than as generic caption-embedding targets.

URL PDF HTML 收藏
2607.10037 2026-07-14 cs.RO 新提交

Plug-and-Play Reweighting for Resilient Collaborative Decision-Making in Connected Autonomous Driving

用于联网自动驾驶中弹性协作决策的即插即用重加权方法

Jiewen Liu, Rui Liu, Matthew Lee, Ming C. Lin, Xiaorui Liu, Peng Gao

机构 * North Carolina State University(北卡罗来纳州立大学) University of Maryland, College Park(马里兰大学帕克分校) University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校)

AI总结 针对联网自动驾驶中协作决策受感知噪声和对抗攻击影响的问题,提出弹性协作决策(RCDM)框架,含基于注意力的编解码器,设计即插即用重加权模块,经高保真模拟评估,性能优于现有方法,实现最优弹性能。

Comments 8 pages, 4 figures, 2 tables. Accepted to the 2026 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2026)

详情
AI中文摘要

协作决策是多机器人系统(如联网自动驾驶车辆)的一项基本能力。然而,协作方中的感知噪声和对抗攻击会严重影响决策可靠性。现有方法通常依赖针对特定攻击的防御进行再训练或基于限制性扰动假设来提高弹性,这限制了其实用性。本文提出了一种新颖的弹性协作决策(RCDM)框架,它由一个基于注意力的编码器和一个基于注意力的解码器组成。为提高对受损观测的弹性,设计了一种即插即用重加权模块,通过分析邻域点相对于局部结构的一致性,对偏离局部中位数较大的点赋予较小权重,从而降低受损输入的影响。该模块可无缝集成到基于注意力的协作决策中,无需额外训练。在高保真模拟中评估了该方法,考虑了感知噪声和五种攻击类型。实验结果表明,该方法始终比现有方法高出26%,并实现了当前最优的弹性能。

英文摘要

Collaborative decision-making is a fundamental capability in multi-robot systems, such as connected autonomous vehicles. However, perceptual noise and adversarial attacks in collaborators can severely affect decision reliability. Overall, existing methods typically rely on retraining with attack-specific defenses or on restrictive perturbation assumptions to improve resilience, which limits their practicality. In this paper, we propose a novel Resilient Collaborative Decision-Making (RCDM) framework that consists of an attention-based encoder for extracting individual robot perceptual embeddings and an attention-based decoder for fusing collaborator perceptions and making decisions. To improve resilience to corrupted observations, we design a novel plug-and-play reweighting module that down-weights the influence of corrupted inputs by analyzing the consistency of neighborhood points relative to the local structure and assigning smaller weights to points that deviate strongly from the local median. This module can be seamlessly integrated into attention-based collaborative decision-making without requiring additional training. We evaluate our method in high-fidelity simulations, considering perceptual noise and five types of attacks across diverse accident-prone scenarios. Experimental results demonstrate that our approach consistently outperforms existing methods by up to 26% and achieves state-of-the-art resilient performance.

URL PDF HTML 收藏
2607.08964 2026-07-14 cs.AI 版本更新

Long-Horizon-Terminal-Bench: Testing the Limits of Agents on Long-Horizon Terminal Tasks with Dense Reward-Based Grading

长视野终端基准测试:使用基于密集奖励的评分方式测试智能体在长视野终端任务中的极限

Zongxia Li, Zhongzhi Li, Yucheng Shi, Ruhan Wang, Junyao Yang, Zhichao Liu, Xiyang Wu, Anhao Li, Yue Yu, Ninghao Liu, Lichao Sun, Haotao Mi, Leowei Liang

机构 * Tencent(腾讯) University of Maryland, College Park(马里兰大学帕克分校) University of Georgia(佐治亚大学) University of Minnesota, Twin Cities(明尼苏达大学双城分校) Indiana University(印第安纳大学) Lehigh University(里海大学) National University of Singapore(新加坡国立大学) The Hong Kong Polytechnic University(香港理工大学)

AI总结 研究针对现有终端基准测试局限,引入长视野终端基准测试(Long-Horizon-Terminal-Bench),含46个长视野任务。通过分解为分级子任务提供密集中间奖励,评估15个前沿模型,揭示改进空间,分析失败模式,发布该基准测试助力长视野终端智能体发展。

Comments 17 pages

详情
AI中文摘要

人工智能智能体已能自主完成简短、明确的任务。然而,现有的终端基准测试大多聚焦于几分钟内就能完成的简单问题,且仅通过最终结果评估。这种设置忽略了中间进展和部分解决方案,产生稀疏奖励信号,无法全面了解智能体能力。我们引入了长视野终端基准测试,它包含46个长视野任务,涵盖九个类别。每个任务采用终端基准测试风格设置,有参考解决方案或模拟引擎,并进一步分解为细粒度的分级子任务。这使得能有密集的中间奖励和部分分数,不仅能评估智能体是否达成最终目标,还能了解其在开放式工作流程中的进展程度。这些任务通常需要数百个情节以及数分钟到数小时的执行时间,强调长视野规划、长上下文管理和迭代调试。我们评估了15个前沿模型,发现智能体平均每个任务消耗990万个令牌,每次运行大约有231个情节和85.3分钟的执行时间,这使得长视野终端基准测试比之前基于终端的基准测试要求更高。即使是测试中最强的模型,在部分奖励阈值为0.95时,通过率为15.2%,在完美奖励阈值为1.0时,通过率为10.9%,而各模型的平均通过率在两个阈值下分别为4.3%和1.7%。这些结果表明仍有改进空间。我们进一步分析了失败模式和错误模式,并发布长视野终端基准测试以支持长视野终端智能体的未来发展。

英文摘要

AI agents have become capable of autonomously completing short, well-specified tasks. However, existing terminal benchmarks largely focus on simple problems that finish within minutes and are evaluated only by their final outcome. This setup overlooks intermediate progress and partial solutions, yielding sparse reward signals and an incomplete picture of agent capability. We introduce Long-Horizon-Terminal-Bench, a terminal benchmark of 46 long-horizon tasks spanning nine categories, including experiment reproduction, software engineering, multimodal analysis, interactive games, and scientific computing. Each task follows a Terminal-Bench-style setup with a reference solution or simulation engine, but is further decomposed into fine-grained graded subtasks. This design enables dense intermediate rewards and partial credit, allowing evaluation to capture not only whether an agent reaches the final goal, but also how far it progresses on open-ended workflows. Tasks in Long-Horizon-Terminal-Bench typically require hundreds of episodes and minutes to hours of execution, stressing long-horizon planning, long-context management, and iterative debugging rather than one-shot problem solving. We evaluate 15 frontier models and find that agents consume on average 9.9M tokens per task, with roughly 231 episodes and 85.3 minutes of execution time per run, making Long-Horizon-Terminal-Bench more demanding than prior terminal-based benchmarks. Even the strongest tested model achieves 15.2% pass@1 at a partial-reward threshold of 0.95 and 10.9% at a perfect-reward threshold of 1.0, while the mean pass rate across models is 4.3% and 1.7% under the two thresholds, respectively. These results reveal headroom for improvement. We further analyze failure modes and error patterns, and release Long-Horizon-Terminal-Bench to support future progress on long-horizon terminal agents.

URL PDF HTML 收藏
2601.22588 2026-07-10 cs.CL cs.AI cs.LG 版本更新

Rethinking LLM-as-a-Judge: Representation-as-a-Judge with Small Language Models via Semantic Capacity Asymmetry

重新思考大语言模型作为评判器:通过语义能力不对称利用小语言模型进行表示作为评判器

Zhuochun Li, Yong Zhang, Ming Li, Yuelyu Ji, Yiming Zeng, Ning Cheng, Yun Zhu, Yanmeng Wang, Shaojun Wang, Jing Xiao, Daqing He

机构 * Ping An Technology (Shenzhen) Co., Ltd.(平安科技(深圳)有限公司) University of Pittsburgh(匹兹堡大学) University of Maryland, College Park(马里兰大学学院公园分校) University of Connecticut(康涅狄格大学)

AI总结 研究探讨小语言模型能否作高效评估器,基于语义能力不对称假设提出“表示作为评判器”范式,通过INSPECTOR框架实例化,实验显示其在推理基准测试中性能出色,为可扩展评估提供了更优选择。

Journal ref ICLR 2026

详情
AI中文摘要

大语言模型(LLMs)被广泛用作无需参考的评估器,但“LLM作为评判器”范式成本高、不透明且对提示设计敏感。本文研究较小模型能否通过利用内部表示而非表面生成来作为高效评估器。发现小语言模型虽生成能力弱,但隐藏状态中编码了丰富评估信号,由此提出语义能力不对称假设。基于此从“LLM作为评判器”转变为“表示作为评判器”,通过INSPECTOR框架实例化该范式,实验表明其性能优于基于提示的小语言模型且接近完整LLM评判器,提供了更高效、可靠和可解释的评估方式。

英文摘要

Large language models (LLMs) are widely used as reference-free evaluators via prompting, but this "LLM-as-a-Judge" paradigm is costly, opaque, and sensitive to prompt design. In this work, we investigate whether smaller models can serve as efficient evaluators by leveraging internal representations instead of surface generation. We uncover a consistent empirical pattern: small LMs, despite with weak generative ability, encode rich evaluative signals in their hidden states. This motivates us to propose the Semantic Capacity Asymmetry Hypothesis: evaluation requires significantly less semantic capacity than generation and can be grounded in intermediate representations, suggesting that evaluation does not necessarily need to rely on large-scale generative models but can instead leverage latent features from smaller ones. Our findings motivate a paradigm shift from LLM-as-a-Judge to Representation-as-a-Judge, a decoding-free evaluation strategy that probes internal model structure rather than relying on prompted output. We instantiate this paradigm through INSPECTOR, a probing-based framework that predicts aspect-level evaluation scores from small model representations. Experiments on reasoning benchmarks (GSM8K, MATH, GPQA) show that INSPECTOR substantially outperforms prompting-based small LMs and closely approximates full LLM judges, while offering a more efficient, reliable, and interpretable alternative for scalable evaluation. The code and data are available at: https://github.com/zhuochunli/Representation-as-a-judge

URL PDF HTML 收藏
2607.07693 2026-07-09 cs.LG cs.AI cs.CV 新提交

Selective Timestep Weighting and Advantage-Based Replay for Sample-Efficient Diffusion RLHF

用于样本高效扩散强化学习从人类反馈中学习的选择性时间步加权和基于优势的重放

Eric Zhu, Abhinav Shrivastava, Soumik Mukhopadhyay

机构 * Carnegie Mellon University(卡内基梅隆大学) University of Maryland, College Park(马里兰大学帕克分校)

AI总结 研究如何提高扩散模型中强化学习从人类反馈的效率,提出选择性时间步加权和基于优势的重放两种互补策略,通过强调信息丰富的时间步和轨迹优化,提升反馈效率,样本效率比基线提高6倍。

Comments 19 pages, 18 figures, 4 tables. Submission under review. A shorter, non-archival 4-page abstract version of this work was accepted to CVPR 2026 Workshops (GCV, CVEU)

详情
AI中文摘要

强化学习从人类反馈(RLHF)已成为使生成模型与人类偏好对齐的强大范式。然而,将RLHF应用于扩散模型在反馈效率方面仍然很低,现有方法通常需要大量人类或奖励模型评估。本文提出两种互补策略,在保持对未见提示的泛化能力的同时大幅提高扩散RLHF的反馈效率。关键观察是扩散轨迹中的奖励信息分布不均。首先引入时间步加权方案,在策略优化期间重新加权去噪步骤,并从理论和经验上进行分析。其次引入重放机制,优先考虑信息丰富的轨迹。这些策略显著提高了扩散RLHF的反馈效率,在相同超参数设置下,样本效率比广泛使用的基线提高了6倍。

英文摘要

Reinforcement learning from human feedback (RLHF) has emerged as a powerful paradigm for aligning generative models with human preferences. However, applying RLHF to diffusion models remains highly feedback inefficient, as existing approaches typically require large amounts of human or reward model evaluations. This limitation reduces the practicality of diffusion RLHF in realworld settings where feedback is the primary bottleneck. In this paper, we propose two complementary strategies that substantially improve the feedback efficiency of diffusion RLHF while preserving generalization to unseen prompts. Our key observation is that reward information in diffusion trajectories is unevenly distributed: not all denoising timesteps or trajectories contribute equally to learning from a reward signal. By emphasizing informative timesteps and trajectories during optimization, we obtain more effective gradient updates. First, we introduce a per-timestep weighting scheme that reweights denoising steps during policy optimization. We theoretically connect this weighting to the optimal convergence properties of proximal policy optimization (PPO) and approximate the resulting weighting trend empirically. Second, we introduce a replay mechanism that prioritizes informative trajectories, enabling the model to reuse past samples instead of repeatedly querying new rewards. Together, these strategies significantly improve the feedback efficiency of diffusion RLHF. Under identical hyperparameter settings, our approach achieves up to a 6$\times$ improvement in sample efficiency compared to widely used diffusion RLHF baselines.

URL PDF HTML 收藏
2607.05861 2026-07-08 cs.CL cs.LG 新提交

Mitigating Factual Hallucination in Large Reasoning Models via Mixed-Mode Advantage Regularization

通过混合模式优势正则化减轻大型推理模型中的事实幻觉

Kaishen Wang, Tong Zheng, Xuehao Cui, Ruibo Chen, Tianyi Xiong, Heng Huang

机构 * Department of Computer Science, University of Maryland, College Park(计算机科学系,马里兰大学,College Park)

AI总结 研究大型推理模型在面向事实问答中存在思维诱导幻觉的问题,提出MARGO框架,通过构建混合模式展开组评估思维价值,抑制幻觉思维,实验证明该方法在提升事实可靠性的同时保留了推理能力。

Comments 19 pages, 3 figures, 8 tables

详情
AI中文摘要

大型推理模型(LRMs)通过在最终答案前生成明确的思维痕迹来提升语言模型能力。在面向事实的问答(QA)中,这种思维通常有助于模型恢复相关知识并完善答案,提升整体性能。然而,我们发现这种好处在实例层面并不一致:明确的思维也可能推翻正确的非思维答案并导致事实偏差。我们将这种失败模式称为“思维诱导幻觉”。为解释此现象,我们将面向事实的QA中的明确思维表述为模型直接回答倾向的思维残差,其既可以恢复缺失知识,也可能引入无根据的关联。基于此表述,我们提出了MARGO(混合模式优势正则化的有界优化),这是一个强化学习框架,在优势估计中使用非思维展开作为同模型参考。通过构建包含思维和非思维轨迹的混合模式展开组,MARGO评估明确思维是否在直接回答之外增加了事实价值,从而抑制易产生幻觉的思维,同时保留有益的思维行为。在多个面向事实的QA基准上的实验表明,MARGO比强大的基线提高了事实可靠性,而在数学基准上的评估表明它保留了一般推理能力。

英文摘要

Large reasoning models (LRMs) improve language model capabilities by generating explicit thinking traces before final answers. In factuality-oriented question answering (QA), such thinking often improves overall performance by helping the model recover relevant knowledge and refine its answers. However, we find that this benefit is not uniform at the instance level: explicit thinking can also overturn correct non-thinking answers and lead to factual drift. We refer to this failure mode as \emph{thinking-induced hallucination}. To explain this phenomenon, we formulate explicit thinking in factuality QA as a thinking residual over the model's direct-answer tendency, which can either recover missing knowledge or introduce unsupported associations. Based on this formulation, we propose MARGO, \underline{\textit{M}}ixed-Mode \underline{\textit{A}}dvantage \underline{\textit{R}}egularization for \underline{\textit{G}}rounded \underline{\textit{O}}ptimization, a reinforcement learning framework that uses non-thinking rollouts as same-model references in advantage estimation. By constructing mixed-mode rollout groups with both thinking and non-thinking trajectories, MARGO evaluates whether explicit thinking adds factual value beyond direct answering, thereby suppressing hallucination-prone thinking while preserving beneficial thinking behaviors. Experiments across multiple factuality-oriented QA benchmarks demonstrate that MARGO improves factual reliability over strong baselines, while evaluations on mathematical benchmarks show that it preserves general reasoning ability.

URL PDF HTML 收藏
2606.27377 2026-07-08 cs.CV cs.CL cs.LG 新提交

DanceOPD: On-Policy Generative Field Distillation

DanceOPD:在策略生成场蒸馏

Wei Zhou, Xiongwei Zhu, Zelin Xu, Bo Dong, Lixue Gong, Yongyuan Liang, Meng Chu, Leigang Qu, Lingdong Kong, Wei Liu, Tat-Seng Chua

机构 * ByteDance Seed(字节跳动Seed) NUS(新加坡国立大学) UMD(马里兰大学) HKUST(香港科技大学)

AI总结 提出DanceOPD框架,通过将每个样本路由到能力场并查询低噪声学生诱导状态,用速度MSE损失训练流匹配模型,实现文本到图像、局部编辑和全局编辑的多能力组合,提升目标能力同时保持生成质量。

Comments Technical Report; 40 pages, 13 figures, 9 tables; Project Page at https://danceopd.github.io/ GitHub Repo at https://github.com/worldbench/DanceOPD

详情
AI中文摘要

现代图像生成需要一个统一多种能力的单一模型,包括文本到图像(T2I)、局部编辑和全局编辑。然而,这些能力很少自然对齐且经常冲突。例如,编辑往往会降低T2I性能,而全局和局部编辑相互干扰。因此,有效组合这些能力已成为图像生成模型训练的核心挑战。为了解决这个问题,我们引入了DanceOPD,一种用于流匹配模型的在策略生成场蒸馏框架,它将每个样本路由到一个能力场,查询一个低噪声学生诱导状态,并使用简单的速度MSE目标进行训练。每个能力源被定义为共享流状态空间上的速度场,学生从其自身展开状态上查询的场中学习以组合专家能力。该公式还吸收了算子定义的场,如无分类器引导。在T2I、编辑、真实感场吸收和CFG吸收上的全面实验表明,我们的方法改进了多能力组合,在保持锚定生成质量的同时增强了目标能力。我们相信这项工作为流匹配模型中的生成场蒸馏建立了一条实用途径。

英文摘要

Modern image generation demands a single model that unifies diverse capabilities, including text-to-image (T2I), local editing, and global editing. However, these capabilities are rarely naturally aligned and often conflict. For instance, editing tends to degrade T2I performance, while global and local editing interfere with each other. Consequently, effectively composing these capabilities has become a central challenge for image generation model training. To tackle this, we introduce DanceOPD, an on-policy generative field distillation framework for flow-matching models that routes each sample to one capability field, queries one low-noise student-induced state, and trains with a simple velocity MSE objective. With each capability source defined as a velocity field over the shared flow state space, the student learns from fields queried on its own rollout states to compose expert capabilities. This formulation also absorbs operator-defined fields such as classifier-free guidance. Comprehensive experiments on T2I, editing, realism-field absorption, and CFG absorption show that our approach improves multi-capability composition, strengthening target capabilities while preserving anchor generation quality. We believe this work establishes a practical route for generative field distillation in flow-matching models.

URL PDF HTML 收藏
2409.06067 2026-07-08 cs.AI cs.CL cs.LG 版本更新

MLLM-LLaVA-FL: Multimodal Large Language Model Assisted Federated Learning

MLLM-LLaVA-FL:多模态大语言模型辅助联邦学习

Jianyi Zhang, Hao Frank Yang, Ang Li, Xin Guo, Pu Wang, Haiming Wang, Yiran Chen, Hai Li

机构 * Duke University(杜克大学) Johns Hopkins University(约翰霍普金斯大学) University of Maryland College Park(马里兰大学学院市分校) Lenovo Research(联想研究院)

AI总结 针对联邦学习中数据异质性问题,提出MLLM-LLaVA-FL框架,利用多模态大语言模型,通过全球视觉文本预训练、客户端本地训练和服务器端全局对齐三个阶段,提升联邦学习性能。

Comments Accepted to WACV 2025

Journal ref IEEE/CVF Winter Conference on Applications of Computer Vision (WACV 2025)

详情
AI中文摘要

以往联邦学习(FL)研究常因不同客户端数据异质性而性能下降。鉴于多模态大语言模型(MLLMs)如GPT-4v和LLaVA在多模态任务中表现出色,本文引入多模态大语言模型辅助联邦学习(MLLM-LLaVA-FL)框架。该框架利用MLLMs的跨模态表示能力和开源数据,在服务器端解决异质性和长尾挑战。其有三个关键阶段:全球视觉文本预训练、客户端本地训练、服务器端全局对齐。实验表明,该框架在FL数据异质性和长尾分布场景中性能良好。

英文摘要

Previous studies on federated learning (FL) often encounter performance degradation due to data heterogeneity among different clients. In light of the recent advances in multimodal large language models (MLLMs), such as GPT-4v and LLaVA, which demonstrate their exceptional proficiency in multimodal tasks, such as image captioning and multimodal question answering. We introduce a novel federated learning framework, named Multimodal Large Language Model Assisted Federated Learning (MLLM-LLaVA-FL), which employs powerful MLLMs at the server end to address the heterogeneous and long-tailed challenges. Owing to the advanced cross-modality representation capabilities and the extensive open-vocabulary prior knowledge of MLLMs, our framework is adept at harnessing the extensive, yet previously underexploited, open-source data accessible from websites and powerful server-side computational resources. Hence, the MLLM-LLaVA-FL not only enhances the performance but also avoids increasing the risk of privacy leakage and the computational burden on local devices, distinguishing it from prior methodologies. Our framework has three key stages. Initially, we conduct global visual-text pretraining of the model. This pretraining is facilitated by utilizing the extensive open-source data available online, with the assistance of MLLMs. Subsequently, the pretrained model is distributed among various clients for local training. Finally, once the locally trained models are transmitted back to the server, a global alignment is carried out under the supervision of MLLMs to further enhance the performance. Experimental evaluations on established benchmarks, show that our framework delivers promising performance in the typical scenarios with data heterogeneity and long-tail distribution across different clients in FL.

URL PDF HTML 收藏
2607.04461 2026-07-07 cs.CV 新提交

Flash-BoN: Instant Drafts for Inference-Time Scaling in Diffusion Models

Flash-BoN:扩散模型推理时缩放的即时草稿

Ruchit Rawal, Reza Shirkavand, Sayak Paul, Yuxin Wen, Heng Huang, Yizheng Chen, Tom Goldstein, Gowthami Somepalli

机构 * University of Maryland, College Park(美国马里兰大学帕克分校) Hugging Face(拥抱脸)

AI总结 研究文本到图像生成推理时缩放,对比简单BoN与引导搜索方法,发现简单BoN在实际评估中表现良好。提出Flash-BoN,结合多种加速方法生成草稿候选,经多阶段验证选最有潜力的优化,在多基准和模型规模下优于基线。

详情
AI中文摘要

文本到图像生成的推理时间缩放已从简单的Best-of-$N$(BoN)采样发展到引导搜索方法,这些方法在中间去噪步骤验证和引导候选轨迹。本文表明在实际评估中简单BoN已能匹配或超越多种引导搜索技术。提出Flash-BoN,通过组合多种加速方法生成草稿候选,经多阶段验证优化,在多基准和模型规模下优于基线,还能与现有技术结合提升效果。

英文摘要

Inference-time scaling for text-to-image generation has progressed from simple Best-of-$N$ (BoN) sampling to guided search methods that verify and steer candidate trajectories at intermediate denoising steps. These approaches focus on when and how often to verify during denoising but largely treat the cost of generation itself as fixed. Moreover, the standard practice of comparing methods by number of function evaluations (NFEs) counts only denoising forward passes and ignores verifier overhead, which can distort efficiency rankings. We show that under wall-clock evaluation, simple BoN already matches or outperforms several guided search techniques, suggesting that compute is better spent on broader exploration than on repeated intermediate verification. This motivates Flash-BoN, which generates a large pool of inexpensive draft candidates by combining three complementary acceleration knobs: timestep truncation, layer skipping, and activation proxies into a single configuration optimized once per model. An efficient multi-stage verification procedure then identifies the most promising draft, which is refined at full quality. Across three benchmarks and three model scales, Flash-BoN consistently outperforms all baselines under fixed wall-clock budgets, with gains that grow at larger model scales (+8% AUC). We further show that our strategy combines well and improves existing orthogonal techniques such as reflection-based prompt optimization (+16% AUC). The gains correlate with increased candidate diversity, which also enables draft-guided selection to accelerate RL post-training convergence.

URL PDF HTML 收藏
2607.03926 2026-07-07 cs.DB cs.AI 新提交

TabQueryBench: A Query-Centric Benchmark for Synthetic Tabular Data

TabQueryBench:用于合成表格数据的以查询为中心的基准测试

Jialin Zhang, Fenghao Dong, Yajie Zhou, Vyas Sekar, Shinan Liu

机构 * University of Hong Kong(香港大学) Tongji University(同济大学) Carnegie Mellon University(卡内基梅隆大学) University of Maryland, College Park(马里兰大学学院公园分校)

AI总结 提出TabQueryBench以查询为中心的基准测试,用SQL查询评估合成数据保真度,将查询分类为模板并生成可执行SQL查询,实验展示了当前表格生成模型在查询保真度等方面的五种模式。

详情
AI中文摘要

合成表格数据支持多种用例,现代生成模型按多种指标评估,但少测试分析查询结构。我们提出TabQueryBench,用SQL查询评估合成数据保真度,提供可扩展基础。实验展示了当前表格生成模型在查询保真度等方面的五种模式。

英文摘要

Synthetic tabular data support use cases like data sharing, model development under access restrictions, and rapid prototyping of analytical workflows. Modern generative models are evaluated by their statistical similarity, correlation structure, privacy, and downstream machine-learning utility. However, such evaluations leave a gap: they rarely test the structure that matters for analytical queries. We present TabQueryBench, a query-centric benchmark that uses SQL-shaped analytical queries as structural assessors for synthetic data fidelity. It provides an extensible foundation for query-centric synthetic-data evaluation. From 12 public sources of analytical queries, TabQueryBench taxonomizes recurring cross-domain logic into 44 reusable query templates and grounds them to each dataset via a policy-guided template-to-SQL pipeline. This makes queries schema-aware while preserving comparability across generative models. Across 49 datasets and 11 generative models, it activates 10-12 templates per dataset, producing more than 100 executable SQL queries per dataset. Our systematic experiments show five main patterns. First, current tabular generative models can have good distance-based fidelity, but they still fall short on query-centric fidelity: RealTabFormer achieves the highest query-centric fidelity, but it only reaches 0.75 +/- 0.15 (REAL data score is 1.00). Second, tabular generative models struggle with very high-cardinality discrete support. Third, SOTA generative models preserve good global conditional query-centric fidelity, but fail more on local queries. Fourth, tail fidelity deteriorates as queries move toward the extreme tail; even the best model recovers only about 40.7% of real rare values. Finally, there is a fidelity-cost tradeoff in tabular generation: BayesNet offers the strongest tradeoff, with slightly lower query-centric fidelity but much lower generation cost.

URL PDF HTML 收藏
2607.03723 2026-07-07 cs.RO cs.AI 新提交

OmniTacTune: Policy-Agnostic Real-World RL for Tactile Residual Adaptation of Visual Policies

OmniTacTune:用于视觉策略触觉残余适应的与策略无关的真实世界强化学习

Kelin Yu, Haode Zhang, Harish Ravichandar, Yunhai Han, Ruohan Gao

机构 * University of Maryland, College Park(马里兰大学帕克分校) Georgia Institute of Technology(佐治亚理工学院)

AI总结 研究针对视觉策略在接触丰富操作中不足的问题,提出OmniTacTune管道,通过残余校正让触觉反馈适应预训练视觉策略,经两阶段设计,实验证明其能跨任务等泛化,有效提升策略成功率。

Comments Project page: https://colinyu1.github.io/omnitactune-site/

详情
AI中文摘要

从人类视频、遥操作和机器人演示中学习的视觉策略提供了可扩展的运动先验,但在接触丰富的操作中常失败。触觉传感提供补充信号,但数据收集成本高且难泛化。我们引入OmniTacTune,通过残余校正使触觉反馈适应预训练视觉策略,经两阶段设计,实验表明其能跨多种情况泛化,有效提升成功率。

英文摘要

Visual policies learned from human videos, teleoperation, and robot demonstrations offer scalable motion priors, but often fail in contact-rich manipulation, where success significantly depends on local force and contact geometry. Tactile sensing provides these complementary signals, yet tactile data remain costly to collect and hard to generalize across sensors, robots, and tasks. We introduce OmniTacTune, a policy-agnostic real-world RL pipeline that adapts tactile feedback to pretrained visual policies through residual correction. OmniTacTune uses a two-stage design: it first bootstraps tactile-aware learning from autonomous base-policy rollouts, then learns a lightweight tactile residual policy through online interaction. Extensive experiments show that OmniTacTune generalizes across diverse contact-rich tasks, visual base policies, and tactile representations. Across four real-world contact-rich tasks, it improves visual base policies from 5-40% success to 85-100% within 40-80 minutes, demonstrating an efficient path for adapting tactile feedback to scalable visual robot policies. Project page: https://colinyu1.github.io/omnitactune-site/

URL PDF HTML 收藏
2606.22726 2026-07-07 cs.AI 新提交

Text Dictates, Music Decorates: Energy-based Attention for Editable Dance Motion Generation

文本主导,音乐装饰:基于能量的注意力机制用于可编辑舞蹈动作生成

Seong Jong Yoo, Siyuan Peng, Felix Gu, Stratis Aloimonos, Cornelia Fermüller

机构 * University of Maryland, College Park(马里兰大学帕克分校)

AI总结 提出STREAM模型,通过模态解耦的扩散变压器分离文本语义和音乐节奏控制,实现可编辑舞蹈动作生成,并引入新数据集Motorica++和评估指标EDS。

Comments Accepted to ECCV 2026

详情
AI中文摘要

编舞动作生成对AI提出了独特挑战,要求对复杂、时间结构化和表现性的全身动态进行精确语义控制。虽然现有模型可以从音乐合成动作,但它们大多是黑箱。相反,尝试同时基于文本和音乐进行条件生成常常导致模态崩溃,密集的声学节奏压倒稀疏的语义文本提示,破坏用户可控性。为了解决这种时空冲突,我们提出了STREAM(结构-时间节奏能量注意力用于动作),一种模态解耦的扩散变压器。STREAM严格分离条件路径:全局文本语义通过自适应层归一化(AdaLN)决定运动学结构,而新颖的双模态能量注意力模块(BEAM)将这些特征路由到音乐节拍而不覆盖语义。我们进一步引入了Motorica++,一个新策划的数据集,从现有Motorica数据集中丰富了领域特定的舞蹈词汇和帧级语义注释。此外,为了严格量化零样本可编辑性,我们提出了交换评估协议和可编辑舞蹈分数(EDS)。通过大量实验,STREAM在动作与音乐之间实现了最先进的对齐,同时完美保留了编舞语义,将AI定位为不仅是反应式合成器,而是可控的、协作的艺术指导伙伴。源代码和数据集可在该https URL获取。

英文摘要

Choreographic motion generation poses unique challenges for AI, demanding precise semantic control over complex, temporally structured, and expressive full-body dynamics. While existing models can synthesize motion from music, they remain largely black boxes. Conversely, attempting to condition generation on both text and music frequently leads to modality collapse, where dense acoustic rhythms overwhelm sparse semantic text prompts, destroying user controllability. To resolve this spatial-temporal conflict, we propose STREAM (Structural-Temporal Rhythmic Energy-based Attention for Motion), a modality-decoupled diffusion transformer. STREAM strictly separates conditioning pathways: global text semantics dictate the kinematic structure via Adaptive Layer Normalization (AdaLN), while a novel Bimodal Energy-Based Attention Module (BEAM) routes these features to the musical beat without overwriting the semantics. We further introduce Motorica++, a newly curated dataset enriched with domain-specific dance vocabulary and frame-level semantic annotations from existing Motorica dataset. Additionally, to rigorously quantify zero-shot editability, we propose the Exchange Evaluation Protocol and Editable Dance Score (EDS). Through extensive experiments, STREAM achieves state-of-the-art alignment between motion and music while perfectly preserving choreographic semantics, positioning AI not merely as a reactive synthesizer, but as a controllable, collaborative partner for artistic direction. The source code and datasets are available at https://github.com/SeongJong-Yoo/STREAM.

URL PDF HTML 收藏
2411.14833 2026-07-07 eess.IV cs.CV q-bio.QM 版本更新

Cell as Point: One-Stage Framework for Efficient Cell Tracking

细胞即点:高效细胞追踪的单阶段框架

Yaxuan Song, Jianan Fan, Heng Huang, Mei Chen, Weidong Cai

机构 * School of Computer Science, The University of Sydney(悉尼大学计算机科学学院) Department of Computer Science, University of Maryland College Park(马里兰大学学院公园分校计算机科学系) Microsoft(微软公司)

AI总结 提出细胞追踪单阶段框架CAP,将细胞视为点,利用轨迹相关性联合追踪,消除检测或分割需求。创新采用自适应事件引导采样和滚动窗口推理策略,解决数据不平衡与长序列推理问题,性能优于现有方法。

Comments 34 pages, 8 figures, 8 tables. Accepted by Pattern Recognition. Code is available at https://github.com/YXSong000/CAP

详情
AI中文摘要

传统多阶段细胞追踪方法依赖各帧检测或分割,资源需求大、预测时间长。我们提出CAP,一种将细胞视为点的新型端到端单阶段框架,通过利用轨迹间固有相关性在一个阶段联合追踪细胞序列,减少标注需求和流水线复杂度。CAP引入自适应事件引导采样和滚动窗口推理策略,解决数据不平衡和长序列推理挑战,性能优于现有方法。

英文摘要

Conventional multi-stage cell tracking approaches rely heavily on detection or segmentation in each frame as a prerequisite, requiring substantial resources for high-quality segmentation masks and increasing the overall prediction time. To address these limitations, we propose CAP, a novel end-to-end one-stage framework that reimagines cell tracking by treating Cell as Point. Unlike traditional methods, CAP eliminates the need for explicit detection or segmentation, instead jointly tracking cells for sequences in one stage by leveraging the inherent correlations among their trajectories. This simplification reduces both labeling requirements and pipeline complexity. However, directly processing the entire sequence in one stage poses challenges related to data imbalance in capturing cell division events and long sequence inference. To solve these challenges, CAP introduces two key innovations: (1) adaptive event-guided (AEG) sampling, which prioritizes cell division events to mitigate the occurrence imbalance of cell events, and (2) the rolling-as-window (RAW) inference strategy, which ensures continuous and stable tracking of newly emerging cells over extended sequences. By removing the dependency on segmentation-based preprocessing while addressing the challenges of imbalanced occurrence of cell events and long-sequence tracking, CAP demonstrates promising cell tracking performance and is 8 to 32 times more efficient than existing methods. The code and model checkpoints are available at https://github.com/YXSong000/CAP.

URL PDF HTML 收藏
2607.02387 2026-07-03 cs.IR cs.LG 新提交

Bringing Agentic Search to Earth Observation Data Discovery

将智能搜索引入地球观测数据发现

Minghan Yu, Youran Sun, Chugang Yi, Yixin Wen, Haizhao Yang

机构 * University of Maryland, College Park(马里兰大学学院公园分校)

AI总结 提出一个基于智能搜索的系统,利用知识图谱和大型语言模型,从自然语言查询中返回匹配的地球科学数据集和工具,显著提升检索性能。

Comments 19 pages, 1 figure, 6 tables

详情
AI中文摘要

NASA及其数据中心拥有数千个地球科学数据集和工具,如Worldview、Giovanni、科学发现引擎和Harmony。即使是领域专家也很难找到合适的数据集。我们提出了一个智能搜索系统,作为面向地球科学社区的公共服务部署,该系统接受自然语言研究查询并返回匹配的数据集和工具。我们证明,在大语言模型时代,知识图谱的潜在价值可以通过智能搜索得到显著放大。从NASA地球观测知识图谱中,我们推导出NASA-EO-Bench,一个包含47k个查询-数据集对(21k个基于任务的查询)的开放基准。在NASA-EO-Bench上微调的神经评分器优于余弦和BM25基线。通过分数融合将其与BM25结合,Recall@10和MRR均提高了5倍以上。在此监督流水线之上,我们添加了一个零样本智能重排序阶段,无需额外训练即可在分层N=200子集上将MRR提高28%,表明LLM推理与监督检索互补。

英文摘要

NASA and its data centers hold thousands of geoscience datasets and tools like Worldview, Giovanni, the Science Discovery Engine, and Harmony. Finding the right one is hard even for domain experts. We present an agentic search system, deployed as a public service for the geoscience community, that takes a natural-language research query and returns the matching datasets and tools. We demonstrate that, in the era of large language models, the latent value of knowledge graphs (KGs) can be substantially amplified through agentic search. From the NASA Earth Observation Knowledge Graph (NASA EO-KG) we derive NASA-EO-Bench, an open benchmark of 47k query-dataset pairs (21k task-based queries). A neural scorer fine-tuned on NASA-EO-Bench beats cosine and BM25 baselines. Further combining it with BM25 via score fusion raises both Recall@10 (R@10) and MRR by over 5x. On top of this supervised pipeline, we add a zero-shot agentic reranking stage that, without any additional training, lifts MRR by 28% on a stratified N=200 subset, showing that LLM reasoning is complementary to supervised retrieval.

URL PDF HTML 收藏
2607.01128 2026-07-02 cs.LG cs.NA math.NA 新提交

GAIA: Geometry-Adaptive Operator Learning for Forward and Inverse Problems

GAIA: 面向正问题和反问题的几何自适应算子学习

Meenakshi Krishnan, Pranav Pulijala, Ke Chen, Haizhao Yang, Ramani Duraiswami

机构 * University of Maryland, College Park(马里兰大学学院公园分校) University of Delaware, Newark(特拉华大学新伯伦分校)

AI总结 提出几何自适应积分自编码器(GAIA),通过几何令牌和交叉注意力机制,统一处理任意几何上的正问题和反问题,在七个基准上取得最优结果。

详情
AI中文摘要

针对任意几何上偏微分方程的算子学习,可以构建用于大规模仿真的快速神经代理模型。尽管近期几何自适应神经算子取得了实质性进展,但它们主要设计用于输入和输出共享相同空间域的正问题。这限制了它们在边界值问题(BVP)和反问题中的适用性,因为输入和输出可能位于不同域上。我们引入了几何自适应积分自编码器(GAIA),这是一种算子学习模型,它将域边界和内部场分布编码为几何令牌,并通过交叉注意力将积分变换层条件化于这些令牌,使核能够局部适应几何特征。这产生了一个单一架构,能够一次性处理任意域上的正问题(包括BVP)和反问题,无需重新训练、迭代优化或构建图。我们在七个2D和3D基准上评估了GAIA,其中四个是针对反问题和BVP的新基准或大幅扩展的基准:电阻抗断层扫描、光学断层扫描、变几何上的3D达西流以及机械部件基准(MCB)上泊松BVP的修改设置。GAIA在每个反问题和BVP任务上均创下了新的最先进结果,在翼型流重建中中位数相对$L^2$误差比次优的摊销方法降低了64%,在EIT中降低了27%,并在MCB的每个形状类别上优于所有基线。在其他正问题上,GAIA与专用求解器竞争力相当,同时在基于Transformer的基线退化的点分辨率上保持稳定的精度。

英文摘要

Operator learning for partial differential equations (PDEs) on arbitrary geometries builds fast neural surrogates for large-scale simulation. Although recent geometry-adaptive neural operators have made substantial progress, they are mainly designed for forward problems in which inputs and outputs share the same spatial domain. This limits their applicability for boundary value problems (BVPs) and inverse problems, where inputs and outputs may live on different domains. We introduce the Geometry-Adaptive Integral Autoencoder (GAIA), an operator learning model that encodes the domain boundary and the interior field distribution into geometry tokens, and conditions integral transform layers on these tokens via cross-attention, allowing the kernel to adapt locally to geometric features. This yields a single architecture for forward (including BVPs) and inverse problems on arbitrary domains in one pass, without retraining, iterative optimization, or graph construction. We evaluate GAIA on seven 2D and 3D benchmarks, four of which are new or substantially extended benchmarks for inverse problems and BVP: electrical impedance tomography, optical tomography, 3D Darcy flow on varying geometries, and a modified setting of Poisson BVP on mechanical components benchmark (MCB). GAIA sets new state-of-the-art results on every inverse and BVP task, reducing median relative $L^2$ error by 64% on airfoil flow reconstruction and 27% on EIT relative to the next best amortized method, and outperforming all baselines on every shape category of MCB. On other forward problems, GAIA is competitive with specialized solvers while maintaining stable accuracy across point resolutions on which transformer-based baselines degrade.

URL PDF HTML 收藏
2606.29173 2026-07-01 cs.RO 版本更新

TacGen: Touch Is a Necessary Dimension of Physical-World Representation -- Addressing Tactile Data Scarcity with Scalable Vision-to-Touch Alignment and Generation

TacGen: 触觉是物理世界表征的必要维度——通过可扩展的视觉到触觉对齐与生成解决触觉数据稀缺问题

Wanghao Ye, Aarosh Das, Sihan Chen, Yiting Wang, Bowei Tian, Guoheng Sun, Shwai He, Zheyu Shen, Ziyao Wang, Yexiao He, Zhaoyi Liu, Meng Liu, Yuning Zhang, Meng Feng, Ziyi Wang, Yilong Dai, Yifei Dong, Siyuan Peng, Zhenle Duan, Joshua Liu, Lang Xiong, Ang Li

机构 * University of Maryland, College Park(马里兰大学帕克分校) Carnegie Mellon University(卡内基梅隆大学) University of Alabama(阿拉巴马大学) University of Washington(华盛顿大学) Stanford University(斯坦福大学)

AI总结 针对触觉数据稀缺问题,提出TacGen方法,通过视觉-触觉对比对齐和潜在空间残差MLP生成器合成触觉潜在表示,在质量、密度、硬度等物理属性预测上显著优于纯视觉模型,并大幅提升触觉操作性能。

Comments 49 pages, 29 figures

详情
AI中文摘要

触觉解决了视觉遗留的物理属性模糊性:探索性接触恢复形状、纹理、顺应性和材料,视觉-触觉物体表征在腹侧视觉皮层中收敛。我们探究表征学习能否复现这种基础。TacGen通过结合预设的V+T对比对齐和潜在空间残差MLP V->T生成器(从RGB合成触觉潜在表示用于触觉数据扩展)来缓解触觉数据稀缺瓶颈。使用匹配的DINOv2骨干网络、数据划分和探针,V+T在质量(Delta R^2=+0.570)、密度(Delta acc=+0.067)、硬度(+0.117)和不确定性带力标签(Delta R^2=+0.281)上优于纯V;所有置信区间排除零。相同表征将匹配容量的TACTO操作从0.246提升至0.979,而纯V容量扩展仅占差距的4.5%,保留了95.5%。生成器达到跨种子+0.589,真实触觉在种子区间内为+0.585;架构比较显示重建质量与表征效用之间存在13个百分点的下游差距。在五种子SSVTP/TVL复现、YCB-Sight迁移、三个骨干网络检查、置换/随机特征控制、哈希验证清单和测量力验证检查中,证据支持触觉为接触依赖属性的表征提供了必要的物理证据通道。

英文摘要

Touch resolves the physical-property ambiguity left by vision: exploratory contact recovers shape, texture, compliance, and material, and visuo-haptic object representations converge in ventral visual cortex. We ask whether representation learning can reproduce this grounding. TacGen mitigates the tactile-data scarcity bottleneck by combining pre-specified V+T contrastive alignment with a latent-space residual-MLP V->T generator that synthesizes tactile latents from RGB for tactile-data scaling. With matched DINOv2 backbones, splits, and probes, V+T improves matched V-only on mass (Delta R^2=+0.570), density (Delta acc=+0.067), hardness (+0.117), and uncertainty-banded force labels (Delta R^2=+0.281); all CIs exclude zero. The same representation lifts matched-capacity TACTO manipulation 0.246->0.979 while V-only capacity scaling accounts for only 4.5% of the gap, preserving 95.5%. The generator reaches cross-seed +0.589, with real tactile +0.585 inside the seed interval; the architecture comparison shows a 13pp downstream gap between reconstruction quality and representation utility. Across five-seed SSVTP/TVL reproductions, YCB-Sight transfer, three-backbone checks, permutation/random-feature controls, hash-verified manifests, and measured-force validation checks, the evidence supports the claim that touch supplies a necessary physical evidence channel for representations of contact-dependent properties.

URL PDF HTML 收藏
2606.31068 2026-07-01 cs.CV 新提交

Hybrid Unet-Transformer Model for Generating Stress and Strain Fields from Composite Geometrics

混合UNet-Transformer模型用于从复合材料几何生成应力应变场

Shrey Patel

机构 * University of Maryland-College Park(马里兰大学帕克分校)

AI总结 提出混合UNet-Transformer架构,直接从复合材料微观结构图像预测应力应变场,作为有限元法的替代,在多种周期性几何上达到高精度,并通过注意力分析验证了物理可解释性。

Comments International Conference on Emerging Digital Intelligence and Generative Engineering

详情
AI中文摘要

准确预测层级复合材料微观结构中的应力应变场对于基于物理的材料设计至关重要,然而传统的有限元法(FEM)模拟在规模上计算成本过高,每次评估需要数分钟到数天。在这项工作中,我们提出了一种混合UNet-Transformer架构,该架构直接从复合材料微观结构几何图像预测复杂的力学场分布,作为FEM的高效替代,涵盖十种不同的应力应变场类型,跨越多种两相复合材料构型,包括方形、六边形和三角形镶嵌、多种边界条件和高分辨率几何。结果表明,所提出的架构在大多数子数据集上实现了强大的预测性能,在周期性镶嵌几何上的峰值精度达到:边界条件子数据集上R²=0.9991,SSIM=0.9936,三角形镶嵌子数据集上MAE=0.0050。在八个评估子数据集中的六个上,归一化[0,1]像素尺度的MAE保持在0.05以下。通过Grad-CAM和Grad-CAM++进行的编码器注意力分析证实,模型发展了物理上有意义的内部表示,将注意力定位在力学关键区域,包括相界、韧带连接点和压头接触区,而无需显式的结构监督。在不规则方形网格几何且稀疏软相夹杂的情况下,性能下降,S11法向应力子数据集的R²=0.7735,SSIM=0.7126,这与平滑损失图像翻译模型在再现尖锐应力不连续性方面的已知局限性一致。

英文摘要

Accurate prediction of stress and strain fields in hierarchical composite microstructures is critical for physics-informed material design, yet conventional finite element method (FEM) simulations are computationally prohibitive at scale, requiring minutes to days per evaluation. In this work, we propose a hybrid UNet-Transformer architecture that predicts complex mechanical field distributions directly from composite microstructure geometry images, serving as an efficient surrogate for FEM across ten distinct stress and strain field types spanning diverse two-phase composite configurations including square, hexagonal, and triangular tessellations, multiple boundary conditions, and high-resolution geometries. Results demonstrate that the proposed architecture achieves strong predictive performance across the majority of subdatasets, with peak accuracy on periodic tessellation geometries reaching R2=0.9991, SSIM=0.9936, and MAE=0.0050 on the boundary condition subdataset and the triangular tessellation subdataset respectively. Across six of the eight evaluated subdatasets, MAE remains below 0.05 on the normalized [0,1] pixel scale. Encoder attention analysis via Grad-CAM and Grad-CAM++ confirms that the model develops physically meaningful internal representations, localizing attention at mechanically critical regions including phase boundaries, ligament junctions, and indenter contact zones without explicit structural supervision. Performance degrades on irregular square-grid geometries with sparse soft-phase inclusions, with the S11 normal stress subdataset yielding R2=0.7735 and SSIM=0.7126, consistent with the known limitation of smooth-loss image translation models in reproducing sharp stress discontinuities.

URL PDF HTML 收藏
2606.30595 2026-06-30 cs.NI cs.CR cs.IT cs.LG eess.SP math.IT

Wireless Backdoor Attack and Defense for Semantic Communications over Multiple Access Channel

无线后门攻击与防御:面向多址接入信道的语义通信

Yalin E. Sagduyu, Tugba Erpek, Aylin Yener, Sennur Ulukus

机构 * Nexcepta The Ohio State University(俄亥俄州立大学) University of Maryland, College Park(马里兰大学学院公园分校)

AI总结 针对多址接入信道的语义通信系统,提出一种选择性无线后门攻击,通过低功率触发波形操纵单一发射机的语义推理,并开发触发感知防御机制以保持正确语义标签。

详情
AI中文摘要

语义通信旨在超越传统无线信道上的消息恢复,保留语义含义和任务导向信息。在共享接入无线网络中采用语义通信为多用户语义推理引入了新的漏洞。本文考虑一个语义通信系统,其中两个发射机通过多址接入信道与一个公共接收机通信。每个发射机将源信息映射为潜在语义表示,而接收机联合重建并分类两个发射机的语义信息。提出一种选择性空中后门(特洛伊)攻击,其中攻击者在训练期间发送低功率触发波形,并将其注入共享接收信号。通过在测试期间再次发送触发,这种隐蔽的低功率攻击选择性操纵一个发射机的语义推理,而对另一个发射机的推理影响最小。为缓解此漏洞,开发了一种触发感知防御机制,以在触发污染的无线观测下保持正确的语义标签。结果证明了共享接入语义通信系统对选择性空中后门攻击的脆弱性,以及触发感知鲁棒训练对语义保护的有效性。

英文摘要

Semantic communication (SemCom) aims to preserve semantic meaning and task-oriented information beyond conventional message recovery over wireless channels. The adoption of SemCom in shared-access wireless networks introduces new vulnerabilities for multi-user semantic inference. This paper considers a SemCom system for two transmitters communicating with a common receiver over a multiple access channel. Each transmitter maps source information into latent semantic representations, while the receiver jointly reconstructs and classifies the semantic information for both transmitters. A selective over-the-air backdoor (Trojan) attack is presented in which an adversary transmits a low-power trigger waveform over the air and injects it into the shared received signal during training. By transmitting the trigger again during testing, this stealthy, low-power attack selectively manipulates the semantic inference for one transmitter while minimally affecting the inference of the other transmitter. To mitigate this vulnerability, a trigger-aware defense mechanism is developed to preserve correct semantic labels under trigger-contaminated wireless observations. The results demonstrate both the vulnerability of shared-access SemCom systems to selective over-the-air backdoor attacks and the effectiveness of trigger-aware robust training for semantic protection.

URL PDF HTML 收藏
2606.29823 2026-06-30 cs.DB cs.AI cs.MA

Experience Graphs: The Data Foundation for Self-Improving Agents

经验图:自我改进智能体的数据基础

Gang Liao, Yujia He, Abdullah Ozturk, Zhouyang Li, Ying Wang, Zhitong Guo, Hongsen Qin, Yaobin Qin, Tao Yang, Zewei Jiang, Dianshi Li, Jort Gemmeke, Jiangyuan Li, Liyuan Li, Nathan Yan, Masha Basmanova, Uladzimir Pashkevich, Matt Steiner, Pedro Pedreira, Rob Fergus, Anirudh Goyal, Carole-Jean Wu, Gaoxiang Liu, Andrew Witten, Daniel J. Abadi

机构 * Meta Platforms(Meta平台) University of Maryland, College Park(马里兰大学学院公园分校)

AI总结 提出Trellis系统,将智能体探索过程产生的经验图作为一等数据库状态,通过查询、重用和物化视图支持跨会话复用和训练数据提取,在Meta的KernelEvolve中实现10倍加速和52%更低token成本。

详情
AI中文摘要

数据库社区通过认识到新工作负载需要新系统架构,不断推动技术前沿。我们认为,长周期智能体任务——代码生成、科学发现、硬件设计——正是这样的工作负载。这些智能体进行探索:它们生成工件、执行工具、观察失败、分支并在数百步中修复。这种搜索产生了一个结构化对象,我们称之为经验图:可执行工件、工具输出、奖励、兄弟比较和因果谱系。然而,现有的智能体框架将这种经验视为可丢弃的状态——JSON检查点和会话日志,在崩溃后无法恢复,无法跨用户查询,也无法物化为训练数据。我们提出Trellis:一个将经验图视为一等公民、可管理、可查询的数据库状态的数据基础。核心洞察是,对经验图的搜索是一种数据库访问模式。前沿选择是一种查询,跨会话重用是向量种子图检索,训练数据提取是物化视图,而重建智能体在过去任何步骤所知道的内容是时间旅行查询。当数据库拥有经验图时,智能体变成无状态计算,崩溃恢复、水平扩展和闭环训练飞轮成为架构的副产品。我们在Meta的生产加速器内核优化器KernelEvolve中验证了该设计,其中跨会话重用达到了大约10倍的目标加速,同时token成本降低52%。更广泛地说,Trellis将推理时搜索从可丢弃的计算转变为持久的机构资产:日志使数据库可靠;经验图可能使智能体具有累积性。

英文摘要

The database community has repeatedly advanced the state of the art by recognizing that new workloads demand new system architectures. We argue that long-horizon agentic tasks -- code generation, scientific discovery, hardware design -- are such a workload. These agents explore: they generate artifacts, execute tools, observe failures, branch, and repair over hundreds of steps. This search produces a structured object we call an experience graph: executable artifacts, tool outputs, rewards, sibling comparisons, and causal lineage. Yet existing agent frameworks treat this experience as disposable state -- JSON checkpoints and session logs that cannot be recovered after a crash, queried across users, or materialized into training data. We propose Trellis: a data foundation that treats the experience graph as first-class, governed, queryable database state. The core insight is that search over experience graphs is a database access pattern. Frontier selection is a query, cross-session reuse is vector-seeded graph retrieval, training-data extraction is a materialized view, and reconstructing what an agent knew at any past step is a time-travel query. When the database owns the experience graph, agents become stateless compute, and crash recovery, horizontal scaling, and a closed-loop training flywheel emerge as architectural byproducts. We ground the design in KernelEvolve, a production accelerator-kernel optimizer at Meta, where cross-session reuse reaches a target speedup roughly 10x faster at 52% lower token cost. More broadly, Trellis turns inference-time search from disposable computation into a durable institutional asset: logs made databases reliable; experience graphs may make agents cumulative.

URL PDF HTML 收藏
2606.28627 2026-06-30 eess.SY cs.RO cs.SY math.OC

Reachability Guarantees for Cart-Pole Swing-Up and Stabilization

小车-摆杆起摆与稳定的可达性保证

Mohamed Khalid M Jaffar

机构 * Department of Aerospace Engineering(航空航天工程系) University of Maryland, College Park(马里兰大学学院公园分校)

AI总结 针对欠驱动小车-摆杆系统,提出基于能量/LQR切换控制的可达性分析,通过Lyapunov函数和LaSalle不变性原理证明从紧致初始集到竖直平衡的收敛,并设计切换区域保证起摆到稳定的形式化过渡。

详情
AI中文摘要

小车-摆杆起摆是欠驱动系统非线性控制的一个经典基准问题,然而将全局起摆操作与局部稳定器联系起来的端到端保证很少被形式化。我们提出了一种切换能量/LQR控制器的可达性分析,该分析证明了从紧致初始条件集到竖直平衡的收敛。起摆律由能量误差Lyapunov函数导出;消除自治保守项产生严格符号确定的Lyapunov导数,并且收敛性由LaSalle不变性原理得出。我们还提出了一种增广Lyapunov函数来调节稳态小车速度为零,对此我们建立了几乎全局收敛性。对于控制器切换,设计了一个严格位于LQR吸引域内的切换区域,形式化地保证了起摆到稳定的过渡。数值模拟证实了理论分析。

英文摘要

The cart-pole swing-up is a canonical benchmark for nonlinear control of underactuated systems, yet an end-to-end guarantee linking the global swing-up maneuver to the local stabilizer is seldom formalized. We present a reachability analysis of a switched energy-based/LQR controller that certifies convergence to the upright equilibrium from a compact set of initial conditions. The swing-up law is derived from an energy-error Lyapunov function; canceling the autonomous conservative term yields a strictly sign-definite Lyapunov derivative, and convergence follows from LaSalle's invariance principle. We also propose an augmented Lyapunov function to regulate the steady-state cart velocity to zero, for which we establish almost-global convergence. For the controller handoff, a switching region is designed to lie strictly within the LQR region of attraction, formally certifying the swing-up-to-stabilization transition. Numerical simulations corroborate the theoretical analysis.

URL PDF HTML 收藏
2606.28531 2026-06-30 cs.MM cs.CL

A Good Talk Does not Look Like a Summary, It Teaches You! Measuring Takeaways from Paper-to-Video Talks

好的演讲不像摘要,它教你!衡量论文到视频演讲的收获

Ishani Mondal, Aparna Garimella, Ananya Sai, Pannaga Shivaswamy, Jordan Boyd-Graber

机构 * University of Maryland, College Park(马里兰大学学院公园分校) Adobe Research(Adobe研究)

AI总结 提出EffectivePresentationScorer框架,通过评估解释清晰度、背景概念引入和细节与贡献的关联,衡量论文讲解视频的教学质量,发现现有视频生成系统缺乏解释性。

Comments Under Submission

详情
AI中文摘要

自动从科学论文生成的视频越来越多地用于教育和研究传播。然而,现有的评估指标主要衡量视觉质量或论文中的关键点是否出现在视频中,而不评估视频是否真正帮助观众理解思想。我们引入了EffectivePresentationScorer,一个评估科学演讲视频教学质量的框架。它检查视频是否清晰地解释了主要思想,引入了必要的背景概念,并将技术细节与论文的主要贡献联系起来。当我们将EffectivePresentationScorer应用于现有的论文到视频生成系统时,我们发现生成的视频提到了正确的主题并遵循论文的结构,但未能解释先决概念或阐明方法为何有效。这些失败常常被现有的视频评估指标忽略,这些指标关注内容存在性而非解释质量。

英文摘要

Automatically generated videos from scientific papers are increasingly used for education and research dissemination. However, existing evaluation metrics mainly measure visual quality or whether key points from the paper appear in the video without assessing whether the video actually helps viewers understand the ideas. We introduce EffectivePresentationScorer, a framework for evaluating the instructional quality of scientific presentation videos. It checks whether a video explains the main ideas clearly, introduces needed background concepts, and connects technical details to the main contribution of the paper. When we apply EffectivePresentationScorer to the existing paper-to-video generation systems, we find that generated videos mention the correct topics and follow the structure of the paper but fail to explain prerequisite concepts or clarify why the method works. These failures are often ignored by existing video evaluation metrics, which focus on content presence rather than explanatory quality.

URL PDF HTML 收藏
2606.27755 2026-06-29 cs.RO cs.AI 新提交

Drop-Then-Recovery: How Redundant Are Vision-Language-Action Models?

丢弃-再恢复:视觉-语言-动作模型有多冗余?

Guoheng Sun, Kaixi Feng, Shwai He, Xiaochuan Gong, Yexiao He, Ziyao Wang, Zheyu Shen, Wanghao Ye, Ramana Rao Kompella, Gaowen Liu, Ang Li

机构 * University of Maryland, College Park(马里兰大学帕克分校) Cisco Research(思科研究院)

AI总结 通过提出Drop-Then-Recovery分析协议和GateProbe敏感性指标,发现VLA模型中语言骨干高度冗余,而视觉和动作路径对移除更敏感。

详情
AI中文摘要

视觉-语言-动作(VLA)模型实现了指令驱动的机器人操作,但它们继承了来自预训练VLM的过大的语言骨干,其容量远超短机器人指令所需。这引出了一个基本问题:VLA模型中有多少部分对于闭环控制是真正必要的?在这项工作中,我们通过使用Transformer块移除作为受控干预,研究VLA模型中的架构冗余。我们引入了\textbf{Drop-Then-Recovery (DTR)},一种分析协议,它从预训练的VLA模型中移除选定的块,然后微调得到的模型,以测量移除的容量对于下游控制是否必要。为了使这种干预可靠,我们提出了\textbf{GateProbe},一种一次性虚拟门敏感性度量,根据块对下游动作损失的贡献对其进行排序。在多种VLA架构、操作基准甚至真实机器人工业场景中,我们发现移除后的可恢复性存在强烈的不对称性:\ul{\textit{语言骨干对于标准机器人操作任务高度冗余,而视觉和动作路径对移除的容忍度显著较低}}。在LIBERO上,移除一半的LLM块甚至在下游微调预算相同的情况下将OpenVLA-OFT从95.0%提升到98.3%,仅保留两个语言块仍能恢复基线性能。这些结果表明,当前的VLA基准可能对深度语言基础和组合指令理解施加了有限压力,未来的VLA架构应在语言、视觉和动作组件之间更谨慎地分配容量。代码可在该网址获取。

英文摘要

Vision-Language-Action (VLA) models enable instruction-driven robotic manipulation, but they inherit oversized language backbones from pretrained VLMs whose capacity far exceeds what is needed for short robotic instructions. This raises a basic question: how much of a VLA model is actually necessary for closed-loop control? In this work, we study architectural redundancy in VLA models by using transformer block removal as a controlled intervention. We introduce \textbf{Drop-Then-Recovery (DTR)}, an analysis protocol that removes selected blocks from a pretrained VLA model and then fine-tunes the resulting model to measure whether the removed capacity was necessary for downstream control. To make this intervention reliable, we propose \textbf{GateProbe}, a one-shot virtual-gate sensitivity metric that ranks blocks by their contribution to the downstream action loss. Across multiple VLA architectures, manipulation benchmarks and even real-robot industrial scenarios, we find a strong asymmetry in post-removal recoverability: \ul{\textit{language backbones are highly redundant for standard robotic manipulation tasks, whereas vision and action pathways are substantially less tolerant to removal}}. On LIBERO, removing half of the LLM blocks even improves OpenVLA-OFT from 95.0% to 98.3% under the same downstream fine-tuning budget, and retaining only two language blocks still recovers baseline-level performance. These results suggest that current VLA benchmarks may exert limited pressure on deep language grounding and compositional instruction understanding, and that future VLA architectures should allocate capacity more deliberately across language, vision, and action components. The code is available at https://github.com/s1ghhh/VLADrop.

URL PDF HTML 收藏
2606.27355 2026-06-26 cs.RO 新提交

RouterVLA: Turning Smoke Tests into Supervision for Heterogeneous VLA Selection

RouterVLA:将冒烟测试转化为异构VLA选择的监督信号

Xingyu Ren, Chugang Yi, Ge Ma, Youran Sun

机构 * The Chinese University of Hong Kong(香港中文大学) University of Maryland, College Park(马里兰大学 College Park 分校) Tsinghua University(清华大学)

AI总结 提出RouterVLA方法,利用预部署评估的冒烟测试记录来监督异构VLA策略的选择,通过结果分离的交叉拟合和透明规则,在LIBERO-Plus数据集上将保留集成功率提升14.64个百分点。

详情
AI中文摘要

我们研究是否可以将预部署评估的 rollout 重复用于监督策略选择。机器人团队通常会冒烟测试候选的视觉-语言-动作(VLA)策略,然后将这些试验压缩为一个全局优胜者。RouterVLA 通过结果分离的交叉拟合来评估这一想法:记录的探测为每个冻结的专家构建一个配置文件,而单独的试验对所选专家进行评分,且不将其结果纳入配置文件。在 34,752 条 LIBERO-Plus rollout 记录中,一个透明的探测成功规则将保留集成功率从 0.4686 提升至 0.6149,提升了 +14.64 个百分点。在本文研究的仅标量配置文件中,学习得到的评分器与这一规则在统计上无法区分,这表明委托(commissioning)承载了路由价值,而额外的标量评分器容量并未创造该价值。重复使用已评分的试验会将测量得到的提升放大 $1.87\ imes$,因此可信的分类账路由需要结果分离;模型扩展改进了单个策略,而委托感知的路由则改进了由这些策略构建的系统。

英文摘要

We study whether pre-deployment evaluation rollouts can be reused to supervise policy selection. Robot teams routinely smoke test candidate vision-language-action (VLA) policies, then compress those trials into a global winner. RouterVLA evaluates this idea with outcome-disjoint cross-fitting: recorded probes build a profile for each frozen expert, and a separate trial scores the selected expert without entering its profile. Across 34,752 LIBERO-Plus rollout records, a transparent probe-success rule raises held-out success from 0.4686 to 0.6149, a +14.64pp gain. Under the scalar-only profiles studied here, learned scorers are statistically indistinguishable from this rule, showing that commissioning carries the routing value while extra scalar scorer capacity does not create it. Reusing the scored trial inflates the measured gain by $1.87\times$, so credible ledger routing needs outcome separation; model scaling improves individual policies, while commissioning-aware routing improves the system built from them.

URL PDF HTML 收藏