arXivDaily arXiv每日学术速递 周一至周五更新
全部学科分类 3405
专题追踪
2605.18172 2026-05-26 cs.AI

Visualizing the Invisible: Generative Visual Grounding Empowers Universal EEG Understanding in MLLMs

可视化不可见:生成式视觉定位赋能多模态大语言模型的通用脑电图理解

Jun-Yu Pan, Yansen Wang, Enze Zhang, Bao-Liang Lu, Wei-Long Zheng, Dongsheng Li

发表机构 * Shanghai Jiao Tong University(上海交通大学) Microsoft Research Asia(微软亚洲研究院)

AI总结 提出生成式视觉定位(GVG)框架,通过脑电图到图像的生成模型作为视觉翻译器,为多模态大语言模型提供结构化视觉上下文,以增强非视觉脑电图的理解和临床状态解释。

详情
AI中文摘要

利用预训练大语言模型和多模态大语言模型的通用表示为脑基础模型提供了一条有前景的路径。然而,视觉诱发的脑电图数据集仍然稀缺,导致现有方法主要将神经信号与抽象文本对齐,这种有损翻译可能丢弃脑活动中编码的细粒度感知信息。我们提出生成式视觉定位(GVG)框架,通过使用脑电图到图像的生成模型作为视觉翻译器,将不可见的信息可视化。GVG 不是仅将脑电图强制转换为文本,而是为非视觉脑电图生成实例特定的代理图像,提供结构化的视觉上下文,使多模态大语言模型能够利用其视觉先验进行临床状态解释。我们在两个多模态大语言模型骨干上验证了这一想法:GVG-X-Omni 和 GVG-Janus。仅图像对齐已具有竞争力:轻量级 GVG-X-Omni 在冻结的 7B 骨干上仅调整 170M 参数,即可匹配 1.7B 参数的文本对齐基线。我们进一步扩展了 GVG-Janus,采用三模态图像+文本对齐,其中文本提供类别语义锚点,视觉代理用感知细节丰富神经表示。实验表明,在脑电图理解和视觉生成方面均取得了一致增益,表明视觉代理定位作为文本对齐的有效补充。

英文摘要

Leveraging the universal representations of pre-trained LLMs and MLLMs offers a promising path toward brain foundation models. However, visually-evoked EEG datasets remain scarce, leading existing methods to align neural signals mainly with abstract text, a lossy translation that may discard fine-grained perceptual information encoded in brain activity. We propose Generative Visual Grounding (GVG), a framework that visualizes the invisible by using an EEG-to-image generative model as a visual translator. Instead of forcing EEG into text alone, GVG hallucinates instance-specific proxy images for non-visual EEG, providing structured visual contexts that allow MLLMs to exploit their visual priors for clinical-state interpretation. We validate this idea on two MLLM backbones, GVG-X-Omni and GVG-Janus. Image-only alignment is already competitive: the lightweight GVG-X-Omni matches 1.7B-parameter text-aligned baselines while tuning only 170M parameters on a frozen 7B backbone. We further extend GVG-Janus with trimodal Image+Text alignment, where text supplies categorical semantic anchors and visual proxies enrich neural representations with perceptual details. Experiments show consistent gains in EEG understanding and visual generation, suggesting visual proxy grounding as an effective complement to textual alignment.

2605.17937 2026-05-26 cs.CL cs.AI

BacktestBench: Benchmarking Large Language Models for Automated Quantitative Strategy Backtesting

BacktestBench:面向自动化量化策略回测的大语言模型基准测试

Zhensheng Wang, Wenmian Yang, Qingtai Wu, Lequan Ma, Yiquan Zhang, Weijia Jia

发表机构 * Beijing Normal University(北京师范大学) Elmleaf Ltd.(Elmleaf公司)

AI总结 提出首个大规模自动化量化回测基准BacktestBench,包含18,246个问答对,并设计多智能体基线AutoBacktest,通过协调摘要器、检索器和编码器实现自然语言策略到可重复回测的转换。

Comments This paper has been accepted by KDD 2026 (Datasets and Benchmarks Track)

详情
AI中文摘要

量化回测对于评估交易策略至关重要,但仍受到高技术门槛和有限可扩展性的阻碍。虽然大语言模型(LLMs)通过先进的代码生成、工具使用和智能体规划为自动化这一复杂的跨学科工作流程提供了变革性路径,但实际实现因当前缺乏专门用于自动化量化回测的大规模基准而面临重大挑战,这阻碍了该领域的进展。为弥补这一关键差距,我们引入了BacktestBench,这是首个用于自动化量化回测的大规模基准。它基于超过600万条真实市场记录构建,包含18,246个精心标注的问答对,涵盖四个任务类别:指标计算、股票选择、策略选择和参数确认。我们还提出了AutoBacktest,一个稳健的多智能体基线,通过协调摘要器进行语义因子提取、检索器进行验证的SQL生成以及编码器进行Python回测实现,将自然语言策略转化为可重复的回测。我们对23个主流LLM的评估,辅以有针对性的消融实验,识别了影响端到端性能的关键因素,并强调了基于事实的验证和标准化指标表示的重要性。

英文摘要

Quantitative backtesting is essential for evaluating trading strategies but remains hampered by high technical barriers and limited scalability. While Large Language Models (LLMs) offer a transformative path to automate this complex, interdisciplinary workflow through advanced code generation, tool usage, and agentic planning, the practical realization is significantly challenged by the current lack of a large-scale benchmark dedicated to automated quantitative backtesting, which hinders progress in this field. To bridge this critical gap, we introduce BacktestBench, the first large-scale benchmark for automated quantitative backtesting. Built from over 6 million real market records, it comprises 18,246 meticulously annotated question-answering pairs across four task categories: metrics calculation, ticker selection, strategy selection, and parameter confirmation. We also propose AutoBacktest, a robust multi-agent baseline that translates natural language strategies into reproducible backtests by coordinating a Summarizer for semantic factor extraction, a Retriever for validated SQL generation, and a Coder for Python backtesting implementation. Our evaluation on 23 mainstream LLMs, complemented by targeted ablations, identifies key factors that influence end-to-end performance and highlights the importance of grounded verification and standardized indicator representations.

2605.17730 2026-05-26 cs.LG cs.AI

L-Drive: Beyond a Single Mapping-Latent Context Drives Time Series Forecasting

L-Drive:超越单一映射——潜在上下文驱动时间序列预测

Fan Zhang, Shijun Chen, Hua Wang

发表机构 * Business University, Yantai, Shandong, China(山东商业大学) Ludong University, Yantai, Shandong, China(鲁东大学)

AI总结 针对分布偏移和机制变化导致直接映射范式在转折点响应滞后的问题,提出L-Drive框架,通过引入潜在上下文表征高层动态并利用门控调制增量表示,提升对变化段的适应能力,同时采用补丁共享相对位置基函数增强段内结构建模,实现预测精度与计算效率的更好平衡。

详情
AI中文摘要

多变量时间序列预测的主流方法主要遵循直接映射范式。它们在观测空间中学习从历史到未来的统一映射,以拟合值级依赖关系。然而,现实世界系统经常经历分布偏移和机制变化。在这种情况下,统一映射在转折点附近可能出现响应滞后,导致切换窗口内误差累积,降低预测可靠性。为解决此问题,我们提出L-Drive,一种变化感知预测框架。L-Drive引入潜在上下文,显式表征随时间演变的高层动态,并使用门控调制增量表示。这提供了更及时的变化线索,并改善了对变化段的适应。此外,它结合了补丁共享相对位置基函数,以加强段内结构建模并减少由绝对位置记忆引起的过拟合。大量实验验证了L-Drive的有效性,并展示了其在预测精度和计算效率之间更好的整体权衡。

英文摘要

Mainstream methods for multivariate time-series forecasting largely follow the Direct-Mapping paradigm. They learn a unified mapping from history to the future in the observation space to fit value-level dependencies. However, real-world systems often undergo distribution shifts and regime changes. In such cases, a unified mapping can exhibit response lag around turning points, causing error accumulation within the switching window and reducing forecasting reliability. To address this issue, we propose L-Drive, a change-aware forecasting framework. L-Drive introduces a Latent-Context, to explicitly characterize high-level dynamics evolving over time, and uses gating to modulate increment representations. This provides more timely change cues and improves adaptation to changing segments. In addition, it incorporates patch-shared relative positional basis functions to strengthen intra-segment structural modeling and reduce overfitting caused by absolute-position memorization. Extensive experiments validate the effectiveness of L-Drive and show a better overall trade-off between forecasting accuracy and computational efficiency.

2605.17537 2026-05-26 cs.AI

Self-supervised Hierarchical Visual Reasoning with World Model

基于世界模型的自监督分层视觉推理

Yuanfei Xu, Lin Liu, Wengang Zhou, Mingxiao Feng, Houqiang Li

发表机构 * Department of Electronic Engineering and Information Science, University of Science and Technology of China(电子工程与信息科学系,中国科学技术大学) Institute of Artificial Intelligence, Hefei Comprehensive National Science Center(人工智能研究院,合肥综合性国家科学中心)

AI总结 提出ResDreamer,一种分层世界模型,通过自监督方式学习残差表示,实现高效视觉推理,在3D开放环境中达到最先进的样本和参数效率。

详情
AI中文摘要

具有对抗对手的3D开放世界环境因其巨大的状态空间仍然是强化学习的核心挑战。有效的推理表示在此类环境中至关重要。虽然现有的自监督视觉预见推理方法常常遭受多步误差累积,许多最近的研究转向注入领域特定知识以提供更稳定的指导。我们的关键洞察是,视觉推理表示的照片级真实感是次要的;真正重要的是提供信息丰富、任务相关的信号。为此,我们提出ResDreamer,一种分层世界模型,其中每个更高层被训练来重建下一层的残差。这种设计使得对日益复杂的世界动态进行渐进抽象成为可能,并促进更丰富潜在表示的出现。受“苦涩教训”启发,ResDreamer以纯自监督方式训练其推理表示。高层残差表示用于调节低层预测,使得世界模型仅以线性增加的跨层通信成本即可有效扩展。实验表明,ResDreamer实现了最先进的样本效率和参数效率。这种可扩展的分层视觉预见推理架构为开放、动态环境中更具能力的在线RL代理铺平了道路。代码可在https://github.com/XuYuanFei01/ResDreamer获取。

英文摘要

3D open-world environments with adversarial opponents remain a core challenge for reinforcement learning due to their vast state spaces. Effective reasoning representations are essential in such settings. While existing self-supervised visual foresight reasoning approaches often suffer from multi-step error accumulation, many recent studies resort to injecting domain-specific knowledge for more stable guidance. Our key insight is that the photorealistic fidelity of visual reasoning representations is secondary; what truly matters is providing informative, task-relevant signals. To this end, we propose ResDreamer, a hierarchical world model in which each higher-level layer is trained to reconstruct the residuals of the layer below. This design enables progressive abstraction of increasingly sophisticated world dynamics and fosters the emergence of richer latent representations. Drawing inspiration from the "Bitter Lesson", ResDreamer trains its reasoning representations in a purely self-supervised manner. The higher-level residual representations are used to modulate lower-level predictions, allowing the world model to scale effectively with only linearly increasing cross-layer communication costs. Experiments show that ResDreamer achieves state-of-the-art sample efficiency and parameter efficiency. This scalable hierarchical visual foresight reasoning architecture paves the way for more capable online RL agents in open-ended, dynamic environments. The code is accessible at https://github.com/XuYuanFei01/ResDreamer.

2605.17287 2026-05-26 cs.CV

LISA: Language-guided Interference-aware Spatial-Frequency Attention for Driver Gaze Estimation

LISA: 语言引导的干扰感知空间-频率注意力用于驾驶员视线估计

Jun Ma, Zhenye Yang, Ruichen Zhou, Pei Zhang, Huan Li, Jinpeng Chen

发表机构 * School of Computer Science (National Pilot Software Engineering School), Beijing University of Posts and Telecommunications(北京邮电大学计算机科学学院(国家级试点软件工程学院)) Key Laboratory of Trustworthy Distributed Computing and Service (BUPT), Ministry of Education(教育部可信分布式计算与服务重点实验室(BUPT)) School of Electrical Engineering, Guangxi University(广西大学电气工程学院) Zhejiang University(浙江大学)

AI总结 提出LISA框架,结合频域先验与视觉语言知识,通过双域融合机制和训练时解耦策略,实现鲁棒的驾驶员视线估计,在遮挡和光照变化下达到最优性能。

Comments 9 pages, 5 figures, 3 tables

详情
AI中文摘要

驾驶员视线估计是现代监控系统中评估驾驶员注意力的一项基本指标。除了易受突然光照变化和传感器噪声影响外,空间域模型难以将真实的视线线索与无关的视觉属性分离。在本文中,我们提出了LISA,一个语言引导的干扰感知空间-频率注意力框架,结合了频域先验与视觉语言知识。观察到即使在空间扰动下幅度谱仍保持相对稳定,我们设计了一种双域融合机制。它将稳定的低频语义集成到高频细节中,利用空间注意力精确定位眼部区域。为减少语义模糊性,我们还引入了一种训练时解耦策略。使用冻结的CLIP编码器和正交正则化,我们将视线特征与外观干扰明确分离。在两个基准上的实验表明,LISA达到了最先进的性能,在遮挡和光照变化下具有显著增强的鲁棒性。代码仓库可在 https://github.com/Mason-bupt/LISA 获取。

英文摘要

Driver gaze estimation serves as a fundamental metric for evaluating driver attentiveness in modern monitoring systems. Beyond being vulnerable to sudden lighting changes and sensor noise, spatial-domain models struggle to disentangle authentic gaze cues from irrelevant visual attributes. In this paper, we propose LISA, a \textbf{L}anguage-guided \textbf{I}nterference-aware \textbf{S}patial-Frequency \textbf{A}ttention framework that combines frequency-domain priors with vision-language knowledge. Observing that the amplitude spectrum remains relatively stable even under spatial perturbations, we design a dual-domain fusion mechanism. It integrates stable low-frequency semantics into high-frequency details, employing spatial attention to precisely target ocular regions. To reduce semantic ambiguity, we also introduce a training-time disentanglement strategy. Using a frozen CLIP encoder and orthogonal regularization, we explicitly separate gaze features from appearance interference. Experiments on two benchmarks show that LISA achieves state-of-the-art performance, with significantly improved robustness against occlusions and lighting variations. The code repository is available at https://github.com/Mason-bupt/LISA.

2605.17234 2026-05-26 cs.LG

Active Budget Allocation for Efficient Scaling Law Estimation via Surrogate-Guided Pruning

通过代理引导剪枝实现高效缩放定律估计的主动预算分配

Viktoria Schram, Markus Hiller, Daniel Beck, Trevor Cohn

发表机构 * The School of Computing and Information Systems, the University of Melbourne, Melbourne, Australia(墨尔本大学计算与信息系统学院) Royal Melbourne Institute of Technology, Melbourne, Australia(皇家墨尔本理工学院) Now at Google Australia(现就职于谷歌澳大利亚)

AI总结 提出结合连续减半与参数/非参数代理模型的主动预算分配方法,在显著降低计算成本(节省高达98.7%)的同时获得更优的损失-计算量前沿,实现精确的缩放定律估计。

Comments Accepted at ICML 2026

详情
AI中文摘要

预测更大规模下的模型性能能够设计针对特定性能目标的训练策略和架构。经验缩放定律研究通过识别函数形式来辅助这一预测任务,这些函数形式利用学习曲线定义的损失-计算量前沿描述损失与计算量之间的关系。由于该方法的经验性质,计算负担巨大,使得战略资源分配至关重要——然而这一领域却出人意料地未被充分探索。在本工作中,我们通过探索连续减半(SH)以及SH与参数化和非参数化代理模型结合的适用性来弥补这一不足。除了能够更系统地分配给定的计算预算外,我们的发现表明,SH与代理模型结合得到的一组学习曲线中,包含一条损失-计算量值低于朴素均匀分配或仅SH方法所能获得的曲线。我们的实验在真实世界和合成学习曲线数据集上分别展示了高达2.84%和5.47%的平均相对改进。这种战略资源分配使我们能够以显著降低的计算成本获得准确的缩放定律,相比传统的穷举方法节省高达98.7%的计算量。

英文摘要

Predicting model performance at larger scales enables the design of training strategies and architectures tailored to specific performance targets. Empirical scaling law research identifies functional forms to aid this prediction task. These describe the relationship between loss and compute using a loss-compute frontier defined by learning curves. Due to the empirical nature of this approach, the computational burden is substantial, making strategic resource allocation essential - yet it remains surprisingly underexplored. In this work, we address this shortcoming by exploring the suitability of Successive Halving (SH) and SH combined with parametric and non-parametric surrogate models. In addition to enabling a more systematic allocation of a given compute budget, our findings show that SH paired with surrogate models yields a set of learning curves that includes one with a lower loss-compute value than what naive uniform allocation or an SH-only approach can obtain. Our experiments demonstrate mean relative improvements of up to 2.84% and 5.47% on real-world and synthetic learning curve datasets. This strategic resource allocation enables us to obtain accurate scaling laws at significantly reduced computational costs, saving up to 98.7% over the traditional exhaustive approach.

2605.16953 2026-05-26 cs.AI cs.CL

How do Humans Process AI-generated Hallucination Contents: a Neuroimaging Study

人类如何处理AI生成的幻觉内容:一项神经影像学研究

Shuqi Zhu, Yi Zhong, Ziyi Ye, Bangde Du, Yujia Zhou, Qingyao Ai, Yiqun Liu

发表机构 * Department of Computer Science and Technology, Tsinghua University, Beijing, China(清华大学计算机科学与技术系) Institute of Trustworthy Embodied AI, Fudan University, Shanghai, China(复旦大学可信具身人工智能研究院)

AI总结 通过EEG实验,研究人类在处理多模态大语言模型生成的幻觉与非幻觉内容时的神经动力学差异,揭示误判的幻觉内容未能触发标准神经认知事实验证通路。

详情
AI中文摘要

尽管AI生成的幻觉带来了相当大的风险,但人类能够成功识别或被这些幻觉误导的潜在认知机制仍不清楚。为了解决这个问题,本文探索了人类的神经动力学,以表征大脑如何处理幻觉内容。我们记录了27名参与者在执行验证任务时的EEG信号,该任务要求判断由多模态大语言模型(MLLM)生成的图像描述的正确性。基于平均事件相关电位(ERP)研究,我们揭示了多种认知过程,例如语义整合、推理处理、记忆检索和认知负荷,在处理幻觉与非幻觉内容时表现出不同的模式。值得注意的是,人类参与者误判与正确判断的幻觉的神经反应显示出显著差异。这表明,被误判的AI生成幻觉未能触发标准的神经认知事实验证通路。

英文摘要

While AI-generated hallucinations pose considerable risks, the underlying cognitive mechanisms by which humans can successfully recognize or be misled by these hallucinations remain unclear. To address this problem, this paper explores humans' neural dynamics to characterize how the brain processes hallucinated content. We record EEG signals from 27 participants while they are performing a verification task to judge the correctness of image descriptions generated by a multi-modal large language model (MLLM). Based on an averaged event-related potential (ERP) study, we reveal that multiple cognitive processes, e.g., semantic integration, inferential processing, memory retrieval, and cognitive load, exhibit distinct patterns when humans process hallucinated versus non-hallucinated content. Notably, neural responses to hallucinations that were misjudged versus correctly judged by human participants showed significant differences. This indicates that misjudged AI-generated hallucinations failed to trigger the standard neurocognitive fact verification pathway.

2605.16023 2026-05-26 cs.CL cs.LG

Judge Circuits

Judge Circuits

Nils Feldhus, Tanja Baeumel, Elena Golimblevskaia, Qianli Wang, Van Bach Nguyen, Aaron Louis Eidt, Selin Kahvecioglu, Christopher Ebert, Wojciech Samek, Jing Yang, Vera Schmitt, Sebastian Möller, Simon Ostermann

发表机构 * Technische Universität Berlin(柏林技术大学) BIFOLD – Berlin Institute for the Foundations of Learning and Data(柏林学习与数据基础研究院) German Research Center for Artificial Intelligence (DFKI)(德国人工智能研究中心) Fraunhofer Heinrich Hertz Institute(弗劳恩霍夫海因里希·赫茨研究所) Marburg University(马尔堡大学) Centre for European Research in Trusted AI (CERTAIN)(欧洲可信人工智能研究中心)

AI总结 本研究利用位置感知边归因修补(PEAP)因果分析Gemma-3、Qwen2.5和Llama-3的内部机制,发现结构化理解和开放式偏好任务中的判断共享一个稀疏、泛化的潜在评估子图,并通过解耦抽象判断与输出格式,揭示了格式诱导不一致性的机制原因。

Comments 39 pages

详情
AI中文摘要

LLM-as-a-judge已成为大规模评估模型输出的主导范式,然而同一模型在其输出格式变化时(例如,1-5评分与真/假标签)会系统地给出不同的分数。现有对这些格式诱导不一致性的诊断停留在输入输出层面。利用位置感知边归因修补(PEAP),我们因果地研究了Gemma-3、Qwen2.5和Llama-3的内部机制。我们发现,跨结构化理解和开放式偏好任务的判断共享一个稀疏、泛化的潜在评估子图,位于中后期多层感知器(MLPs)中;将其零消融会破坏判断,同时保留架构模块化模型中的世界知识。通过结构上解耦抽象判断与输出格式,我们为我们研究的开放权重模型上的格式诱导不一致性提供了机制解释:在共享主干中计算的连续判断信号通过脆弱、格式特定的终端分支映射,使得格式无关的偏好能够在请求的输出格式下游被隔离。我们的发现意味着跨格式的基准级可靠性比较部分测量的是格式化器几何形状而非评估质量。

英文摘要

LLM-as-a-judge has become the dominant paradigm for grading model outputs at scale, yet the same model assigns systematically different scores when its output format changes (e.g., a 1-5 rating vs. a True/False label). Existing diagnoses of these format-induced inconsistencies stop at the input-output level. Using Position-aware Edge Attribution Patching (PEAP), we causally investigate the internal mechanism in Gemma-3, Qwen2.5, and Llama-3. We find that judgments across structured understanding and open-ended preference tasks share a sparse, generalized Latent Evaluator sub-graph in the mid-to-late multi-layer perceptrons (MLPs); zero-ablating it collapses judgment while preserving world knowledge in architecturally modular models. By structurally decoupling abstract judging from output formatting, we provide a mechanistic account of format-induced inconsistency on the open-weight models we study: a continuous judgment signal computed in the shared trunk is mapped through fragile, format-specific terminal branches, enabling format-independent preference to be isolated downstream of the requested output format. Our findings imply that benchmark-level reliability comparisons across formats are partially measuring formatter geometry rather than evaluation quality.

2605.14769 2026-05-26 cs.LG

Composable Crystals: Controllable Materials Discovery via Concept Learning

可组合晶体:通过概念学习实现可控材料发现

Nian Liu, Yuwei Zeng, Ryoji Kubo, Nikita Kazeev, Stephen Gregory Dale, Artem Maevskiy, Pengru Huang, Thomas Laurent, Kostya S. Novoselov, Xavier Bresson

发表机构 * National University of Singapore(新加坡国立大学) Loyola Marymount University(洛桑玛丽蒙大学)

AI总结 提出基于概念组合的晶体生成框架,利用向量量化变分自编码器自动发现可重用晶体概念,通过概念重组实现可控的新晶体探索,在MP-20和Alex-MP-20上V.S.U.N指标提升最高53.2%和51.7%。

详情
AI中文摘要

从头晶体生成是材料发现中的核心任务,旨在生成同时有效、稳定、独特且新颖的晶体。现有方法主要依赖黑盒随机采样,对生成结构如何超越观测分布的控制有限。本文提出了一种基于概念的组合式晶体生成框架。我们训练了一个向量量化变分自编码器,自动发现一组可重用的晶体概念,这些概念作为引导生成的构建块。这些学习到的概念在局部原子环境和全局对称模式上自然表现出可解释性,并能泛化到不同分布的晶体。通过重组这些概念,我们的框架能够可控地探索训练分布之外的新颖晶体,而非仅依赖无约束的随机采样。为进一步提高组合效率,我们引入了一个组合生成器,并使用模型自身生成的高质量样本对其进行迭代优化。最终的概念组合用于条件化下游晶体生成。在MP-20和Alex-MP-20上的数值实验表明,分别组合概念使基础模型在V.S.U.N指标上提升高达53.2%和51.7%,尤其在新颖性方面增益显著。

英文摘要

De novo crystal generation, a central task in materials discovery, aims to generate crystals that are simultaneously valid, stable, unique, and novel. Existing methods mainly rely on black-box stochastic sampling, providing limited control over how generated structures move beyond the observed distribution. In this paper, we introduce a concept-based compositional framework for crystal generation. We train a vector-quantized variational autoencoder to automatically discover a shared set of reusable crystal concepts, which serve as building blocks for guided generation. These learned concepts naturally exhibit interpretability from both local atomic environments and global symmetry patterns, and generalize to crystals from different distributions. By recombining such concepts, our framework enables controllable exploration of novel crystals beyond the training distribution, rather than relying solely on unconstrained random sampling. To further improve composition efficiency, we introduce a composition generator and iteratively refine it using high-quality samples generated by the model itself. The resulting concept compositions are then used to condition downstream crystal generation. Numerical experiments on MP-20 and Alex-MP-20 show that compositing concepts separately increase base model up to 53.2% and 51.7% on V.S.U.N metric, with particular gains in novelty.

2605.14759 2026-05-26 cs.LG

Crys-JEPA: Accelerating Crystal Discovery via Embedding Screening and Generative Refinement

Crys-JEPA:通过嵌入筛选和生成精炼加速晶体发现

Nian Liu, Nikita Kazeev, Stephen Gregory Dale, Artem Maevskiy, Yuwei Zeng, Ryoji Kubo, Pengru Huang, Thomas Laurent, Yann LeCun, Kostya S. Novoselov, Xavier Bresson

发表机构 * National University of Singapore(国立新加坡大学) Loyola Marymount University(洛约拉玛丽蒙特大学) New York University(纽约大学) AMI

AI总结 提出Crys-JEPA联合嵌入预测架构,通过能量感知的潜在空间和筛选-精炼流程,解决晶体生成中稳定性和新颖性的冲突,在MP-20和Alex-MP-20数据集上V.S.U.N.指标分别提升53.8%和72.7%。

详情
AI中文摘要

从头晶体生成旨在发现不仅真实而且稳定和新颖的材料。然而,大多数现有生成模型被训练为最大化观测晶体的似然,这鼓励样本接近已知材料,但不一定与发现中重要的标准一致。我们的实证分析表明,当前晶体生成模型在稳定性和新颖性之间存在明显冲突:接近观测分布的样本倾向于保持稳定性但提供有限的新颖性,而远离分布的样本通常迅速失去稳定性。这表明发现既稳定又新颖晶体的有用区域极其狭窄。为了突破这一限制,我们引入了Crys-JEPA,一种用于晶体的联合嵌入预测架构,它学习一个能量感知的潜在空间,保留形成能差异。在这个空间中,稳定性评估可以重新表述为基于嵌入的与可访问训练晶体的比较,减少了对昂贵能量评估和特定任务外部参考的依赖。基于Crys-JEPA,我们进一步开发了一个筛选-精炼流程,识别有前景的生成晶体并重新引入它们以精炼生成模型。在MP-20和Alex-MP-20数据集上,我们在V.S.U.N.指标上分别比基线提升了53.8%和72.7%。

英文摘要

De novo crystal generation seeks to discover materials that are not merely realistic, but also stable and novel. However, most existing generative models are trained to maximize the likelihood of observed crystals, which encourages samples to stay close to known materials yet not necessarily align with the criteria that matter in discovery. Our empirical analysis shows that current crystal generative models exhibit a clear conflict between stability and novelty: samples near the observed distribution tend to retain stability but offer limited novelty, whereas samples farther from it often lose stability rapidly. This suggests that the useful region for discovering crystals that are both stable and novel is extremely narrow. To move beyond this limitation, we introduce Crys-JEPA, a joint embedding predictive architecture for crystals that learns an energy-aware latent space preserving formation-energy differences. In this space, stability assessment can be reformulated as an embedding-based comparison against accessible training crystals, reducing the reliance on expensive energy evaluation and task-specific external references. Building on Crys-JEPA, we further develop a screening-and-refinement pipeline that identifies promising generated crystals and reintroduces them to refine the generative model. On MP-20 and Alex-MP-20 datasets, we achieve improvements over baselines up to 53.8% and 72.7% on V.S.U.N. metric, respectively.

2605.13643 2026-05-26 cs.CL

Prefix Teach, Suffix Fade: Local Teachability Collapse in Strong-to-Weak On-Policy Distillation

前缀教导,后缀消退:强到弱在线策略蒸馏中的局部可教性崩溃

Kaiyuan Liu, Ziyuan Zhuang, Yang Bai, Bing Wang, Rongxiang Weng, Jieping Ye

发表机构 * College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院) Meituan LongCat Team, China(美团LongCat团队,中国) College of Computer Science and Technology, Jilin University(吉林大学计算机科学与技术学院)

AI总结 本文发现强到弱在线策略蒸馏中,教师反馈在生成轨迹的后缀部分缺乏局部对比度,导致“局部可教性崩溃”,并提出基于变化点检测的截断规则来优化监督区域,实验表明该方法优于全轨迹蒸馏。

详情
AI中文摘要

在线策略蒸馏(OPD)利用来自更强教师的密集反馈,在学生模型自身的生成轨迹上训练学生模型。先前文献表明,只要教师反馈可用,监督完整响应令牌序列应能单调提升性能。然而,我们证明这一假设在强到弱OPD设置中有时不成立。虽然生成轨迹的后缀部分可能仍存在非零的师生优势,但它们通常缺乏使密集反馈有效优先学生学习所需的局部对比度。我们将这种失败模式称为局部可教性崩溃。由此得出的原则很简单:监督应集中在教师反馈仍具有判别性的轨迹区域,而非均匀覆盖整个响应。我们通过一种轨迹特定的释放规则来操作这一原则。该规则测量教师相对于学生前K个候选集的边际,将该边际在NLTK分词的句子片段上聚合,并在检测到BIC风格的下行变化点时截断密集OPD监督。使用Qwen3模型系列在强到弱蒸馏任务上的实验结果表明,该释放规则在不同学生规模下的五个域内基准上始终优于标准全轨迹OPD。此外,与基线蒸馏方法相比,我们的方法在域外任务上更好地保留了模型能力。这些结果表明,有效的强到弱OPD需要评估教师指导的可用性及其局部效用,确保生成的反馈保持可教性。

英文摘要

On-policy distillation (OPD) trains a student model on its own rollouts using dense feedback from a stronger teacher. Prior literature suggests that, provided teacher feedback is available, supervising the full sequence of response tokens should monotonically improve performance. However, we demonstrate that this assumption sometimes fails to hold in strong-to-weak OPD settings. While later segments of a generated trajectory may still exhibit a non-zero teacher-student advantage, they frequently lack the local contrast that makes dense feedback effective for prioritizing student learning. We term this failure mode local teachability collapse. The resulting principle is straightforward: supervision should concentrate on trajectory regions where the teacher's feedback remains discriminative, rather than uniformly covering the entire response. We operationalize this principle through a trajectory-specific release rule. This rule measures the teacher's margin over the student's top-$K$ candidate set, aggregates this margin across NLTK-tokenized sentence segments, and truncates dense OPD supervision upon detecting a BIC-style downward change point. Experimental results across strong-to-weak distillation tasks using the Qwen3 model family indicate that this release rule consistently outperforms standard full-trajectory OPD across five in-domain benchmarks at various student scales. Furthermore, compared to baseline distillation methods, our approach better preserves model capabilities on out-of-domain task. These results suggest that effective strong-to-weak OPD requires evaluating not only the availability of teacher guidance but also its local utility, ensuring that the generated feedback remains teachable.

2605.12964 2026-05-26 cs.CV

Asymmetric Flow Models

非对称流模型

Hansheng Chen, Jan Ackermann, Minseo Kim, Gordon Wetzstein, Leonidas Guibas

发表机构 * Stanford University(斯坦福大学)

AI总结 提出非对称流建模(AsymFlow),通过秩非对称速度参数化将噪声预测限制在低秩子空间,同时保持数据预测全维,从而在高维空间中实现高效的流生成,在ImageNet 256×256上取得领先的1.57 FID,并首次提供将预训练潜在流模型微调为像素空间模型的途径。

Comments Code: https://github.com/Lakonik/LakonLab Webpage: https://hanshengchen.com/asymflow

详情
AI中文摘要

高维空间中的基于流的生成是困难的,因为即使数据具有强低秩结构,速度预测也需要建模高维噪声。我们提出非对称流建模(AsymFlow),一种秩非对称速度参数化,将噪声预测限制在低秩子空间,同时保持数据预测全维。通过这种非对称预测,AsymFlow在不改变网络架构或训练/采样过程的情况下,解析地恢复全维速度。在ImageNet 256×256上,AsymFlow取得了领先的1.57 FID,大幅优于先前的DiT/JiT类像素扩散模型。AsymFlow还首次提供了将预训练潜在流模型微调为像素空间模型的途径:将低秩像素子空间与潜在空间对齐,得到无缝初始化,保留潜在模型的高级语义和结构,因此微调主要改善低级不匹配,而非重新学习像素生成。我们展示了从FLUX.2 klein 9B微调得到的像素AsymFlow模型在像素空间文本到图像生成中建立了新的最先进水平,在HPSv3、DPG-Bench和GenEval上击败了其潜在基础模型,并在定性上显示出显著改善的视觉真实感。

英文摘要

Flow-based generation in high-dimensional spaces is difficult because velocity prediction requires modeling high-dimensional noise, even when data has strong low-rank structure. We present Asymmetric Flow Modeling (AsymFlow), a rank-asymmetric velocity parameterization that restricts noise prediction to a low-rank subspace while keeping data prediction full-dimensional. From this asymmetric prediction, AsymFlow analytically recovers the full-dimensional velocity without changing the network architecture or training/sampling procedures. On ImageNet 256$\times$256, AsymFlow achieves a leading 1.57 FID, outperforming prior DiT/JiT-like pixel diffusion models by a large margin. AsymFlow also provides the first-ever route for finetuning pretrained latent flow models into pixel-space models: aligning the low-rank pixel subspace to the latent space gives a seamless initialization that preserves the latent model's high-level semantics and structure, so finetuning mainly improves low-level mismatches rather than relearning pixel generation. We show that the pixel AsymFlow model finetuned from FLUX.2 klein 9B establishes a new state of the art for pixel-space text-to-image generation, beating its latent base on HPSv3, DPG-Bench, and GenEval while qualitatively showing substantially improved visual realism.

2605.12906 2026-05-26 cs.LG cs.AI

Data Difficulty and the Generalization--Extrapolation Tradeoff in LLM Fine-Tuning

数据难度与LLM微调中的泛化-外推权衡

Siyuan Liu, Tinghong Chen, Xinghan Li, Yifei Wang, Jingzhao Zhang

发表机构 * IIIS, Tsinghua University(清华大学人工智能学院) College of AI, Tsinghua University(清华大学人工智能学院) Shanghai Qi Zhi Institute(上海启智研究院) Amazon AGI SF Lab(亚马逊AGI旧金山实验室)

AI总结 本文通过实证和理论分析,研究了监督微调中数据难度对模型行为的影响,发现数据难度与数据量共同决定泛化与外推之间的权衡,并存在最优难度随数据量增加而向更难数据偏移的规律。

Comments Accepted to ICML 2026

详情
AI中文摘要

监督微调(SFT)期间的数据选择可以显著改变大型语言模型(LLMs)的行为。尽管已有工作研究了基于困惑度、难度或长度等启发式方法选择数据的效果,但报告的结果往往不一致或依赖于上下文。在这项工作中,我们从实证和理论角度系统地研究了数据难度在微调中的作用,并发现不存在普遍最优的难度水平;相反,其有效性取决于数据集大小。我们表明,对于固定的数据预算,SFT存在一个最优的数据难度,并且随着数据预算的增加,该最优难度向更难的数据偏移。为了解释这一现象,我们进行了受控的合成实验,揭示了一个简单的底层机制:分布内泛化差距与外推差距之间的相互作用。我们通过使用PAC-Bayesian泛化界限的理论分析进一步支持了这一机制。总的来说,我们的结果阐明了数据大小和难度如何共同影响SFT中泛化与外推之间的权衡,为在特定模型和数据条件下基于难度的数据选择提供了指导。

英文摘要

Data selection during supervised fine-tuning (SFT) can critically change the behavior of large language models (LLMs). Although existing work has studied the effect of selecting data based on heuristics such as perplexity, difficulty, or length, the reported findings are often inconsistent or context-dependent. In this work, we systematically study the role of data difficulty in fine-tuning from both empirical and theoretical perspectives, and find that there is no universally optimal difficulty level; rather, its effectiveness depends on the dataset size. We show that for a fixed data budget, there exists an optimal data difficulty for SFT, and that this optimal difficulty shifts toward harder data as the data budget increases. To explain this phenomenon, we conduct controlled synthetic experiments that reveal a simple underlying mechanism: the interplay between the (in-distribution) generalization gap and the extrapolation gap. We further support this mechanism through a theoretical analysis using PAC-Bayesian generalization bounds. Overall, our results clarify how data size and difficulty jointly affect the trade-off between generalization and extrapolation in SFT, providing guidance for difficulty-based data selection under certain model and data conditions.

2605.12649 2026-05-26 cs.CV

DIVER:Diving Deeper into Distilled Data via Expressive Semantic Recovery

DIVER: 通过表达性语义恢复深入挖掘蒸馏数据

Qianxin Xia, Zhiyong Shu, Wenbo Jiang, Jiawei Du, Jielei Wang, Guoming Lu

发表机构 * University of Electronic Science and Technology of China, Chengdu, China(电子科技大学,成都,中国) Institute of High Performance Computing (IHPC), Agency for Science, Technology and Research (A*STAR), Singapore(高性能计算研究所(IHPC),科技研究局(A*STAR),新加坡) Centre for Frontier AI Research (CFAR), Agency for Science, Technology and Research (A*STAR), Singapore(前沿人工智能研究中心(CFAR),科技研究局(A*STAR),新加坡)

AI总结 提出双阶段蒸馏框架DIVER,利用预训练扩散模型通过语义继承、引导和融合恢复蒸馏数据的表达性语义,提升跨架构泛化能力。

Comments Accepted by ICML 2026

详情
AI中文摘要

数据集蒸馏旨在从原始数据集中合成一个紧凑的代理数据集,该数据集不可读或非原始,以保护隐私并实现高效学习。然而,先前的方法通常采用单阶段蒸馏范式,该范式会学习过度适应先验架构的特定模式,从而抑制语义表达并导致跨异构架构的性能下降。为了解决这个问题,我们提出了一种新颖的双阶段蒸馏框架,称为${ extbf{DIVER}}$,它利用预训练的扩散模型通过表达性语义恢复深入挖掘蒸馏数据,整个过程包括语义继承、引导和融合。语义继承将抽象蒸馏图像的高级语义蒸馏到潜在空间中,以过滤掉架构特定的“噪声”并保留内在语义。此外,语义引导通过指导反向过程来改善原始语义的保留。最后,语义融合被设计为仅在反向过程的具体阶段提供语义引导,防止语义模糊和伪影,同时保持引导信息。大量实验验证了DIVER在改进经典蒸馏技术和显著提升跨架构泛化方面的有效性和效率,在ImageNet(256×256)上仅需与原始DiT相当的处理时间,且仅使用4 GB GPU内存。

英文摘要

Dataset distillation aims to synthesize a compact proxy dataset that is unreadable or non-raw from the original dataset for privacy protection and highly efficient learning. However, previous approaches typically adopt a single-stage distillation paradigm, which suffers from learning specific patterns that overfit on a prior architecture, consequently suppressing the expression of semantics and leading to performance degradation across heterogeneous architectures. To address this issue, we propose a novel dual-stage distillation framework called ${\textbf{DIVER}}$, which leverages the pre-trained diffusion model to dive deeper into $\textbf{DI}$stilled data $\textbf{V}$ia $\textbf{E}$xpressive semantic $\textbf{R}$ecovery, an entire process of semantic inheritance, guidance, and fusion. Semantic inheritance distills high-level semantics of abstract distilled images into the latent space to filter out architecture-specific ``noise" and retain the intrinsic semantics. Furthermore, semantic guidance improves the preservation of the original semantics by directing the reverse procedure. Finally, semantic fusion is designed to provide semantic guidance only during the concrete phase of the reverse process, preventing semantic ambiguity and artifacts while maintaining the guidance information. Extensive experiments validate the effectiveness and efficiency of DIVER in improving classical distillation techniques and significantly improving cross-architecture generalization, requiring processing time comparable to raw DiT on ImageNet (256$\times$256) with only 4 GB of GPU memory usage.

2605.12374 2026-05-26 cs.CV cs.AI cs.LG

Fill the GAP: A Granular Alignment Paradigm for Visual Reasoning in Multimodal Large Language Models

填补GAP:多模态大语言模型中视觉推理的粒度对齐范式

Yanting Miao, Yutao Sun, Dexin Wang, Mengyu Zhou, Pascal Poupart, Lei Lv, Qi Zhao, Li Wang, Hao Li, Xiaoxi Jiang, Guanjun Jiang

发表机构 * Qwen Large Model Application Team, Alibaba(阿里云大模型应用团队) Alibaba University of Waterloo(阿里大学水力学院) Vector Institute(向量研究所) Zhejiang University(浙江大学)

AI总结 提出GAP(粒度对齐范式),通过特征级、上下文级和能力引导级对齐,解决多模态大语言模型中视觉潜在推理的特征空间不匹配问题,提升感知与推理性能。

详情
AI中文摘要

视觉潜在推理让多模态大语言模型(MLLM)以连续令牌形式创建中间视觉证据,避免外部工具或图像生成器。然而,现有方法通常遵循输出即输入的潜在范式,产生不稳定的收益。我们识别出特征空间不匹配是导致这种不稳定的证据:主流的视觉潜在模型建立在预归一化MLLM上,重用解码器隐藏状态作为预测的潜在输入,尽管这些状态与模型训练时消耗的输入嵌入处于截然不同的范数范围(Xie et al., 2025; Li et al., 2026; Team et al., 2026)。这种不匹配可能使直接潜在反馈不可靠。受此诊断启发,我们提出GAP,一种用于视觉潜在建模的粒度对齐范式。GAP在三个层面对齐视觉潜在推理:特征级对齐通过轻量级PCA对齐潜在头将解码器输出映射为输入兼容的视觉潜在;上下文级对齐通过可检查的辅助视觉监督锚定潜在目标;能力引导对齐选择性地将潜在监督分配给基础MLLM难以处理的示例。在Qwen2.5-VL 7B上,所得模型在我们监督变体中实现了最佳平均聚合感知和推理性能。推理时干预探测进一步表明,生成的潜在提供了任务相关的视觉信号,而不仅仅是增加令牌槽位。

英文摘要

Visual latent reasoning lets a multimodal large language model (MLLM) create intermediate visual evidence as continuous tokens, avoiding external tools or image generators. However, existing methods usually follow an output-as-input latent paradigm and yield unstable gains. We identify evidence for a feature-space mismatch that can contribute to this instability: dominant visual-latent models build on pre-norm MLLMs and reuse decoder hidden states as predicted latent inputs, even though these states occupy a substantially different norm regime from the input embeddings the model was trained to consume (Xie et al., 2025; Li et al., 2026; Team et al., 2026). This mismatch can make direct latent feedback unreliable. Motivated by this diagnosis, we propose GAP, a Granular Alignment Paradigm for visual latent modeling. GAP aligns visual latent reasoning at three levels: feature-level alignment maps decoder outputs into input-compatible visual latents through a lightweight PCA-aligned latent head; context-level alignment grounds latent targets with inspectable auxiliary visual supervision; and capacity-guided alignment assigns latent supervision selectively to examples where the base MLLM struggles. On Qwen2.5-VL 7B, the resulting model achieves the best mean aggregate perception and reasoning performance among our supervised variants. Inference-time intervention probing further suggests that generated latents provide task-relevant visual signal beyond merely adding token slots.

2605.10913 2026-05-26 cs.AI cs.PL cs.SE

Shepherd: A Runtime Substrate Empowering Meta-Agents with a Formalized Execution Trace

Shepherd: 一个为元代理提供形式化执行迹的运行时基座

Simon Yu, Derek Chong, Ananjan Nandi, Dilara Soylu, Jiuding Sun, Christopher D Manning, Weiyan Shi

发表机构 * Northeastern University(东北大学) Stanford University(斯坦福大学)

AI总结 提出Shepherd,一个基于函数式编程的Python运行时基座,将代理执行作为一等对象,通过类似Git的执行迹支持元代理的检查、分叉和重放,在三个用例中显著提升性能。

Comments 50 pages, 22 figures, 14 tables

详情
AI中文摘要

随着LLM代理系统承担更复杂的任务,它们越来越依赖元代理:对其他代理进行操作的高阶代理,就像管理者监督员工一样。无论元代理做什么:协调代理、在执行前停止风险动作、或修复失败的运行,都需要在运行时操纵代理执行。现有的代理基座使得这变得困难:它们只给元代理提供纯文本记录和环境快照,要求元代理构建自己的工具来重建和编排执行状态。因此,我们引入了Shepherd,一个基于函数式编程原则的Python基座,其中代理的执行本身是一个一等对象,元代理可以检查和转换它。每个模型调用、工具调用和环境变化都成为类似Git的执行迹中的一个结构化事件,任何过去的状态都可以被分叉(比docker commit快5倍)并重放。三个示例用例展示了Shepherd的多功能性:(1)一个监督代理防止并行编码代理之间的冲突,将CooperBench的性能从28.8%提升到54.7%;(2)一个反事实优化器通过提出编辑并从行为改变点重放运行来修复代理工作流,在TerminalBench-2上比MetaHarness低58%的挂钟时间;(3)一个元代理在展开期间选择分叉点以改进长程代理强化学习中的信用分配,在TerminalBench-2上将GRPO的增益翻倍。我们开源Shepherd,以通过原则性和高效的代理执行操作赋能未来的元代理。

英文摘要

As LLM agent systems take on more complex tasks, they increasingly rely on meta-agents: higher-order agents that operate on other agents, much as managers supervise employees. Whatever a meta-agent does: coordinating agents, halting risky actions before execution, or repairing failed runs, requires manipulation of agentic execution at runtime. Existing agentic substrates make this hard: they give meta-agents only plain transcripts and environment snapshots, requiring it to build it's own tooling to reconstruct and orchestrate execution state. Therefore, we introduce Shepherd, a Python substrate grounded in functional programming principles, where an agent's execution is itself a first-class object that a meta-agent can inspect and transform. Every model call, tool call, and environment change becomes a structured event in a Git-like execution trace, where any past state can be forked 5x faster than docker commit and replayed. Three example use cases show Shepherd's versatility: (1) a supervisor agent prevents conflicts among parallel coding agents, lifting CooperBench performance from 28.8% to 54.7%; (2) a counterfactual optimizer repairs agent workflows by proposing edits and replaying runs from the point of changed behavior, outperforming MetaHarness on TerminalBench-2 with 58% lower wall-clock; (3) a meta-agent picks fork points during rollouts to improve credit assignment in long-horizon agentic RL, doubling GRPO's gains on TerminalBench-2. We open-source Shepherd to empower future meta-agents with principled and efficient operations over agentic execution.

2605.09270 2026-05-26 cs.LG cs.AI

Memorize Theorems, Not Instances: Probing SFT Generalization through Mathematical Reasoning

记忆定理而非实例:通过数学推理探究SFT泛化

Ruiying Peng, Mengyu Yang, Jing Lei, Xiaohui Li, Xueyu Wu, Xinlei Chen

发表机构 * Tsinghua Shenzhen International Graduate School(清华大学深圳国际研究生院) Huawei Technologies(华为技术)

AI总结 针对监督微调(SFT)损害推理泛化的问题,提出Theorem-SFT方法,通过显式定理应用训练,在多个基准上取得显著提升,并揭示前馈层是推理规则的主要存储位置。

详情
AI中文摘要

监督微调(SFT)广泛用于任务特定适配,但近期工作表明它会系统性地削弱推理泛化。我们认为根本原因不在于记忆本身,而在于其目标:标准SFT驱动模型利用并记忆问题-答案对中的虚假表面相关性,使其对表面输入变化脆弱。为解决此问题,我们提出Theorem-SFT,通过教授模型规则如何被调用而非答案看起来像什么,将监督重新导向显式定理应用。Theorem-SFT在多个基准和模型家族上取得一致提升:在MATH上(LLaMA3.2-3B-Instruct)提升8.8%,在GeoQA上(Qwen2.5-VL-7B-Instruct)提升20.27%,无需特定模态的重新训练。仅微调MLP层即可达到全层性能,表明前馈组件是推理规则的主要存储位置。我们的发现重新定义了争论:泛化失败并非源于记忆机制本身,而是源于记忆了错误的归纳目标。

英文摘要

Supervised Fine-Tuning (SFT) is widely used for task-specific adaptation, yet recent work shows it systematically undermines reasoning generalization. We argue the root cause is not memorization itself, but its target: vanilla SFT drives models to exploit and memorize spurious surface correlations in problem-solution pairs, leaving them brittle to superficial input variations. To address this, we propose Theorem-SFT, which reorients supervision toward explicit theorem application by teaching models how rules are invoked rather than what answers look like. Theorem-SFT yields consistent gains across benchmarks and model families: +8.8% on MATH (LLaMA3.2-3B-Instruct) and +20.27% on GeoQA (Qwen2.5-VL-7B-Instruct) without modality-specific re-training. Fine-tuning MLP layers alone matches full-layers performance, implicating feed-forward components as the primary locus of reasoning rules. Our findings reframe the debate: Generalization failures stem not from memorization as a mechanism, but from memorizing the wrong inductive targets.

2605.09223 2026-05-26 cs.CV

CREST: Curvature-Regulated Event-Centric Sampling for Efficient Long-Video Understanding

CREST: 曲率调节的事件中心采样用于高效长视频理解

Mehrajul Abadin Miraj, Abdul Mohaimen Al Radi, Shariful Islam Rayhan, Md. Tanvir Alam, Ismat Rahman, Yu Tian, Md Mosaddek Khan

发表机构 * Dept. of CSE, University of Dhaka(达卡大学计算机科学与工程系) Dept. of CSE, University of Central Florida(中央佛罗里达大学计算机科学与工程系)

AI总结 提出一种无训练帧选择方法CREST,利用查询-帧相关性的时间几何(局部曲率)来指导采样,在固定预算下实现高效长视频理解。

详情
AI中文摘要

从长视频中选择信息帧是一个组合问题,现有方法要么通过高效启发式方法处理,但未显式建模查询条件的时间结构,要么通过多阶段检索流水线处理,但预处理成本高。我们提出 extbf{CREST},一种基于查询-帧相关性的时间几何的无训练帧选择方法。CREST基于观察:相关性随时间表现出结构化的局部变化——显著事件周围曲率陡峭,冗余段区域平坦。通过使用局部曲率指导选择,CREST在短暂决定性事件和缓慢演变的证据之间更有效地分配固定帧预算。在固定主干网络和帧预算下,CREST在LongVideoBench和VideoMME上比轻量级相关性-覆盖基线AKS获得更高准确率,同时保留了更强多阶段检索流水线MIRA的93-95%准确率,而预处理成本仅为后者的3-4%。 ootnote{代码和实现细节包含在补充材料中,将在接收后公开发布。}在时间帧选择的诊断基准TempRel上,CREST比AKS相对提高6.88%。成对LLM-as-a-judge评估进一步表明,CREST选择的帧产生更连贯的帧条件描述,在两个基准上胜率分别为60.58%和54.50%。这些结果表明,局部时间几何为长视频帧选择提供了简单高效的基础。

英文摘要

Selecting informative frames from long videos is a combinatorial problem that existing methods address either through efficient heuristics without explicit modeling of query-conditioned temporal structure, or through multi stage retrieval pipelines with substantial preprocessing cost. We propose \textbf{CREST}, a training-free frame selection method grounded in the temporal geometry of query--frame relevance. CREST is based on the observation that relevance over time exhibits structured local variation: sharp curvature around salient events and flatter regions in redundant segments. By using local curvature to guide selection, CREST allocates a fixed frame budget more effectively across brief decisive events and slowly evolving evidence. Under a fixed backbone and frame budget, CREST achieves higher accuracy than AKS, a lightweight relevance--coverage baseline, on LongVideoBench and VideoMME, while retaining 93--95\% of the accuracy of MIRA, a stronger multi-stage retrieval pipeline, at only 3--4\% of its preprocessing cost.\footnote{Code and implementation details are included in the supplementary material and will be released publicly upon acceptance.} On TempRel, our diagnostic benchmark for temporal frame selection, CREST achieves a 6.88\% relative improvement over AKS. Pairwise LLM-as-a-judge evaluation further shows that CREST-selected frames yield more coherent frame-conditioned descriptions, with win rates of 60.58\% and 54.50\% on the two benchmarks. These results show that local temporal geometry provides a simple and efficient basis for long-video frame selection.

2605.07607 2026-05-26 cs.CV

FS-I2P:A Hierarchical Focus-Sweep Registration Network with Dynamically Allocated Depth

FS-I2P:一种具有动态分配深度的分层聚焦扫描配准网络

Zhixin Cheng, Yujia Chen, Xujing Tao, Bohao Liao, Xiaotian Yin, Baoqun Yin, Tianzhu Zhang

发表机构 * School of Information Science and Technology, University of Science and Technology of China(信息科学与技术学院,中国科学技术大学) School of Computer Science and Information Engineering, Hefei University of Technology(计算机科学与信息工程学院,合肥工业大学) National Key Laboratory of Deep Space Exploration, Deep Space Exploration Laboratory(深空探测国家实验室,深空探测实验室) Institute of Advanced Technology, University of Science and Technology of China(先进技术研究院,中国科学技术大学)

AI总结 提出一种基于聚焦-扫描范式的分层交互模块和动态层分配策略,用于解决图像到点云配准中的尺度模糊和注意力漂移问题,在RGB-D Scenes V2和7-Scenes数据集上达到最优性能。

详情
AI中文摘要

图像到点云的配准常常受到视角变化、跨模态差异和重复纹理的挑战,这些因素会导致尺度模糊,进而产生错误的对应关系。最近的无检测方法通过利用多尺度特征和基于Transformer的交互来缓解这一问题。然而,它们仍然存在跨层的注意力漂移和层内不一致性,阻碍了精确配准。受人类行为启发,我们提出了一种“聚焦-扫描”范式,并在基于SSM的框架内开发了分层聚焦-扫描交互模块,以增强多层次跨模态特征关联。此外,我们引入了一种动态层分配策略,自适应地确定迭代深度,以更好地利用几何约束并提高匹配鲁棒性。在两个基准数据集RGB-D Scenes V2和7-Scenes上的大量实验和消融研究表明,我们的方法达到了最先进的性能。

英文摘要

Image-to-point cloud registration is often challenged by viewpoint changes, cross-modal discrepancies, and repetitive textures, which induce scale ambiguity and consequently lead to erroneous correspondences. Recent detection-free methods alleviate this issue by leveraging multi-scale features and transformer-based interactions. However, they still suffer from attention drift across layers and intra-scale inconsistencies, hindering precise registration. Inspired by human behavior, we propose a ``Focus--Sweep'' paradigm and develop a Hierarchical Focus--Sweep Interaction Module within an SSM-based framework to enhance multi-level cross-modal feature association. In addition, we introduce a Dynamic Layer Allocation Strategy that adaptively determines the iteration depth to better exploit geometric constraints and improve matching robustness. Extensive experiments and ablations on two benchmarks, RGB-D Scenes V2 and 7-Scenes, demonstrate that our approach achieves state-of-the-art performance.

2605.07233 2026-05-26 cs.LG cs.CR stat.ML

Modulated learning for private and distributed regression with just a single sample per client device

调制学习:每个客户端设备仅有一个样本的私有分布式回归

Praneeth Vepakomma, Amirhossein Reisizadeh, Samuel Horváth, Munther A. Dahleh

发表机构 * MIT(麻省理工学院)

AI总结 针对每个客户端仅有一个样本的分布式学习场景,提出一种通过注入校准噪声并共享后处理表示来实现隐私保护的全局模型学习方法,在期望上匹配非私有中心化梯度更新。

Comments 30 pages

详情
AI中文摘要

本文聚焦于从大量设备中学习的问题,每个设备仅持有一个数据样本。这种每客户端一个样本的设置存在于多个实际应用中,包括从健身追踪器、数据/应用使用聚合器、可穿戴传感设备和日常事件监测器等学习。当客户端只有一个样本时,标准的联邦学习范式会失效,因为基于单个点的局部更新远非有用,尤其是在模型系数估计的早期轮次中。这种效用进一步被每轮添加的隐私诱导噪声削弱。本文针对这一问题,使此类客户端能够协作贡献,有效学习全局模型,同时不泄露其数据隐私。所提出的方法在每个客户端注入一个精心校准的噪声扰动来变换样本,然后共享经过后处理的表示给服务器。服务器聚合这些表示,处理得到无偏梯度更新,该更新在期望上匹配非私有中心化梯度,同时保护数据隐私。这种方法不同于传统的私有联邦学习,其中通信负载涉及模型系数而非私有变换的数据样本。该方法使数据极其有限的设备能够协作学习准确、保护隐私的模型,无需大量本地数据集或牺牲个体隐私。

英文摘要

This work focuses on the question of learning from a large number of devices with each device holding only a single sample of data. Several real-world applications exist to this one sample per client setup up including learning from fitness trackers, data/app usage aggregators, body-worn sensing devices, and daily event monitors to name a few. When a client has only one sample, the standard federated learning paradigm breaks down as a local update based on that single point is far from being useful, especially in the earlier rounds for estimation of the model coefficients. This utility is further weakened by the privacy-inducing noise applied at every round. This work caters to this problem to enable such clients to collaboratively contribute to effectively learn a global model without leaking the privacy of their data. The proposed approach injects a single, carefully calibrated noisy perturbation to transform the sample at each client, followed by a post-processed representation which is shared with the server. These representations aggregated at the server are processed to obtain an unbiased gradient update that in expectation matches the non-private centralized gradient while preserving data privacy. This approach is different than traditional private federated learning, where the communication payloads involve model coefficients as opposed to privately transformed data samples. This method enables devices with extremely limited data to collaborate and learn accurate, privacy-preserving models without requiring large local datasets or sacrificing individual privacy.

2605.04906 2026-05-26 cs.AI

Strat-Reasoner: Reinforcing Strategic Reasoning of LLMs in Multi-Agent Games

Strat-Reasoner:在多智能体游戏中增强大语言模型的战略推理能力

Yidong He, Yutao Lai, Pengxu Yang, Jiarui Gan, Jiexin Wang, Yi Cai, Mengchen Zhao

发表机构 * School of Software Engineering, South China University of Technology(华南理工大学软件学院) Department of Computer Science, University of Oxford(牛津大学计算机科学系)

AI总结 提出Strat-Reasoner框架,通过递归推理范式和集中式思维链比较模块,结合混合优势与组相对强化学习,提升大语言模型在多智能体游戏中的战略推理能力。

详情
AI中文摘要

虽然大语言模型(LLMs)在某些推理任务中表现出色,但在最终结果取决于所有智能体联合策略的多智能体游戏中,它们却难以应对。在多智能体游戏中,其他智能体的非平稳性给推理过程的评估和多个推理步骤上的信用分配带来了重大挑战。现有的单智能体强化学习(RL)方法及其多智能体扩展未能解决这些挑战,因为它们没有将其他智能体纳入推理过程。在这项工作中,我们提出了Strat-Reasoner,一种新颖的基于强化学习的框架,旨在提升LLMs在多智能体游戏中的战略推理能力。我们引入了一种新颖的递归推理范式,其中智能体的推理也整合了其他智能体的推理过程。为了为中间推理序列提供有效的奖励信号,我们采用了一个集中的思维链(CoT)比较模块来评估推理质量。最后,我们计算了一个准确的混合优势,并开发了一种组相对强化学习方法以优化LLM策略。实验结果表明,Strat-Reasoner显著提升了底层LLMs的战略能力,在各种多智能体游戏中平均性能提升了22.1%。代码已公开在https://github.com/ydhe1012/Strat-Reasoner。

英文摘要

While Large Language Models (LLMs) excel in certain reasoning tasks, they struggle in multi-agent games where the final outcome depends on the joint strategies of all agents. In multi-agent games, the non-stationarity of other agents brings significant challenges on the evaluation of the reasoning process and the credit assignment over multiple reasoning steps. Existing single-agent reinforcement learning (RL) approaches and their multi-agent extensions fail to address these challenges as they do not incorporate other agents in the reasoning process. In this work, we propose Strat-Reasoner, a novel RL-based framework that improves LLMs' strategic reasoning ability in multi-agent games. We introduce a novel recursive reasoning paradigm where an agent's reasoning also integrates other agents' reasoning processes. To provide effective reward signals for the intermediate reasoning sequences, we employ a centralized Chain-of-Thought (CoT) comparison module to evaluate the reasoning quality. Finally, we compute an accurate hybrid advantage and develop a group-relative RL approach to optimize the LLM policy. Experimental results show that Strat-Reasoner substantially improves strategic abilities of underlying LLMs, achieving 22.1\% average performance improvements across various multi-agent games. Code is publicly available at https://github.com/ydhe1012/Strat-Reasoner.

2605.03804 2026-05-26 cs.AI

ScrapMem: A Bio-inspired Framework for On-device Personalized Agent Memory via Optical Forgetting

ScrapMem: 一种基于生物启发的光学遗忘机制用于设备端个性化智能体记忆

Jiale Chang, Yuxiang Ren

发表机构 * Nanjing Agricultural University(南京农业大学) Nanjing University(南京大学)

AI总结 提出ScrapMem框架,通过光学遗忘机制压缩旧记忆并构建情节记忆图,在资源受限设备上实现高效多模态长期记忆,在ATM-Bench上取得51.0% Joint@10新最优,存储降低93%,召回率提升至70.3%。

Comments 10 pages, 4 figures

详情
AI中文摘要

对于LLM智能体而言,在资源受限的边缘设备上实现长期个性化记忆因高存储成本和多模态复杂性而具有挑战性。为此,我们提出ScrapMem,一个将多模态数据整合为“剪贴簿页面”的框架。ScrapMem引入了光学遗忘机制,一种逐步降低旧记忆分辨率的光学压缩机制,从而降低存储成本并抑制低价值细节。为保持语义一致性,我们构建了情节记忆图(EM-Graph),将关键事件组织成因果-时间结构。在多模态ATM-Bench上的大量实验表明,ScrapMem提供了三个主要优势:(1)强大性能,以51.0%的Joint@10分数实现了新的最优结果;(2)高存储效率,通过光学遗忘将内存使用量降低高达93%;(3)改进的召回率,通过结构化聚合将Recall@10提升至70.3%。ScrapMem为多模态LLM智能体在设备上的长期记忆提供了一种有效且存储高效的解决方案。

英文摘要

Long-term personalized memory for LLM agents is challenging on resource-limited edge devices due to high storage costs and multimodal complexity. To address this, we propose ScrapMem, a framework that integrates multimodal data into "Scrapbook Page." ScrapMem introduces Optical Forgetting, an optical compression mechanism that progressively reduces the resolution of older memories, lowering storage cost while suppressing low-value details. To maintain semantic consistency, we construct an Episodic Memory Graph (EM-Graph) that organizes key events into a causal-temporal structure. Extensive experiments on the multimodal ATM-Bench showcase that ScrapMem provides three main benefits: (1) strong performance, achieving a new state-of-the-art with a 51.0% Joint@10 score; (2) high storage efficiency, reducing memory usage by up to 93% via optical forgetting; and (3) improved recall, increasing Recall@10 to 70.3% through structured aggregation. ScrapMem offers an effective and storage-efficient solution for on-device long-term memory in multimodal LLM agents.

2605.03675 2026-05-26 cs.AI

MEMTIER: Tiered Memory Architecture and Retrieval Bottleneck Analysis for Long-Running Autonomous AI Agents

MEMTIER:面向长期运行的自主AI智能体的分层内存架构与检索瓶颈分析

Bronislav Sidik, Lior Rokach

发表机构 * Institute for Applied AI Research(应用人工智能研究所) Faculty of Computer and Information Science(计算机与信息科学学院) Ben-Gurion University of the Negev(贝内尔-加里翁大学)

AI总结 提出MEMTIER三层内存架构,通过结构化事件存储、五信号加权检索、注意力归因权重更新、异步合并机制和PPO策略,在LongMemEval-S基准上将全上下文基线准确率从5%提升至38%,并支持本地6GB GPU运行。

Comments 11 pages, 1 figure, 5 tables. Under review

详情
AI中文摘要

长期运行的自主AI智能体面临一个记录充分的内存一致性问题:由于现有平面文件内存系统中的四种复合故障模式,工具执行成功率在72小时运行窗口内下降14个百分点。我们提出MEMTIER,这是OpenClaw智能体运行时的三方内存架构,引入了结构化事件JSONL存储、五信号加权检索引擎、注意力归因的认知权重更新循环、将事件事实提升到语义层的异步合并守护进程,以及基于PPO的检索权重自适应策略框架(基础设施已验证;性能提升待最终版本确认)。在完整的500问题LongMemEval-S基准测试(Wu等人,2025)上,MEMTIER在消费级6GB GPU上使用Qwen2.5-7B达到Acc=0.382,F1=0.412——比全上下文基线(0.050 -> 0.382,即5% -> 38%)提高了33个百分点。通过DeepSeek-V4-Flash事实预填充,单会话召回率达到0.686-0.714,超过了论文中RAG BM25 GPT-4o基线(0.560)在这些类别上的表现。时间推理提升至0.323,多会话综合提升至0.173,表明结构化语义预填充从根本上改变了轻量级检索所能达到的效果。所有阶段均在配备6GB GPU的消费级笔记本电脑上本地运行。

英文摘要

Long-running autonomous AI agents suffer from a well-documented memory coherence problem: tool-execution success rates degrade 14 percentage points over 72-hour operation windows due to four compounding failure modes in existing flat-file memory systems. We present MEMTIER, a tripartite memory architecture for the OpenClaw agent runtime that introduces a structured episodic JSONL store, a five-signal weighted retrieval engine, an attention-attributed cognitive weight update loop, an asynchronous consolidation daemon promoting episodic facts to a semantic tier, and a PPO-based policy framework for adapting retrieval weights (infrastructure validated; performance gains pending camera-ready). On the full 500-question LongMemEval-S benchmark (Wu et al., 2025), MEMTIER achieves Acc=0.382, F1=0.412 with Qwen2.5-7B on a consumer 6GB GPU - a +33 percentage point improvement over the full-context baseline (0.050 -> 0.382, i.e., 5% -> 38%). With DeepSeek-V4-Flash fact pre-population, single-session recall reaches 0.686-0.714, exceeding the paper's RAG BM25 GPT-4o baseline (0.560) on those categories. Temporal reasoning rises to 0.323 and multi-session synthesis to 0.173, demonstrating that structured semantic pre-population qualitatively changes what lightweight retrieval can achieve. All phases run locally on a consumer laptop with a 6GB GPU.

2605.03472 2026-05-26 cs.CL cs.AI

Auditing Stealth Sycophancy in Mental-Health Dialogue: Structured Clinical-State Diagnostics and Clean Matched Benchmarks

审计心理健康对话中的隐性谄媚:结构化临床状态诊断与干净匹配基准

Tianze Han, Beining Xu, Hanbo Zhang, Yongming Lu

发表机构 * Shenzhen MSU-BIT University(深圳MSU-BIT大学)

AI总结 针对心理健康对话模型中隐式谄媚(表面共情但强化消极认知)的问题,提出基于动态情感签名图(DESG)的结构化离线审计框架,通过临床状态转移评估响应方向,并在干净匹配基准上实现最优有害风险检测。

详情
AI中文摘要

心理健康对话模型越来越多地由基于AI的评估器进行评估,但这些评估器通常将表面共情、支持性或流畅性视为安全的证据。在本文中,我们研究了一种隐藏的失败模式,称为隐式谄媚:一个响应可能看似共情,但暗中强化灾难化、回避、绝望预测或CBT式标签。为了检查这个问题,我们引入了一个用于隐式谄媚检测的诊断基准,该基准基于三个代表性的心理健康对话来源构建,涵盖日常同伴支持、咨询式情感支持和危机导向互动,并进一步构建了一个泄漏审计的干净单响应匹配基准,包含500个上下文和1500个匹配响应窗口。然后,我们提出了动态情感签名图(DESG),一个结构化的离线审计框架,将基于LLM的状态提取与最终评分分离,并通过语义、情感和认知扭曲状态转移而非自由形式的LLM判断来评估临床方向。与元数据、表面风格、词汇、嵌入和基于规则的LLM基线不同,DESG对响应引起的临床状态变化方向进行评分;在泄漏审计的干净匹配基准上,DESG-StateRisk比最强的非DESG基线提高了0.0488 macro-F1,并实现了最佳的有害风险检测结果。这些结果表明,评估隐式谄媚需要显式的临床状态建模以及泄漏检查、捷径控制和竞争性基线。

英文摘要

Mental-health dialogue models are increasingly evaluated by AI-based evaluators, yet these evaluators often treat surface empathy, supportiveness, or fluency as evidence of safety. In this paper, we study a hidden failure mode that we call implicit sycophancy: a response may appear empathetic while implicitly reinforcing catastrophizing, avoidance, hopeless prediction, or CBT-style labeling. To examine this problem, we introduce a diagnostic benchmark for implicit-sycophancy detection, built from three representative mental-health dialogue sources covering everyday peer support, counseling-style emotional support, and crisis-oriented interaction, and further construct a leakage-audited clean single-response matched benchmark with 500 contexts and 1,500 matched response windows. We then propose Dynamic Emotional Signature Graphs (DESG), a structured offline audit framework that separates LLM-based state extraction from final scoring and evaluates clinical direction through semantic, affective, and cognitive-distortion state transitions rather than free-form LLM judgment. Unlike metadata, surface-style, lexical, embedding, and rubric-LLM baselines, DESG scores the direction of clinical-state change induced by a response; on the leakage-audited clean matched benchmark, DESG-StateRisk improves over the strongest non-DESG baseline by 0.0488 macro-F1 and achieves the best harmful-risk detection result. These results suggest that evaluating implicit sycophancy requires explicit clinical-state modeling together with leakage checks, shortcut controls, and competitive baselines.

2605.02764 2026-05-26 cs.CV

FoR-Net: Learning to Focus on Hard Regions for Efficient Semantic Segmentation

FoR-Net:学习聚焦困难区域以实现高效语义分割

Sheng-Wei Chan, Hsin-Jui Pan, Chun-Po Shen, Yung-Che Wang, Meng-Qian Li, Chia-Min Lin, Jen-Shiun Chiang

发表机构 * Department of Electrical and Computer Engineering, Tamkang University(东海大学电子与计算机工程系)

AI总结 提出FoR-Net框架,通过可学习的重要性图和Top-K激活机制聚焦困难区域(如细长结构和物体边界),在有限计算资源下实现高效语义分割。

Comments 9 pages, 2 figures, 2 tables. Efficient semantic segmentation under resource-constrained settings. Code will be released

详情
AI中文摘要

我们提出FoR-Net,一种高效的语义分割框架,专注于识别和增强困难区域。FoR-Net不依赖沉重的全局建模,而是采用一种高效策略,通过可学习的重要性图和Top-K激活机制选择性强调信息丰富的区域。具体来说,选择器模块预测区域重要性,使模型能够聚焦于挑战性区域,如细长结构和物体边界。使用不同感受野的卷积分支实现多尺度推理,允许多样化的空间上下文聚合。我们在有限计算资源下对Cityscapes基准评估FoR-Net。尽管其设计高效且训练配置标准,FoR-Net仍取得了有竞争力的性能,并表现出对困难区域的改进关注。这些结果表明,选择性区域聚焦推理可以作为语义分割的一种实用且高效的替代方案。本工作探索了资源受限环境下的区域聚焦推理,并为开发高效且区域感知的分割模型提供了见解。

英文摘要

We present FoR-Net, an efficient semantic segmentation framework that focuses on identifying and enhancing hard regions. Instead of relying on heavy global modeling, FoR-Net adopts an efficient strategy that selectively emphasizes informative regions through a learned importance map and a Top-K activation mechanism. Specifically, a selector module predicts region-wise importance, enabling the model to focus on challenging areas such as thin structures and object boundaries. Multi-scale reasoning is achieved using convolutional branches with different receptive fields, allowing diverse spatial context aggregation. We evaluate FoR-Net on the Cityscapes benchmark under limited computational resources. Despite its efficient design and standard training configuration, FoR-Net achieves competitive performance and exhibits improved attention to difficult regions. These results suggest that selective region-focused reasoning can serve as a practical and efficient alternative for semantic segmentation. This work explores region-focused reasoning under resource-constrained settings and provides insights for developing efficient and region-aware segmentation models.

2605.02495 2026-05-26 cs.LG cs.AI stat.ML

Efficient Preference Poisoning Attack on Offline RLHF

高效偏好投毒攻击离线RLHF

Chenye Yang, Weiyu Xu, Lifeng Lai

发表机构 * Department of Electrical and Computer Engineering, University of California, Davis, Davis, CA, USA(加州大学戴维斯分校电气与计算机工程系) Department of Electrical and Computer Engineering, University of Iowa, Iowa City, IA, USA(爱荷华大学电气与计算机工程系)

AI总结 针对离线RLHF中的偏好投毒攻击,提出基于梯度字典的二进制稀疏近似方法(BAL-A和BMP-A),实现高效标签翻转攻击。

Comments Accepted to ICML 2026

详情
AI中文摘要

离线人类反馈强化学习(RLHF)流程(如直接偏好优化DPO)在预收集的偏好数据集上训练,使其容易受到偏好投毒攻击。我们研究了对数线性DPO的标签翻转攻击。首先说明翻转一个偏好标签会在DPO梯度中引起与参数无关的偏移。利用这一关键性质,我们可以将目标投毒问题转化为结构化的二进制稀疏近似问题。为解决该问题,我们开发了两种攻击方法:二进制感知格点攻击(BAL-A)和二进制匹配追踪攻击(BMP-A)。BAL-A将二进制翻转选择问题嵌入二进制感知格点,并应用Lenstra-Lenstra-Lovász约简和Babai最近平面算法;我们提供了强制二进制系数并恢复最小翻转目标的充分条件。BMP-A将二进制匹配追踪适应于我们的非归一化梯度字典,并给出基于相干性的恢复保证和$K$翻转预算的鲁棒性(不可能性)证书。在合成字典和斯坦福人类偏好数据集上的实验验证了理论,并突出了字典几何如何决定攻击成功。

英文摘要

Offline Reinforcement Learning from Human Feedback (RLHF) pipelines such as Direct Preference Optimization (DPO) train on a pre-collected preference dataset, which makes them vulnerable to preference poisoning attack. We study label flip attacks against log-linear DPO. We first illustrate that flipping one preference label induces a parameter-independent shift in the DPO gradient. Using this key property, we can then convert the targeted poisoning problem into a structured binary sparse approximation problem. To solve this problem, we develop two attack methods: Binary-Aware Lattice Attack (BAL-A) and Binary Matching Pursuit Attack (BMP-A). BAL-A embeds the binary flip selection problem into a binary-aware lattice and applies Lenstra-Lenstra-Lovász reduction and Babai's nearest plane algorithm; we provide sufficient conditions that enforce binary coefficients and recover the minimum-flip objective. BMP-A adapts binary matching pursuit to our non-normalized gradient dictionary and yields coherence-based recovery guarantees and robustness (impossibility) certificates for $K$-flip budgets. Experiments on synthetic dictionaries and the Stanford Human Preferences dataset validate the theory and highlight how dictionary geometry governs attack success.

2605.01017 2026-05-26 cs.CL

Psychologically Potent, Computationally Invisible: LLMs Generate Social-Comparison-Eliciting Posts They Fail to Detect

心理上有效,计算上不可见:LLM 生成引发社会比较的帖子但无法检测

Hua Zhao, Jiapei Gu, Michelle Mingyue Gu

发表机构 * Department of English Language Education(英语语言教育系) Analytics/Assessment Research Centre(分析/评估研究中心)

AI总结 本研究通过构建小红书社会比较读者诱发基准(XHS-SCoRE),发现 LLM 在生成引发社会比较的帖子时存在生成-检测不匹配,即该信号在领域内可学习但无法通过提示分类稳健访问。

Comments 19 pages, preprint Title change: Psychologically Potent, Computationally Invisible: LLMs Generate Social-Comparison-Eliciting Posts They Fail to Detect

详情
AI中文摘要

我们引入了小红书社会比较读者诱发基准(XHS-SCoRE),这是一个基于读者视角的基准,用于检测纯文本小红书(RedNote)帖子是否从第一人称读者视角引发向上、向下或中性/无明确社会比较。该任务针对一种具有社会意义的关系性、行为上真实的信号,该信号不可简化为情感。在提示型 LLM 分类器和有监督的中文编码器中,我们发现了一致的生成-检测不匹配:该信号在领域内是文本可学习的,但无法通过基于提示的分类稳健访问。提示型 LLM 分类器表现出稳定的失败,特别是对引发比较的帖子进行中和以及模型特定的方向偏差。一项受控试点表明,即使基于提示的同一构念检测仍然脆弱,LLM 生成的小红书风格帖子也能改变感知地位和比较相关情感。XHS-SCoRE 为基于读者的比较检测提供了一个基准,并为研究社会意义的关系线索何时仅部分可见于基于提示的推理提供了一个诊断框架。

英文摘要

We introduce Xiaohongshu Social Comparison Reader Elicitation (XHS-SCoRE), a reader-grounded benchmark for detecting whether text-only Xiaohongshu (RedNote) posts elicit Upward, Downward, or Neutral/no clear social comparison from a first-person reader perspective. The task targets a socially meaningful relational, behaviorally real signal not reducible to sentiment. Across prompted LLM classifiers and supervised Chinese encoders, we find a consistent generation--detection mismatch: the signal is textually learnable in-domain, but not robustly accessible to prompt-based classification. Prompted LLM classifiers show stable failures, especially neutralization of comparison-eliciting posts and model-specific directional skew. A controlled pilot shows that LLM-generated Xiaohongshu-style posts can shift perceived standing and comparison-related affect even when prompt-based detection of the same construct remains fragile. XHS-SCoRE contributes a benchmark for reader-grounded comparison detection and a diagnostic framework for studying when socially meaningful relational cues remain only partially visible to prompt-based inference.

2605.00419 2026-05-26 cs.LG cs.CL

Rethinking LLM Ensembling from the Perspective of Mixture Models

从混合模型的角度重新思考大语言模型集成

Jiale Fu, Yuchu Jiang, Peijun Wu, Chonghan Liu, Joey Tianyi Zhou, Xu Yang

发表机构 * Key Laboratory of New Generation Artificial Intelligence Technology(新一代人工智能技术关键实验室) Its Interdisciplinary Applications (Southeast University), Ministry of Education(交叉应用(东南大学),教育部) Southeast University(东南大学) Centre for Frontier AI Research (CFAR), Agency for Science, Technology(前沿人工智能研究(CFAR),科技研究局) Research (A STAR), Singapore(研究(A STAR),新加坡) Institute of High Performance Computing (IHPC), Agency for Science, Technology(高性能计算(IHPC),科技研究局)

AI总结 本文提出混合模型式集成(ME),通过将集成重新解释为混合模型,随机选择单个模型生成下一个token,避免显式计算完整集成分布,实现1.78x-2.68x加速,并揭示了集成与token级路由方法的联系。

Comments ICML 2026 Spotlight

详情
AI中文摘要

模型集成是提升机器学习模型性能的成熟技术。传统上,这涉及对多个模型的输出分布进行平均,并选择最可能的标签。这一思想已自然扩展到大型语言模型(LLMs),在提升性能的同时也带来了巨大的计算成本。这种低效源于将传统集成实现直接应用于LLMs,需要为每个模型单独进行前向传播以显式计算集成分布。在本文中,我们提出了混合模型式集成(ME)。通过将集成重新解释为混合模型,ME在每一步随机选择一个模型来生成下一个token,从而避免显式计算完整的集成分布。ME在数学上等价于从集成分布中采样,但只需调用一个模型,使其比传统集成快1.78x-2.68x倍。此外,这一视角将LLM集成与token级路由方法联系起来,表明LLM集成是路由方法的一个特例。我们的发现为高效的LLM集成开辟了新途径,并激励了对LLM token级路由策略的进一步探索。我们的代码可在https://github.com/Kamichanw/Mixture-model-like-Ensemble获取。

英文摘要

Model ensembling is a well-established technique for improving the performance of machine learning models. Conventionally, this involves averaging the output distributions of multiple models and selecting the most probable label. This idea has been naturally extended to large language models (LLMs), yielding improved performance but incurring substantial computational cost. This inefficiency stems from directly applying conventional ensemble implementation to LLMs, which require a separate forward pass for each model to explicitly compute the ensemble distribution. In this paper, we propose the Mixture-model-like Ensemble (ME). By reinterpreting the ensemble as a mixture model, ME stochastically selects a single model at each step to generate the next token, thereby avoiding the need to explicitly compute the full ensemble distribution. ME is mathematically equivalent to sampling from the ensemble distribution, but requires invoking only one model, making it 1.78x-2.68x faster than conventional ensembling. Furthermore, this perspective connects LLM ensembling and token-level routing methods, suggesting that LLM ensembling is a special case of routing methods. Our findings open new avenues for efficient LLM ensembling and motivate further exploration of token-level routing strategies for LLMs. Our code is available at https://github.com/Kamichanw/Mixture-model-like-Ensemble.

2604.23853 2026-05-26 cs.AI

ClawTrace: Cost-Aware Tracing for LLM Agent Skill Distillation

ClawTrace: 面向LLM智能体技能蒸馏的成本感知追踪

Boqin Yuan, Yue Su, Renchu Song, Sen Yang, Jing Qin

发表机构 * University of California San Diego(加州大学圣地亚哥分校) Carnegie Mellon University(卡内基梅隆大学)

AI总结 针对技能蒸馏管道缺乏每步成本信号的问题,提出ClawTrace记录成本归因轨迹并生成TraceCard,通过CostCraft生成保留、剪枝和修复三类技能补丁,发现剪枝补丁作为质量护栏而保留补丁导致回归,主张按规则类型评估可复用技能。

Comments Accepted at Agent Skills '26 Workshop, ACM Conference on AI and Agentic Systems (CAIS 2026), San José, CA, May 26, 2026

详情
AI中文摘要

技能蒸馏管道从LLM智能体轨迹中学习可重用规则,但它们缺乏一个关键信号:每一步的成本。没有每步成本,管道无法区分添加缺失步骤以修复错误与移除从未影响结果的昂贵步骤。我们利用成本归因差距来探究蒸馏技能内部的规则类型是否以相同方式迁移到新任务。ClawTrace记录成本归因的智能体轨迹,并将每个会话编译成TraceCard;CostCraft读取TraceCard并编写三种技能补丁:保留、剪枝和修复。我们发现了一个聚合指标隐藏的模式。在30个保留的SpreadsheetBench任务上(两个种子),移除剪枝补丁大致使质量回归计数增加了三倍,而未降低中位成本。在整个84任务的SkillsBench迁移中,CostCraft未节省总成本。所有三个质量回归都追溯到保留通道,而两个质量提升都追溯到剪枝通道:剪枝补丁充当质量护栏,而保留补丁驱动回归。我们认为可重用的智能体技能应在规则类型层面进行评估,而不是作为整体指令包。为支持这一点,我们发布了ClawTrace、TraceCard模式以及全套类型化技能。

英文摘要

Skill-distillation pipelines learn reusable rules from LLM agent trajectories, but they lack a key signal: how much each step costs. Without per-step cost, a pipeline cannot distinguish adding a missing step to fix a bug from removing an expensive step that never affected the outcome. We use the cost-attribution gap to ask whether the rule types inside a distilled skill transfer the same way to new tasks. ClawTrace records cost-attributed agent traces and compiles each session into a TraceCard; CostCraft reads TraceCards and writes three kinds of skill patches: preserve, prune, and repair. We find a pattern aggregate metrics hide. On 30 held-out SpreadsheetBench tasks across two seeds, removing prune patches roughly tripled the quality-regression count without lowering median cost. Across the full 84-task SkillsBench transfer, CostCraft saves no aggregate cost. All three quality regressions trace to the preserve lane, and both quality wins trace to the prune lane: prune patches act as quality guardrails while preserve patches drive regressions. We argue that reusable agent skills should be evaluated at the rule-type level, not as monolithic instruction packages. To support this, we release ClawTrace, the TraceCard schema, and the full set of typed skills.

2604.23728 2026-05-26 cs.CV cs.AI

ESIA: An Energy-Based Spatiotemporal Interaction-Aware Framework for Pedestrian Intention Prediction

ESIA:基于能量的时空交互感知框架用于行人意图预测

Yanping Wu, Meiting Dang, Lin Wu, Edmond S. L. Ho, Zhenghua Chen, Chongfeng Wei

发表机构 * James Watt School of Engineering, University of Glasgow(格拉斯哥大学詹姆斯·瓦特工程学院)

AI总结 提出ESIA框架,利用条件随机场和能量函数建模时空交互,通过结构一致性约束和模拟退火算法实现行人意图预测,在标准基准上达到最先进性能并提升可解释性。

Comments 13 pages, 6 figures, 3 tables

详情
AI中文摘要

自动驾驶的最新进展推动了行人意图预测的研究,该研究旨在通过建模时间动态、社交互动和环境背景来推断未来的过街决策和行动。然而,现有研究仍受限于过度简化的多智能体交互模式、不透明的推理逻辑以及行为预测中缺乏全局一致性,这损害了鲁棒性和可解释性。在这项工作中,我们提出了ESIA(基于能量的时空交互感知框架),一种新颖的基于条件随机场(CRF)的范式。我们将意图预测任务视为一个基于统一图表示的结构化预测问题,将行人和环境视为时空节点。为了表征它们的不同角色,我们为节点分配一元势能以捕捉个体意图,为边分配成对势能以编码社交和环境交互。这些势能被整合到一个统一的全局能量函数中,以确保行为预测的场景级一致性。为了在没有真实标签监督的情况下进一步约束推理,我们引入了结构一致性项来惩罚逻辑矛盾。该优化通过一种新颖的一元种子模拟退火(U-SSA)算法高效求解,该算法利用高置信度的一元先验快速收敛到高质量解。在标准基准上的大量实验表明,ESIA在现有方法中实现了最先进的性能,并具有更好的可解释性。

英文摘要

Recent advances in autonomous driving have motivated research on pedestrian intention prediction, which aims to infer future crossing decisions and actions by modeling temporal dynamics, social interactions, and environmental context. However, existing studies remain constrained by oversimplified multi-agent interaction patterns, opaque reasoning logic, and a lack of global consistency in behavioral predictions, which compromise both robustness and interpretability. In this work, we propose ESIA (Energy-based Spatiotemporal Interaction-Aware framework), a novel Conditional Random Field (CRF)-based paradigm. We cast the intention prediction task as a structured prediction problem over a unified graph-based representation, treating pedestrians and the environment as spatiotemporal nodes. To characterize their distinct roles, we assign unary potentials to nodes to capture individual intentions, and pairwise potentials to edges to encode social and environmental interactions. These potentials are integrated into a unified global energy function to ensure scene-level consistency across behavioral predictions. To further constrain inference without ground-truth supervision, we introduce structural consistency terms to penalize logical contradictions. This optimization is efficiently solved via a novel Unary-Seeded Simulated Annealing (U-SSA) algorithm, which leverages high-confidence unary priors to rapidly converge to a high-quality solution. Extensive experiments on standard benchmarks demonstrate that ESIA achieves state-of-the-art performance with improved interpretability over existing methods.