arXivDaily arXiv每日学术速递 周一至周五更新
重置
全部学科分类 3947
2606.16847 2026-06-16 cs.CL cs.AI 新提交

Follow the Latent Roadmap: Navigating Revocable Decoding for Diffusion LLMs with Anchor Tokens

遵循潜在路径:利用锚定令牌导航扩散LLM的可撤销解码

Yizhen Yao, Qinglin Zhu, Runcong Zhao, Xiangxiang Dai, Yanzheng Xiang, Yulan He, Lin Gui

发表机构 * King's College London(伦敦国王学院) The Chinese University of Hong Kong(香港中文大学) The Alan Turing Institute, UK(英国艾伦·图灵研究所)

AI总结 针对扩散大语言模型解码速度与质量的权衡,提出无训练框架ASRD,通过锚定令牌解耦上下文,结合锚定引导生成与锚定扰动验证,在数学和编码基准上提升准确率6.4%,加速推理7.2倍。

详情
AI中文摘要

扩散大语言模型(dLLMs)为并行生成提供了有前景的途径,但面临解码速度与质量之间的权衡。虽然可撤销解码策略尝试通过验证和重新掩码来减轻错误,但它们通常在混合质量上下文中操作。这导致两个关键失败:\textit{错误传播},即新令牌从错误上下文中吸收有毒信息;以及\textit{局部错误强化},即错误相互强化以逃避检测。为缓解这些挑战,我们提出ASRD(锚定监督可撤销解码),一种在嵌入空间内运行的无训练框架。ASRD明确将解码上下文解耦为通过时间一致性识别的可信\textit{锚定令牌}和不确定候选令牌。利用动态锚定令牌缓存,我们引入两种互补机制:(1)锚定引导生成,将熵加权锚定信号注入掩码位置,以隐式地将注意力引导向可靠的全局骨架;(2)锚定扰动验证,对不确定候选令牌施加正交扰动,破坏并重新掩码由脆弱局部共识驱动的错误。在数学和编码基准上的大量实验表明,ASRD优于最近的重新掩码基线,准确率提升高达6.4%,同时推理吞吐量加速高达7.2倍。

英文摘要

Diffusion Large Language Models (dLLMs) offer a promising avenue for parallel generation but face a trade-off between decoding speed and quality. While revocable decoding strategies attempt to mitigate errors by verifying and remasking tokens, they typically operate within a mixed-quality context. This leads to two critical failures: \textit{Error Propagation}, where new tokens absorb toxic information from erroneous context, and \textit{Local Error Reinforcement}, where errors mutually reinforce each other to evade detection. To alleviate these challenges, we propose ASRD (Anchor Supervised Revocable Decoding), a training-free framework that operates within the embedding space. ASRD explicitly decouples the decoding context into trusted \textit{Anchor Tokens}, which are identified via temporal consistency, and uncertain candidates. Leveraging a dynamic Anchor Tokens Cache, we introduce two complementary mechanisms: (1) Anchor-Guided Generation, which injects entropy-weighted anchor signals into masked positions to implicitly rectify attention toward the reliable global skeleton; and (2) Anchor-Perturbed Verification, which applies orthogonal perturbations to uncertain candidate tokens, destabilizing and remasking errors driven by fragile local consensus. Extensive experiments on math and coding benchmarks demonstrate that ASRD outperforms recent remasking baselines, achieving accuracy improvements of up to 6.4\% while accelerating inference throughput by up to 7.2$\times$.

2606.16846 2026-06-16 cs.LG cs.AI 新提交

Deep Q-Learning on Hölder Spaces

Hölder空间上的深度Q学习

Qian Qi

发表机构 * Peking University(北京大学)

AI总结 研究连续时间随机控制中Q学习的算子核心,通过分析扩散设置下Bellman最优性目标的正则性和逼近复杂度,提出适应混合正则性的张量积DeepONet架构,并给出显式逼近和资源界限。

详情
AI中文摘要

我们研究了具有连续状态和动作的连续时间随机控制中Q学习的算子理论核心。在基于价值的强化学习中,每次Q学习或DQN更新都基于Bellman最优性目标;我们的分析在扩散设置中分离出该目标,并研究其正则性和逼近复杂度。在均匀椭圆性和Hölder正则系数下,我们证明Bellman更新将有界输入映射到各向异性正则类,平滑状态变量而仅保留对动作变量的Lipschitz依赖性。这产生了Bellman迭代的紧族,并激发了适应问题混合正则性的张量积DeepONet架构。然后我们推导出显式的逼近和资源界限,以及时间步长$δ\ o 0$时的刚度-复杂度权衡。所得理论在连续随机控制中Bellman目标正则性和逼近层面直接贡献于Q学习理论。同时,我们并未声称对包含探索、经验回放和随机梯度更新的实际采样Q学习有完整的收敛定理。

英文摘要

We study the operator-theoretic core of Q-learning in continuous-time stochastic control with continuous states and actions. In value-based reinforcement learning, each Q-learning or DQN update is built from a Bellman optimality target; our analysis isolates this target in a diffusion setting and studies its regularity and approximation complexity. Under uniform ellipticity and Hölder-regular coefficients, we show that a Bellman update maps bounded inputs into an anisotropic regularity class, smoothing the state variable while leaving only Lipschitz dependence on the action variable. This yields a compact family of Bellman iterates and motivates a tensor-product DeepONet architecture adapted to the mixed regularity of the problem. We then derive explicit approximation and resource bounds, together with a stiffness--complexity trade-off as the time step $δ\to 0$. The resulting theory makes a direct contribution to Q-learning theory at the level of Bellman target regularity and approximation in continuous stochastic control. At the same time, we do not claim a full convergence theorem for practical sampled Q-learning with exploration, replay, and stochastic gradient updates.

2606.16845 2026-06-16 cs.CL cs.AI 新提交

Robust Dual-Signal Fusion: Hybrid Neuro-Symbolic Gating with Compressed Chain-of-Thought Refinement for Irony Detection in Social Media Texts

鲁棒双信号融合:混合神经符号门控与压缩链式思维精炼用于社交媒体文本讽刺检测

Ankit Bhattacharjee, Krityapriya Bhaumik

发表机构 * Indian Institute of Technology Kharagpur(印度理工学院克勒格布尔分校)

AI总结 提出RDS融合框架,结合神经符号架构与压缩链式思维推理,在TweetEval和iSarcasm数据集上达到与微调BERTweet相当的性能,并显著优于监督方法。

Comments 11 pages total, 10 figures

详情
AI中文摘要

大型语言模型(LLM)默认倾向于字面语义解释,使得零样本讽刺检测成为一个持续的挑战。我们引入了鲁棒双信号(RDS)融合框架,这是一种混合神经符号架构,无需监督微调(SFT)即可压缩链式思维(CoT)推理轨迹。在严格保留的TweetEval测试集(N=734)上,RDS达到了78.1%的准确率和0.777的宏F1分数,与微调BERTweet的绝对性能上限相匹配。在高度不平衡的iSarcasm数据集上,冻结的CoT管道过滤了22.5%的分布外幻觉,实现了0.6726的零样本宏F1和0.4821的讽刺F1,优于多个强监督的SemEval Transformer集成。统计消融实验证实了这种结构协同作用:将符号先验添加到神经基线没有显著提升(p=0.242),而将CoT管道添加到该先验的边际收益被高度压缩(p=0.149)。只有所有三个信号的完整并发融合才能实现相对于基线的统计验证改进(p=0.005)。

英文摘要

Large Language Models (LLMs) natively default to literal semantic interpretations, making zero-shot irony detection a persistent challenge. We introduce the Robust Dual-Signal (RDS) Fusion framework, a hybrid neuro-symbolic architecture that compresses Chain-of-Thought (CoT) reasoning trajectories without Supervised Fine-Tuning (SFT). Evaluated on a strictly held-out TweetEval test set (N=734), RDS achieves 78.1% accuracy and a Macro F1 of 0.777, matching the absolute performance ceiling of the fine-tuned BERTweet. On the heavily imbalanced iSarcasm dataset, the frozen CoT pipeline filters 22.5% of out-of-distribution hallucinations, yielding a zero-shot Macro F1 of 0.6726 and Ironic F1 of 0.4821, outperforming multiple heavily supervised SemEval transformer ensembles. A statistical ablation confirms this structural synergy: adding the symbolic prior to the neural baseline yields no significant gain (p = 0.242), and the marginal benefit of adding the CoT pipeline to that prior is heavily compressed (p = 0.149). Only the complete, concurrent fusion of all three signals achieves a statistically validated improvement over the baseline (p = 0.005).

2606.16843 2026-06-16 cs.CL 新提交

Data-Driven Decoding of Russell's Circumplex Model of Affect

基于数据驱动的Russell情感环状模型解码

Amdjed Belaref, Samir Sadok, Zineb Noumir, Renaud Seguier

发表机构 * Alten CentraleSupélec IETR UMR CNRS 6164(中央理工-高等电力学院 IETR CNRS 6164 联合研究单位) Inria at Univ. Grenoble Alpes, CNRS, LJK(法国国家信息与自动化研究所,格勒诺布尔阿尔卑斯大学,CNRS,LJK)

AI总结 本文研究Transformer嵌入是否恢复Russell环状模型的几何规律,通过文本和语音模型实验,发现多模态融合完美对齐情感排序,零样本下细粒度情感词接近人类映射坐标。

Comments This work has been submitted to the IEEE for possible publication

详情
AI中文摘要

情感计算日益依赖深度学习来表示情感,然而潜在空间通常是不透明的高维黑箱。本文研究Transformer的嵌入是否恢复Russell环状模型的几何规律。我们统一了两个互补实验,检验以下假设:在文本和语音上训练模型后,其潜在空间编码了与效价-唤醒一致的拓扑结构,并再现了类似人类的邻域关系。具体而言,我们评估了基于Transformer的文本(RoBERTa)和语音(wav2vec 2.0)编码器以及多模态Transformer融合架构提取的深度表示,使用了MSP-Podcast等自然数据集和受控的LLM生成刺激。我们的分析表明,文本和音频的多模态融合与Russell的主要情感排序实现了完美的拓扑对齐。此外,在零样本设置中,使用通用文本嵌入,投影的细粒度情感术语接近其已建立的人类映射坐标。我们的贡献是一个新颖的数据驱动框架,用于验证情感模型,证明Russell环状结构内在地编码于这些模态的嵌入中,而不仅仅是人类标注的产物,从而弥合了心理学理论与表示学习之间的差距。

英文摘要

Affective computing increasingly relies on deep learning to represent emotions, yet latent spaces often remain opaque, high-dimensional black boxes. This paper investigates whether Transformers' embeddings recover the geometric regularities of Russell's circumplex model. We unify two complementary experiments testing the hypothesis that, after training models on text and speech, their resulting latent spaces encode a topology consistent with valence-arousal and reproduce human-like neighborhood relations. Specifically, we evaluate deep representations extracted from Transformer-based text (RoBERTa) and speech (wav2vec 2.0) encoders, along with a multimodal Transformer fusion architecture, across naturalistic datasets like MSP-Podcast and controlled LLM-generated stimuli. Our analysis reveals that multimodal fusion of text and audio yields perfect topological alignment with Russell's primary emotion ordering. Furthermore, in a zero-shot setting using generic text embeddings, projected fine-grained emotion terms fall close to their established human-mapped coordinates. Our contribution is a novel, data-driven framework for validating emotion models, demonstrating that Russell's circumplex structure is intrinsically encoded in the embeddings of these modalities rather than being solely an artifact of human labeling, thereby bridging the gap between psychological theory and representation learning.

2606.16837 2026-06-16 cs.CV cs.AI cs.SD 新提交

Robust Spoofed Speech Detection via Temporal Pyramid Modeling

基于时间金字塔建模的鲁棒语音伪造检测

Mahtab Masoudi Nezhad, Nima Karimian

发表机构 * Lane Department of Computer Science and Electrical Engineering, West Virginia University(西弗吉尼亚大学莱恩计算机科学与电气工程系) Bellini College of Artificial Intelligence, Cybersecurity and Computing, University of South Florida(南佛罗里达大学贝利尼人工智能、网络安全与计算学院)

AI总结 提出时间金字塔适配器,通过多尺度时间卷积捕获局部伪影和全局韵律异常,结合自监督XLS-R表示,在多个数据集上显著优于基线模型。

详情
AI中文摘要

伪造语音检测日益受到逼真合成、语音转换和重放攻击的挑战,跨数据集泛化仍然是主要限制。本文提出时间金字塔适配器,利用具有不同感受野的并行时间卷积来捕获多尺度伪造线索,从局部伪影到全局韵律异常。我们还集成了自监督XLS-R表示,并结合前端适配器,包括Mel、Sinc和用于多尺度时间建模的时间金字塔设计。所提出的模型在多个基准上进行了评估,包括ASVspoof 2017、ASVspoof 2021 (DF/LA)、PartialSpoof、DiffSSD和多语言HQ-MPSD数据集。实验结果表明,时间金字塔模型在PartialSpoof数据库上获得了99.24%的AUC和3.87%的EER,显著优于基础模型和多个SOTA基线,如LCNN-BLSTM(9.87% EER)和TRACE(8.08% EER)。此外,多语言评估证实,虽然伪造伪影与语言无关,但自监督表示提高了鲁棒性,在领域和语言偏移下性能下降,凸显了需要更好的适应和校准策略。

英文摘要

Spoofed speech detection is increasingly challenged by realistic synthesis, voice conversion, and replay attacks, with cross-dataset generalization remaining a major limitation. This work we propose a Temporal Pyramid Adapter that utilize parallel temporal convolutions with varying receptive fields to capture multi-scale spoofing cues, ranging from local artifacts to global prosodic irregularities. We also integrated self-supervised XLS-R representations combined with front-end adapters, including Mel, Sinc, and a Temporal Pyramid design for multi-scale temporal modeling. The proposed model is evaluated cross multiple benchmark including ASVspoof 2017, ASVspoof 2021 (DF/LA), PartialSpoof, DiffSSD, and multilingual HQ-MPSD datasets. Experimental results demonstrate that Temporal Pyramid model obtained AUC of 99.24% and a EER of 3.87% on the PartialSpoof database, which is significantly outperforming the base model and several SOTA baseline such as LCNN-BLSTM (9.87% EER) and TRACE (8.08% EER). Additionally, multilingual evaluations confirm that while spoofing artifact are independent from language. While self-supervised representations improve robustness, performance degrades under domain and language shifts, highlighting the need for better adaptation and calibration strategies.

2606.16836 2026-06-16 cs.CL 新提交

Does Traversal Order Matter? A Systematic Study of Tree Traversal Methods in Transformer Grammars

遍历顺序重要吗?Transformer语法中树遍历方法的系统研究

Zongru Liu, Pengyu Ji, Pengcheng Wang, Kewei Tu

发表机构 * School of Information Science and Technology, ShanghaiTech University(上海科技大学信息科学与技术学院)

AI总结 本文系统研究了Transformer语法中不同树遍历方法(深度优先、广度优先及新提出的产生式规则遍历)对语言建模、句法泛化和摘要生成的影响,揭示了嵌套组合与全局前瞻之间的权衡。

详情
AI中文摘要

Transformer语法(TGs)通过融入句法树结构增强了语言建模。尽管句法树在TGs中的线性化方式可能对模型性能产生显著影响,但现有研究仅依赖深度优先遍历(DFT)进行线性化。在本文中,我们通过探索广度优先遍历(BFT)和一种新颖的混合遍历策略——产生式规则遍历(PRT)来扩展遍历设计空间,该策略结合了BFT的结构前瞻性和DFT的早期词汇生成。我们将这些遍历方法与不同的树配置和掩码策略相结合,并在语言建模、句法泛化和摘要生成上实证评估其性能。我们揭示了嵌套组合与全局前瞻之间的固有权衡,为设计任务感知的Transformer语法提供了可操作的建议。

英文摘要

Transformer Grammars (TGs) enhance language modeling by incorporating syntactic tree structures. Despite the potentially significant impact on model performance of how syntactic trees are linearized in TGs, existing studies rely solely on Depth-First Traversal (DFT) for linearization. In this paper, we expand the traversal design space by exploring Breadth-First Traversal (BFT) and a novel hybrid traversal strategy, Production-Rule Traversal (PRT), which combines the structural lookahead of BFT with the early lexical generation of DFT. We integrate these traversal methods with varying tree configurations and masking strategies, and empirically evaluate their performance on language modeling, syntactic generalization and summarization. We reveal the inherent trade-offs between nested composition and global lookahead, providing actionable recommendations for designing task-aware Transformer Grammars.

2606.16826 2026-06-16 cs.RO cs.AI 新提交

ATOM-Bench: A Real-World Benchmark for Atomic Skills and Compositional Generalization in Manipulation Policies

ATOM-Bench:用于操作策略中原子技能与组合泛化的真实世界基准

Zenan Wu, Bingqing Wei, Lu Liu, Zheqi He, Xi Wang, Jiakang Liu, Zehui Li, Guocai Yao, Jing-Shu Zheng, Xi Yang, Yongtao Wang

发表机构 * Beijing Academy of Artificial Intelligence(北京人工智能研究院) Peking University(北京大学)

AI总结 提出ATOM-Bench基准,通过分解桌面操作为原子任务和组合任务,评估操作策略的原子技能获取与组合泛化能力,发现当前策略在细粒度原子技能和组合重用上存在不足。

Comments Homepage: https://flageval-baai.github.io/AtomBenchPage

详情
AI中文摘要

通用操作策略越来越多地被呈现为机器人控制的基础模型,但它们的真实世界泛化能力仍然难以诊断。一个策略可能在演示任务上成功,但仍无法执行细粒度的原子技能或在新的任务结构中重新组合已学习的技能。我们引入了\ extbf{ATOM-Bench},一个用于评估操作策略中原子技能和组合泛化的真实世界基准。ATOM-Bench将桌面操作分解为运动原子和指令原子,包含30个原子任务和24个保留的组合任务,涵盖配对单臂和双臂机器人轨道。我们收集了3000个人类演示用于原子微调,并发布演示数据和评估回滚数据以支持可重复的真实世界评估。策略在原子任务上进行微调,并在原子技能获取和保留的组合任务上进行评估。我们进一步引入了原子分数(AS)和组合失败份额(CFS),以区分由弱原子技能引起的失败和由有限组合重用引起的失败。通过对五种代表性操作策略进行2700次物理回滚,我们发现当前策略可以获取简单的指令接地技能,但在细粒度运动原子、计数和逻辑过滤方面仍然困难。更重要的是,强大的原子性能并不能可靠地迁移到保留的组合任务上。ATOM-Bench提供了一个诊断测试平台,用于研究失败是由弱运动执行、差指令接地还是有限组合重用引起的。

英文摘要

Generalist manipulation policies are increasingly presented as foundation models for robotic control, but their real-world generalization remains difficult to diagnose. A policy may succeed on demonstrated tasks while still failing to execute fine-grained atomic skills or recombine learned skills in new task structures. We introduce \textbf{ATOM-Bench}, a real-world benchmark for evaluating both atomic skills and compositional generalization in manipulation policies. ATOM-Bench factorizes tabletop manipulation into motor atoms and instruction atoms, and contains 30 atomic tasks and 24 held-out compositional tasks across paired single-arm and dual-arm robot tracks. We collect 3,000 human demonstrations for atomic fine-tuning and release both the demonstration data and evaluation rollout data to support reproducible real-world evaluation. Policies are fine-tuned on atomic tasks and evaluated on both atomic skill acquisition and held-out compositional tasks. We further introduce Atomic Score (AS) and Compositional Failure Share (CFS) to distinguish failures caused by weak atomic skills from failures caused by limited compositional reuse. Through 2,700 physical rollouts on five representative manipulation policies, we find that current policies can acquire simple instruction-grounding skills, but still struggle with fine-grained motor atoms, counting, and logical filtering. More importantly, strong atomic performance does not reliably transfer to held-out compositional tasks. ATOM-Bench provides a diagnostic testbed for studying whether failures arise from weak motor execution, poor instruction grounding, or limited compositional reuse.

2606.16825 2026-06-16 cs.CL cs.AI cs.LG 新提交

Tying the Loop -- Tied Expert Layers in Mixture-of-Experts Language Models

循环绑定——混合专家语言模型中的专家层绑定

Martin Jaggi

发表机构 * EPFL(瑞士联邦理工学院洛桑)

AI总结 提出专家绑定方法,通过共享连续Transformer层的专家参数,在保持独立路由和注意力的同时,将MoE模型内存占用降低近2倍,且不损失困惑度或下游性能。

Comments Code available at https://github.com/epfml/looped-moe

详情
AI中文摘要

混合专家(MoE)架构通过每个令牌仅激活一小部分专家来高效扩展大型语言模型(LLM),但全部参数计数——主要由专家参数主导——必须保留在训练和推理内存中。为了解决这个问题,我们引入了专家绑定(Expert Tying),这是一种架构修改,它在连续Transformer层之间共享专家参数,同时保留独立的逐层路由和注意力。我们在常见的先进架构上评估了这种方法,包括OLMoE、Qwen3和DeepSeek风格的MoE。我们的预训练实验表明,绑定专家可以将内存占用减少近2倍,而几乎不降低困惑度或下游质量。通过利用MoE路径中固有的参数冗余,我们的方法提供了高度有利的计算-内存权衡,推动了下一代LLM的高效训练和扩展。

英文摘要

Mixture-of-Experts (MoE) architectures efficiently scale Large Language Models (LLMs) by activating only a small fraction of their experts per token, yet the full parameter count - dominated by the expert parameters - must be held in training and inference memory. To address this, we introduce Expert Tying, an architectural modification that shares expert parameters across consecutive transformer layers while preserving independent, layer-wise routing and attention. We evaluate this approach across common, state-of-the-art architectures, including OLMoE, Qwen3, and DeepSeek-style MoEs. Our pretraining experiments demonstrate that tying experts can reduce memory footprint by almost 2x at virtually no degradation in perplexity or downstream quality. By exploiting the parameter redundancy inherent in MoE pathways, our method provides a highly favorable compute-to-memory trade-off, advancing efficient training and scaling of next-generation LLMs.

2606.16821 2026-06-16 cs.CL cs.CR cs.CY cs.IR 新提交

How Much Can We Trust LLM Search Agents? Measuring Endorsement Vulnerability to Web Content Manipulation

我们能在多大程度上信任LLM搜索智能体?衡量对网络内容操纵的认可脆弱性

Yimeng Chen, Zhe Ren, Firas Laakom, Yu Li, Dandan Guo, Jürgen Schmidhuber

发表机构 * Center of Excellence for Generative AI, KAUST(KAUST生成式人工智能卓越中心) Jilin University(吉林大学) Zhejiang University(浙江大学) The Swiss AI Lab, IDSIA-USI/SUPSI(瑞士人工智能实验室 IDSIA-USI/SUPSI) NNAISENSE

AI总结 提出SearchGEO框架,通过五类攻击模式和输出级指标评估13种LLM后端对网络内容操纵的认可脆弱性,发现攻击成功率从0%到31.4%不等,且不同后端对攻击模式的敏感性和部署框架的影响存在显著差异。

Comments 23 pages, 3 figures

详情
AI中文摘要

基于大型语言模型(LLM)的搜索智能体将开放网络内容综合成可操作的建议,代表用户行事,这造成了攻击者发布的页面可能被转化为认可声明的风险。我们引入了SearchGEO,一个用于衡量基于LLM的网络搜索智能体中认可腐败的受控评估框架,它结合了网络证据操纵流水线、五模式攻击分类法和多个输出级指标。我们在308个案例上评估了13个LLM后端。结果显示,脆弱性模式因后端而异:总体攻击成功率(ASR)从Claude-Sonnet-4.6的0.0%到Gemini-3-Flash的31.4%不等,最强攻击模式因模型系列而异,且相同的部署框架可能在不同后端上放大或降低ASR。一个辅助智能体技能探测(其中认可变为安装命令)揭示了原本鲁棒的后端之间的明显分裂:Claude过度拒绝而GPT过度信任。这些发现主张将对抗性搜索内容下的推荐可靠性视为后端安全评估的一级维度。

英文摘要

Large language model (LLM)-based search agents synthesize open-web content into actionable recommendations on behalf of users, creating a risk that attacker-published pages are transformed into endorsed claims. We introduce SearchGEO, a controlled evaluation framework for measuring endorsement corruption in LLM-based web-search agents, combining a web-evidence manipulation pipeline, a five-mode attack taxonomy, and multiple output-level metrics. We evaluate 13 LLM backends on 308 cases each. Results show that vulnerability patterns vary across backends: overall attack success rate (ASR) ranges from 0.0% on Claude-Sonnet-4.6 to 31.4% on Gemini-3-Flash, the strongest attack mode differs by model family, and the same deployment scaffold could amplify or decrease ASR on different backends. An auxiliary agent-skill probe, where endorsement becomes an install command, exposes a sharp split among otherwise robust backends: Claude over-rejects while GPT over-trusts. These findings argue for treating recommendation reliability under adversarial search content as a first-class dimension of backend safety evaluation.

2606.16817 2026-06-16 cs.CL cs.IR 新提交

Understanding the Behaviors of Environment-aware Information Retrieval

理解环境感知的信息检索行为

Ruifeng Yuan, Chaohao Yuan, David Dai, Yu Rong, Hong Cheng, Hou Pong Chan, Chenghao Xiao

发表机构 * Fudan University(复旦大学) Alibaba DAMO Academy(阿里巴巴达摩院) Chinese University of Hong Kong(香港中文大学) Stanford University(斯坦福大学) Shanghai University of Finance and Economics(上海财经大学)

AI总结 通过强化学习使LLM适应不同检索器的查询策略,发现不同检索器偏好不同查询风格,并提出分支式滚动技术提升训练稳定性。

Comments ACL 2026 Main

详情
AI中文摘要

最近的检索增强生成(RAG)方法在处理复杂查询方面展示了强大的能力,但当前研究忽略了一个关键挑战:不同的检索器需要根本不同的查询制定策略才能达到最佳性能。在这项工作中,我们首次系统分析了LLM如何通过强化学习(RL)学习适应不同检索器的查询制定策略。我们的实证研究表明,RL有效地教会了LLM根据特定检索器特征定制其查询。我们发现不同的检索器表现出令人惊讶的不同最优查询风格(例如,描述性 vs. 问题式),表明为一种检索器学习的策略对另一种检索器无效。我们进一步表明,通过结合检索器特定的人类指导和扩大模型规模可以提升性能。为了促进多检索步骤轨迹的学习,我们引入了一种基于分支的滚动技术,提高了训练稳定性。我们的工作为构建真正检索器感知的RAG系统提供了首个实证证据和可操作的见解。代码和资源可在 https://github.com/LCO-Embedding/Envs-aware-Information-Retrieval 获取。

英文摘要

Recent retrieval-augmented generation (RAG) approaches have demonstrated strong capability in handling complex queries, yet current research overlooks a critical challenge: different retrievers require fundamentally different query formulation strategies for optimal performance. In this work, we present the first systematic analysis of how LLMs can learn to adapt their query formulation strategies for different retrievers via reinforcement learning (RL). Our empirical study reveals that RL effectively teaches an LLM to tailor its queries to specific retriever characteristics. We discover that different retrievers exhibit surprisingly distinct optimal query styles (e.g., descriptive vs. question-like), suggesting strategies learned for one retriever ineffective for another. We further show that performance can be enhanced by incorporating retriever-specific human guidance and by scaling model size. To facilitate learning over multi-retrieval-step trajectories, we introduce a branching-based rollout technique that improves training stability. Our work provides the first empirical evidence and actionable insights for building truly retriever-aware RAG systems. Code and resources are available at https://github.com/LCO-Embedding/Envs-aware-Information-Retrieval.

2606.16811 2026-06-16 cs.AI cs.CL 新提交

Scaling LLM Reasoning from Minimal Labels: A Semi-Supervised Framework with a Lightweight Verifier

从最小标签扩展LLM推理:一种带有轻量级验证器的半监督框架

Keizo Kato, Chenhui Chu, Yugo Murawaki, Sado Kurohashi

发表机构 * Fujitsu Limited(富士通株式会社) Kyoto University(京都大学) National Institute of Informatics(国立信息学研究所)

AI总结 提出半监督框架,用轻量级推理正确性分类器和熵过滤从少量标注数据生成高质量伪推理链,在数学和视觉问答任务上达到10-15倍标注数据效果。

Comments LREC 2026. Section 3.3 is updated

详情
AI中文摘要

对于大型语言模型(LLMs)的发展,最近生成伪中间推理的方法取得了显著进展。但它们通常依赖大量正确标注的答案来评估推理质量。本文提出一种半监督框架,从最小监督中扩展推理学习,将推理验证本身转变为数据创建机制。我们仅在少量标注样本上训练一个轻量级推理正确性分类器,用于判断LLM生成的中间推理轨迹是否有效。此外,基于熵的置信度阈值过滤掉不可靠样本,剩余的高置信度推理轨迹用于微调模型。在可验证数学问题(Orca-Math子集)和基于视觉编程的图像场景图问答(GQA)上的实验表明,我们的方法达到了与使用10-15倍标注数据相当的准确率。消融分析证实,分类器和熵过滤对于可扩展且抗噪声的伪标签生成都是必不可少的。通过用轻量级推理验证替代昂贵的答案级监督,我们的方法为构建大规模推理资源提供了一条实用路径,并为未来从最小人工输入中学习的自主推理系统铺平了道路。

英文摘要

For the development of Large language models (LLMs), recent approaches to generating pseudo intermediate reasoning have shown remarkable progress. But they typically rely on large numbers of correctly annotated answers to assess reasoning quality. This paper presents a semi-supervised framework that scales reasoning learning from minimal supervision, turning reasoning verification itself into a data creation mechanism. We train a lightweight reasoning-correctness classifier on only a few labeled samples, which judges whether intermediate reasoning traces generated by an LLM are valid. Furthermore, an entropy-based confidence threshold filters out unreliable samples, and the remaining high-confidence reasoning traces are used to fine-tune the model. Experiments on Verifiable Math Problems (Orca-Math subset) and Question Answering on Image Scene Graphs (GQA) with Visual Programming show that our method achieves accuracy comparable to using 10-15x more labeled data. Ablation analyses confirm that both the classifier and entropy filtering are essential for scalable and noise-resistant pseudo-labeling. By replacing expensive answer-level supervision with lightweight reasoning verification, our method provides a practical path toward constructing large-scale reasoning resources and paves the way for future autonomous reasoning systems that learn from minimal human input.

2606.16808 2026-06-16 cs.AI 新提交

Adaptive and Explicit safe: Triggering Latent Safety Awareness in Large Reasoning Models

自适应且显式安全:触发大型推理模型中的潜在安全意识

Ke Miao, Jiaxin Li, Hongliang Chen, Yuke Hu, Zhan Qin

发表机构 * The State Key Laboratory of Blockchain and Data Security, Zhejiang University(浙江大学区块链与数据安全全国重点实验室) Hangzhou HighTech Zone (Binjiang) Blockchain and Data Security Research Institute, China(杭州高新区(滨江)区块链与数据安全研究院) Li Auto Inc.(理想汽车) Tsinghua University(清华大学) King Abdullah University Of Science And Technology(阿卜杜拉国王科技大学)

AI总结 针对大型推理模型易受越狱攻击的问题,提出Safe Trigger方法,通过SFT显式诱导安全标签触发安全分析,并用DPO优化,显著降低攻击成功率而不影响通用性能。

详情
AI中文摘要

尽管大型推理模型(LRMs)在复杂任务上表现出色,但它们仍然极易受到复杂的越狱攻击和直接的有害查询。为了解决这一脆弱性,先前的工作严重依赖外部手动数据注释进行安全对齐。然而,我们观察到,当原始查询与其自身的推理轨迹一起重新呈现时,LRMs能够固有地识别安全风险——我们将这种能力称为潜在安全意识。为了利用这种安全意识,我们首先采用监督微调(SFT)显式诱导安全标签,以在初始推理内容之后触发对不安全查询的安全分析和指导,同时保留对一般查询的标准响应以确保自适应触发。随后,我们应用直接偏好优化(DPO)进一步增强安全分析和指导的正确性和稳定性。值得注意的是,两个训练阶段所需的响应完全由正在优化的模型生成。通过(Safe Trigger)SFT和DPO,实验结果表明安全性显著增强。例如,DeepSeek-R1-Distill-Llama-8B在有害和越狱基准上的平均攻击成功率(ASR)分别下降了24.65%和36.72%。最后,我们的Safe Trigger方法对通用性能或用户体验几乎没有负面影响。

英文摘要

While Large Reasoning Models (LRMs) excel at complex tasks, they remain highly vulnerable to sophisticated jailbreaks and direct harmful queries. To address this vulnerability, prior works depend heavily on external manual data annotation for safety alignment. However, we observe that LRMs can inherently identify safety risks when being re-presented with original queries alongside their own reasoning trajectories -- a capability we term Latent Safety Awareness. To leverage this safety awareness, we first employ Supervised Fine-Tuning (SFT) to explicitly induce safe tags to trigger safety analysis and guidance following the initial reasoning content for unsafe queries, while preserving standard responses for general queries to ensure adaptive triggering. Subsequently, we apply Direct Preference Optimization (DPO) to further enhance the correctness and stability of the safety analysis and guidance. Notably, responses required for both training stages are entirely generated by models being optimized. With (Safe Trigger) SFT and DPO, experimental results demonstrate significant safety enhancement. For example, the Attack Success Rate (ASR) of DeepSeek-R1-Distill-Llama-8B, on average, drops 24.65% and 36.72% on harmful and jailbreak benchmarks, respectively. Finally, our Safe Trigger method exerts almost no negative impact on general performance or user experience.

2606.16806 2026-06-16 cs.CL 新提交

LLM-based Visual Code Completion for Aerospace Geometric Design

基于LLM的航空航天几何设计视觉代码补全

Hau Kit Yong, Robert Marsh, Edmar A. Silva, András Sóbester, Stuart E. Middleton

发表机构 * Faculty of Engineering and Physical Sciences, University of Southampton(南安普顿大学工程与物理科学学院) School of Electronics and Computer Science, University of Southampton(南安普顿大学电子与计算机科学学院)

AI总结 提出基于LLM的视觉编程副驾驶系统,结合ReAct方法和GPT 5.4,用于航空航天几何设计,并构建Wingbuilder插件库和AVPD数据集,用户试验表明系统能生成有用建议但推理速度慢。

详情
AI中文摘要

近年来,大型语言模型(LLMs)和视觉语言模型(VLMs)在视觉代码补全能力上取得了显著进步,但航空航天行业优先考虑安全性和可解释性而非快速采用LLM,目前尚无公开宣布的基于LLM的几何设计副驾驶系统在商业上被航空航天原始设备制造商(OEMs)使用。本文提出了一种基于LLM的视觉编程副驾驶应用,用于航空航天工程设计任务,采用ReAct方法的视觉编程变体和GPT 5.4。除了副驾驶系统,我们还描述了Wingbuilder,这是一个新的Grasshopper插件库,包含用于航空航天特定几何抽象的自定义组件,以及一个相关的航空航天视觉编程数据集(AVPD),包含18个由航空航天专家设计的不同难度级别的任务及其真实解决方案。我们通过用户试验评估了副驾驶应用,试验涉及来自一家大型飞机制造公司的两位经验丰富的航空航天工程师。我们发现,我们的副驾驶视觉编程ReAct方法成功生成了参与者认为有帮助的建议,但缓慢的ReAct推理时间限制了其在更复杂、耗时的任务中的实用性,因为等待好的副驾驶解决方案建议是值得的。参与者表示他们喜欢这个工具,并愿意在未来使用它。

英文摘要

Recent advances in both Large Language Models (LLMs) and Vision Language Models (VLMs) have seen a step change in their ability to perform visual code completion, but the aerospace industry, which prioritizes safety and explainabilty over rapid LLM adoption, currently has no publicly announced LLM-based geometric design copilot systems in commercial use by aerospace Original Equipment Manufacturers (OEMs). This paper presents a LLM-based visual programming copilot application for aerospace engineering design tasks, using a visual programming variant of the ReAct methodology and GPT 5.4. In addition to the copilot, we describe Wingbuilder, a new Grasshopper plugin library with custom components for aerospace-specific geometry abstraction, and an associated Aerospace Visual Programming Dataset (AVPD) with 18 aerospace expert designed tasks at different levels of difficulty alongside ground truth solutions. We evaluate our copilot application with a user trial involving two experienced aerospace engineers from a large aircraft manufacturing company. We find our copilot visual programming ReAct methodology was successful in generating suggestions that participants found helpful, but slow ReAct inference times limit its usefulness to more complex time-consuming tasks where waiting for good copilot solution suggestion was worthwhile. Participants reported they liked the tool and would be willing to use it in the future.

2606.16802 2026-06-16 cs.AI 新提交

LabOSBench: Benchmarking Computer Use Agents for Scientific Instrument Control

LabOSBench: 科学仪器控制的计算机使用智能体基准测试

Anqi Zou, Han Deng, Chengyu Zhang, Junquan Hu, Yu Wang, Yuxiang Xing, Aokai Zhang, Hanling Zhang, Zhaoyang Liu, Ben Fei, Zhihui Wang, Wanli Ouyang

发表机构 * Shenzhen Loop Area Institute(深圳循环区域研究所) Dalian University of Technology(大连理工大学) The Chinese University of Hong Kong(香港中文大学) The Hong Kong University of Science and Technology(香港科技大学)

AI总结 提出LabOSBench基准,基于Web科学仪器模拟器评估多模态GUI智能体在仪器控制中的表现,揭示现有智能体在反馈驱动操作和长流程执行上的不足。

详情
AI中文摘要

当前的计算机使用基准主要关注虚拟化系统中的软件操作任务,而科学仪器场景需要协调控制复杂界面和反馈驱动的参数调整。然而,直接在物理高精度仪器上评估智能体因高成本、安全风险、有限可访问性和难以保证可重复评估而不切实际。这促使需要一个模拟但真实的测试平台,既能保留科学仪器的操作挑战,又能实现可扩展和安全的基准测试。为此,我们引入了LabOSBench,这是一个基于一套基于Web的科学仪器模拟器构建的多模态GUI智能体的挑战性基准。LabOSBench通过浏览器直接操作,避免了资源密集型的操作系统虚拟化,同时支持灵活的任务配置和基于执行的评估。具体来说,LabOSBench在八个仪器模拟器上构建了96个子任务,涵盖了从样品加载、对准、参数调整、数据采集到结果检查的工作流程。我们在子任务和端到端级别评估了通用视觉语言模型、专用GUI智能体模型和高级智能体框架。我们的实验表明,尽管现有智能体可以完成许多结构化的GUI子任务,但它们仍然在反馈驱动操作和长周期工作流执行中挣扎。总体而言,LabOSBench为推进计算机使用智能体向科学仪器控制发展提供了一个可重复、低成本的测试平台。

英文摘要

Current computer-use benchmarks primarily focus on software operation tasks in virtualized systems, whereas scientific instrumentation scenarios require coordinated control over complex interfaces, and feedback-driven parameter adjustment. However, directly evaluating agents on physical high-precision instruments is impractical due to high cost, safety risks, limited accessibility, and difficulty in ensuring reproducible evaluation. This motivates the need for a simulated yet realistic testbed that preserves the operational challenges of scientific instruments while enabling scalable and safe benchmarking. To this end, we introduce LabOSBench, a challenging benchmark for multimodal GUI agents built on a suite of web-based scientific-instrument simulators. Operating directly via a browser, LabOSBench avoids resource-heavy OS virtualization while supporting flexible task configuration and execution-based evaluation. Specifically, LabOSBench constructs 96 subtasks across eight instrument simulators, covering workflows from sample loading, alignment, parameter tuning, and data acquisition to result inspection. We evaluate general-purpose vision-language models, specialized GUI agent models, and advanced agentic frameworks at both subtask and end-to-end levels. Our experiments reveal that while existing agents can complete many structured GUI subtasks, they still struggle with feedback-driven operations and long-horizon workflow execution. Overall, LabOSBench provides a reproducible, low-cost testbed for advancing computer-using agents toward scientific-instrument control.

2606.16801 2026-06-16 cs.CL 新提交

The Art of Mixology: Mixup-based Obfuscation for Privacy-Preserving Split Learning in Large Language Models

混合艺术:基于混合的混淆方法用于大型语言模型中隐私保护的分割学习

Chen Chen, Xiang Gao, Xianshun Wang, Chengran Li, Shengyu Xia, Xueluan Gong, Linru Zhang, Qian Wang, Kwok-Yan Lam

发表机构 * College of Computing and Data Science, Nanyang Technological University, Singapore(南洋理工大学计算与数据科学学院) School of Cyber Science and Engineering, Wuhan University, China(武汉大学网络空间安全学院)

AI总结 提出MIXGUARD框架,通过令牌级混淆、表示级混淆和自适应梯度扰动机制,在保护隐私的同时保持模型效用,实验表明其优于现有方法。

Comments 19 pages, 5 figures

详情
AI中文摘要

分割学习为资源受限的用户提供了一种实用范式,通过将计算密集型层卸载到服务器同时保留原始数据在本地,来训练大型语言模型(LLMs)。然而,现有的隐私保护分割学习方法仍然在效用、隐私、效率和稳定性之间面临艰难的权衡。具体来说,这些方法常常遭受显著的效用下降,仍然容易受到高级数据重建攻击,产生高昂的计算和通信开销,或者在不同任务上表现出不稳定的性能。在本文中,我们提出了MIXGUARD,一种新颖的基于混合的隐私保护分割学习框架,用于LLMs。MIXGUARD引入了令牌级混淆、表示级混淆和自适应梯度扰动机制,这些机制联合运作以保留有用的学习信号,同时防止隐私泄露给服务器。技术上,MIXGUARD首先在公共数据集上构建一个轻量级校准模型,以细化近似的目标表示,然后在私有数据上的隐私保护微调期间应用该模型。我们在多个LLM家族、模型大小、架构和微调策略上,对四个分类任务和四个文本生成任务进行了大量实验。结果表明,MIXGUARD保持了与非分割训练基线相当的模型效用,在针对最先进的数据重建攻击时,始终比现有的分割学习防御方法实现更强的隐私保护,并且在自适应攻击设置下保持鲁棒性。

英文摘要

Split learning provides a practical paradigm for resource-constrained users to train Large Language Models (LLMs) by offloading computation-intensive layers to a server while keeping raw data local. However, existing privacy-preserving split learning methods still face a difficult trade-off among utility, privacy, efficiency, and stability. Specifically, these methods often suffer from substantial utility degradation, remain vulnerable to advanced data reconstruction attacks, incur prohibitive computational and communication overhead, or exhibit unstable performance across different tasks. In this paper, we propose MIXGUARD, a novel mixup-based privacy-preserving split learning framework for LLMs. MIXGUARD introduces token-level obfuscation, representation-level obfuscation, and adaptive gradient perturbation mechanisms, which operate jointly to preserve useful learning signals while preventing privacy leakage to the server. Technically, MIXGUARD first constructs a lightweight calibration model on a public dataset to refine the approximated target representation, and then applies this model during privacy-preserving fine-tuning on private data. We conduct extensive experiments on four classification tasks and four text generation tasks across multiple LLM families, model sizes, architectures, and fine-tuning strategies. The results show that MIXGUARD preserves model utility comparable to non-split training baselines, consistently achieves stronger privacy protection than existing split learning defense methods against state-of-the-art data reconstruction attacks, and remains robust under adaptive attack settings.

2606.16795 2026-06-16 cs.CV 新提交

WaveDINO: Learning-Based Atmospheric Correction of Unwrapped InSAR Interferograms Validated by GNSS: Results at Laguna del Maule and Campi Flegrei Volcanoes

WaveDINO: 基于学习的解缠InSAR干涉图大气校正方法——通过GNSS验证:在Laguna del Maule和Campi Flegrei火山的结果

Robert Popescu, Juliet Biggs, Tianyuan Zhu, Nantheera Anantrasirichai

发表机构 * University of Bristol(布里斯托大学) NERC Centre for the Observation and Modelling of Earthquakes, Volcanoes, and Tectonics (COMET)(NERC地震、火山与构造观测与建模中心(COMET)) British Geological Survey(英国地质调查局)

AI总结 提出WaveDINO,一种基于小波的多尺度去噪框架,结合DINOv3基础模型特征和地形信息,通过混合训练策略(物理合成形变+真实大气噪声)校正InSAR干涉图大气相位延迟,在智利和意大利火山数据上优于现有方法,GNSS验证显示均方根误差降低3%-19%。

Comments 11 pages, 6 figures

详情
AI中文摘要

干涉合成孔径雷达(InSAR)能够有效监测火山形变;然而,观测信号常受到大气相位延迟、季节性地表变化和去相关效应的干扰。现有的大气校正方法,如基于数值天气模型的方法,可以减少这些影响,但无法始终消除大气伪影,并可能引入残余偏差。为解决这些局限性,我们提出了一种新颖的基于学习的解缠InSAR干涉图去噪方法,采用结合物理驱动合成形变与真实大气噪声的混合训练策略。具体而言,我们引入了WaveDINO,一种基于小波的多尺度去噪框架,其条件依赖于冻结的DINOv3基础模型特征和地形信息。训练使用叠加在短周期干涉图上的合成岩浆源形变,使网络暴露于真实大气统计特征的同时保留已知真值。性能在受控合成数据和来自智利Laguna del Maule及意大利Campi Flegrei的长期真实干涉图上进行评估,并使用独立的GNSS测量进行验证。WaveDINO持续优于竞争模型,提高了与GNSS测量的一致性,在两个站点分别将平均GNSS拟合误差降低了约3%和19%,同时超越了基于天气模型的校正方法。

英文摘要

Interferometric Synthetic Aperture Radar (InSAR) enables effective monitoring of volcanic deformation; however, the observed signals are often corrupted by atmospheric phase delays, seasonal surface changes, and decorrelation effects. Existing atmospheric correction methods, such as numerical weather model-based methods, can reduce these effects but do not consistently remove atmospheric artefacts and may introduce residual biases. To address these limitations, we propose a novel learning-based method for denoising unwrapped InSAR interferograms, using a hybrid training strategy that combines physically motivated synthetic deformation with real atmospheric noise. Specifically, we introduce WaveDINO, a wavelet-based multi-scale denoising framework conditioned on frozen DINOv3 foundation-model features and terrain information. Training uses synthetic magma-source deformation superimposed on short-term interferograms to expose the network to realistic atmospheric statistics while retaining known ground truth. Performance is evaluated on both controlled synthetic data and long-term real interferograms from Laguna del Maule (Chile) and Campi Flegrei (Italy), with independent GNSS measurements used for validation. WaveDINO consistently outperforms competing models, improving agreement with GNSS measurements, and reducing mean GNSS misfit by approximately 3% and 19% at two sites, respectively, while surpassing weather-model-based corrections.

2606.16790 2026-06-16 cs.LG cs.AI 新提交

Decision-Weighted Flow Matching for Contextual Stochastic Optimization

决策加权流匹配用于上下文随机优化

Jize Xie, Haomiao Wu, Qiang Chen, Xiu Su, Yi Chen

发表机构 * Hong Kong University of Science and Technology(香港科技大学) Central South University(中南大学) Big Data Institute(大数据研究院)

AI总结 提出决策加权流匹配(DW-FM)框架,通过重加权速度回归目标对齐下游遗憾,在CVaR基准上优于标准方法。

详情
AI中文摘要

条件生成模型越来越多地被用作随机优化的场景生成器,但标准训练目标强调均匀分布拟合,而非生成场景所引发的下游决策。这造成了目标不匹配:统计常见区域的误差对决策遗憾影响很小,而决策敏感区域的误差可能显著改变最优行动。我们提出决策加权流匹配(DW-FM),一种遗憾对齐的训练框架,它保留了标准流匹配的简单性,同时使用决策敏感的端点信息对其速度回归目标进行重加权。理论上,我们通过损失诱导的决策差异和伴随输运论证将下游遗憾与路径速度不匹配联系起来,得到一个理想的遗憾对齐替代目标以及具有遗憾保证的实用端点加权目标。实验上,我们在三个基于CVaR的上下文随机优化基准(涵盖合成投资组合、半真实金融和交通CVaR任务)上展示了DW-FM的有效性,其中DW-FM在标准基线上改善了下游遗憾。

英文摘要

Conditional generative models are increasingly used as scenario generators for stochastic optimization, but standard training objectives emphasize uniform distributional fit rather than the downstream decisions induced by generated scenarios. This creates an objective mismatch: errors in statistically common regions may have little effect on decision regret, whereas errors in decision-sensitive regions can substantially change the optimal action. We propose Decision-Weighted Flow Matching (DW-FM), a regret-aligned training framework that preserves the simplicity of standard flow matching while reweighting its velocity-regression objective using decision-sensitive endpoint information. Theoretically, we connect downstream regret to pathwise velocity mismatch through a loss-induced decision discrepancy and an adjoint transport argument, yielding an ideal regret-aligned surrogate and practical endpoint-weighted objectives with regret guarantees. Empirically, we demonstrate the effectiveness of DW-FM on three CVaR-based contextual stochastic optimization benchmarks spanning synthetic portfolio, semi-real financial, and traffic-CVaR tasks, where DW-FM improves downstream regret over standard baselines.

2606.16788 2026-06-16 cs.RO 新提交

SoK: Security and Privacy of Foundation-Model-Powered Robots

SoK: 基础模型驱动机器人的安全与隐私

Xueluan Gong, Chen Chen, Jinxin Liu, Qian Wang, Kwok-Yan Lam

发表机构 * College of Computing and Data Science, Nanyang Technological University(南洋理工大学计算与数据科学学院) School of Cyber Science and Engineering, Wuhan University(武汉大学网络空间安全学院)

AI总结 本文提出F-E-S-G结构边界框架,系统分析基础模型驱动机器人的安全与隐私风险,并基于96篇论文揭示威胁模式、防御不匹配和评估差距。

Comments 21 pages, 2 figures

详情
AI中文摘要

基础模型正在重塑机器人技术,使机器人能够解释开放式指令、推理多模态上下文并在复杂的开放世界环境中运行。然而,它们的集成也引入了安全与隐私(S&P)风险,这些风险从基础模型本身扩展到具身执行管道、支持生态系统以及更广泛的治理影响。现有文献综述提供了宝贵的见解,但通常侧重于特定的基础模型类型、风险类别、缓解策略或信任边界。因此,该领域缺乏一个统一的结构来分析风险源自何处、如何在机器人系统中传播以及缓解措施应在何处干预。为填补这一空白,我们提出了一个渐进式的F-E-S-G结构边界框架,用于分析基础模型驱动机器人的安全与隐私。该框架包含四个层次:基础模型层(F)、具身系统层(E)、支持生态系统层(S)和治理影响层(G)。基于此结构,我们开发了一个多级分类法,沿三个层次组织先前的研究:F-E-S-G信任边界、安全-隐私关注点以及风险-缓解视角。我们进一步使用细粒度编码属性对每项研究进行注释,包括目标、生命周期阶段、机制、系统访问和效果。在此框架和分类法的指导下,我们对96篇论文进行了系统化分析。我们的分析揭示了从单一边界视角难以识别的多种威胁模式、防御不匹配和评估差距。基于这些发现,我们确定了开放挑战和未来方向,为开发安全、隐私保护且负责任治理的基础模型驱动机器人系统提供了研究议程。

英文摘要

Foundation models are reshaping robotics by enabling robots to interpret open-ended instructions, reason over multimodal contexts, and operate in complex, open-world environments. However, their integration also introduces security and privacy (S&P) risks that extend beyond the FMs themselves to embodied execution pipelines, supporting ecosystems, and broader governance impacts. Existing literature reviews provide valuable insights but often focus on specific FM types, risk categories, mitigation strategies, or trust boundaries. Consequently, the field lacks a unified structure for analyzing where risks originate, how they propagate across robotic systems, and where mitigations should intervene. To address this gap, we propose a progressive F-E-S-G structural boundary framework for analyzing the S&P of FM-powered robots. The framework comprises four layers: the Foundation model layer (F), Embodied system layer (E), Supporting ecosystem layer (S), and Governance impact layer (G). Building on this structure, we develop a multi-level taxonomy that organizes prior studies along three levels: F-E-S-G trust boundary, security-privacy concerns, and risk-mitigation perspectives. We further annotate each study using fine-grained coding attributes, including target, lifecycle stage, mechanism, system access, and effect. Guided by this framework and taxonomy, we systematize 96 papers. Our analysis uncovers multiple threat patterns, defense mismatches, and evaluation gaps that are difficult to identify from a single-boundary perspective. Based on these findings, we identify open challenges and future directions to provide a research agenda for developing secure, privacy-preserving, and responsibly governed FM-powered robotic systems.

2606.16786 2026-06-16 cs.LG 新提交

We Need Explanation Cards to Connect Explanation Algorithms to the Real World

我们需要解释卡来连接解释算法与现实世界

Eric Günther, Balázs Szabados, Kristof Meding, Gunnar König, Sebastian Bordt, Ulrike von Luxburg

发表机构 * University of Tübingen(蒂宾根大学) Tübingen AI Center(蒂宾根人工智能中心) HUN-REN Institute for Computer Science and Control (SZTAKI), Budapest, Hungary(匈牙利科学院计算机科学与控制研究所(SZTAKI))

AI总结 针对算法解释在实践中含义模糊且信息不足的问题,提出解释卡,通过补充鲁棒性和有效性信息及解释说明,帮助用户正确解读,并满足欧盟AI法案的可解释性要求。

详情
AI中文摘要

算法解释旨在帮助利益相关者理解不透明的算法决策,但在实践中往往达不到预期。首先,算法解释的含义通常不是人们直观期望的那样,因此需要专业知识才能正确解释。其次,最近的研究表明,流行的解释算法对于复杂决策函数的行为信息不足。这些共同导致了解释表面传达的内容与实际提供的内容之间的差距。在这项工作中,我们提出了解释算法的解释卡,它用关于鲁棒性和有效性的补充信息以及清晰的解释说明来增强标准解释。补充信息可以使原本无信息的解释变得实际有用,同时也有助于检测它们不适用的情况。重要的是,解释卡中的解释说明将责任从用户转移到提供者:提供者必须事先明确说明从解释中可以得出什么和不能得出什么,而不是期望用户自己识别。使用反事实解释和SHAP作为示例,我们展示了提供者如何构建解释卡,以及这些卡为用户提供了正确解释所需的指导。我们进一步论证了解释卡是实践欧盟AI法案可解释性规定的实用手段。总体而言,解释卡是使解释算法适应现实世界用例的重要一步。

英文摘要

Algorithmic explanations are intended to help stakeholders understand opaque algorithmic decisions, but in practice, they often fall short. First, the meaning of algorithmic explanations is often not what one might intuitively expect, so expert knowledge is required to interpret them correctly. Second, recent work has shown that popular explanation algorithms are uninformative about the behavior of complex decision functions. Together, these issues create a gap between what explanations appear to convey and what they actually provide. In this work, we propose Explanation Cards for Explanation Algorithms, which augment standard explanations with complementary information about robustness and validity, as well as clear instructions for interpretation. The complementary information can render otherwise uninformative explanations practically useful, while also helping to detect cases where they are not. Importantly, the interpretation instructions in explanation cards shift responsibility from users to providers: Rather than expecting users to recognize what can and cannot be concluded from an explanation, providers must make this explicit upfront. Using counterfactual explanations and SHAP as examples, we demonstrate how providers can construct explanation cards and that these cards provide users with the guidance needed for sound interpretation. We further argue that explanation cards offer a practical means of operationalising the explainability provisions of the EU AI Act. Overall, explanation cards are a significant step toward making explanation algorithms fit for real-world use cases.

2606.16783 2026-06-16 cs.CV cs.AI cs.LG 新提交

Gen-VCoT: Generative Visual Chain-of-Thought Reasoning via Diffusion-Based RGB Intermediate Representations

Gen-VCoT: 基于扩散的RGB中间表示的生成式视觉思维链推理

Zhiqiang Zhou, Junliang Dai, Xu ling

发表机构 * Hunan Chemical Industry Vocational and Technical College(湖南化工职业技术学院)

AI总结 提出Gen-VCoT框架,利用专家视觉模型生成RGB图像作为推理中间步骤,通过自适应路由器选择推理深度,在空间和深度问题上分别提升25%和50%,但简单事实查询性能下降,表明最优表示依赖于任务。

Comments 12 pages, 5 figures

详情
AI中文摘要

多模态大语言模型(MLLMs)在视觉推理方面表现出色,但依赖基于文本的思维链(CoT),缺乏可解释的视觉中间表示。现有方法使用不透明的标记或外部工具,缺失关键属性。我们提出Gen-VCoT,一个使用专家视觉模型生成RGB图像作为推理中间表示的框架。它包含三个阶段:视觉定位(SAM分割)、几何推理(Marigold深度图)和语义推理(Qwen2-VL集成)。一个自适应路由器选择推理深度。评估显示,Gen-VCoT在空间问题(提升25%)和深度问题(提升50%)上表现更好,但可能损害简单事实查询。文本CoT在CLEVR上优于视觉中间表示(91.2% vs 62.5%),表明最优表示依赖于任务。Gen-VCoT为可解释的多模态推理建立了新范式。

英文摘要

Multimodal large language models (MLLMs) excel at visual reasoning but rely on text-based chain-of-thought (CoT), lacking interpretable visual intermediates. Existing methods use opaque tokens or external tools, missing key properties. We propose Gen-VCoT, a framework using expert vision models to generate RGB images as reasoning intermediates. It has three stages: visual grounding (SAM segmentation), geometric reasoning (Marigold depth maps), and semantic reasoning (Qwen2-VL integration). An adaptive router selects reasoning depth. Evaluations show Gen-VCoT improves spatial (25% better) and depth (50% better) questions, but may hurt simple factual queries. Text CoT outperforms visual intermediates on CLEVR (91.2% vs 62.5%), showing task-dependent optimal representations. Gen-VCoT establishes a new paradigm for interpretable multimodal reasoning.

2606.16780 2026-06-16 cs.RO 新提交

DIFF-IPPO: Diffusion-Based Informative Path Planning with Open-Vocabulary Belief Maps

DIFF-IPPO:基于扩散的开放词汇信念地图信息路径规划

Sausar Karaf, Oleg Sautenkov, Mikhail Martynov, Dzmitry Tsetserukou

发表机构 * Intelligent Space Robotics Laboratory, CDE, Skoltech(智能空间机器人实验室,CDE,斯科尔科沃科学技术研究院)

AI总结 提出DIFF-IPPO框架,结合开放词汇信念地图生成器与扩散规划器,在非高斯信念图上生成全局轨迹,实现高效目标搜索,检测得分达81.49%-86.55%。

详情
AI中文摘要

探索和物体搜索要求机器人感知环境、识别感兴趣区域,并规划提高目标检测可能性或最大化信息增益的轨迹。许多IPP方法,特别是在连续环境监测中,依赖于高斯过程信念模型,而物体搜索场景通常从语义或开放词汇感知中产生复杂的多模态信念地图。直接基于这种非高斯信念地图的全局轨迹生成仍然相对未被充分探索。尽管基于扩散的规划器为此类分布建模提供了强大能力,但它们在信息路径规划中的应用仍然有限。在这项工作中,我们提出了DIFF-IPPO,一个集成了开放词汇信念地图生成器和基于扩散的规划器的流水线,用于在信念地图上生成全局轨迹。该方法生成的轨迹将传感器覆盖集中在高信念区域,在不同数据集场景下实现了81.49%至86.55%的归一化检测得分。我们在一个模拟的搜索与救援场景中验证了该系统,其中规划器搜索候选建筑区域以定位燃烧的建筑。在此设置中,一个由五架无人机组成的团队使用批处理信念地图条件轨迹生成,在3.5分钟内实现了首次检测。

英文摘要

Exploration and object search require robots to perceive their environment, identify regions of interest, and plan trajectories that improve target-detection likelihood or maximize information gain. Many IPP methods, especially in continuous environmental monitoring, rely on Gaussian-process belief models, while object-search settings often produce complex, multimodal belief maps from semantic or open-vocabulary perception. Global trajectory generation directly conditioned on such non-Gaussian belief maps remains comparatively underexplored. Although diffusion-based planners offer strong capabilities for modeling such distributions, their use in informative path planning remains limited. In this work, we propose DIFF-IPPO, a pipeline that integrates an open-vocabulary belief map generator with a diffusion-based planner for global trajectory generation over belief maps. The method generates trajectories that concentrate sensor coverage over high-belief regions, achieving normalized detection scores between 81.49% and 86.55% across different dataset scenarios. We validate the system in a simulated search-and-rescue scenario where the planner searches candidate building regions to locate a burning building. In this setting, a team of five drones using batched belief-map-conditioned trajectory generation achieves first detections in 3.5 minutes.

2606.16776 2026-06-16 cs.RO 新提交

DataLadder: A Simulation-Enabled Interconversion Toolchain for the Embodied Data Pyramid

DataLadder:面向具身数据金字塔的仿真赋能互转换工具链

Peidong Liu, Yongce Liu, Songyan Guo, Fuyuan Ma, Zhihao Yuan, Ao Li, Zengjue Chen, Wenhao Li, Tianle Zhang, Mingyang Li, Jiale Zhang, Junzhe Xiong, Zhiyuan Xiang, Dafeng Chi, Yuzheng Zhuang, Yihang Li, Qingrong He, Jiaming Liang, Chen Cai, Peng Hao, Mingxi Luo, Song Wang, Junwu Xiong, Ruodai Li, Liyi Luo, Wei Tan, Dongjiang Li, Jiawei Li, Hui Shen, Yicheng Gong, Liang Lin

发表机构 * Joy Future Academy, JD Group(京东集团未来研究院) JD Technology, JD Group(京东集团京东科技)

AI总结 提出DataLadder工具链,通过机器人↔仿真↔人类双向路径,实现人机对齐的模型评估与数据生成,利用数字孪生和仿真一致性过滤解决物理机器人扩展难题。

Comments Project Page: https://joyai-sim.github.io/

详情
AI中文摘要

通用机器人策略需要可信的评估和机器人可用的训练数据,但仅靠物理机器人难以规模化。真实机器人试验和演示仍然是部署信号最可靠的来源,但它们缓慢、昂贵且难以复现。我们提出DataLadder,一个仿真赋能的人机对齐模型评估与数据生成互转换工具链,记为Robot $\ ightleftharpoons$ Simulation $\ ightleftharpoons$ Human。一方面,Robot $\ ightarrow$ Simulation $\ ightarrow$ Human路径通过将真实机器人桌面整理任务重建为校准的数字孪生体以进行可扩展评估,同时利用人类具身反馈检查和优化仿真运动的自然性,支持人机对齐的模型评估。另一方面,Human $\ ightarrow$ Simulation $\ ightarrow$ Robot路径支持人机对齐的数据生成:它将自我中心的人类演示提升到仿真中,在机器人物理约束下检查它们,并将其转换为以机器人为中心的轨迹、标注和视觉观察。这些路径共同使用JoySim仿真器作为机器人数据生成的可扩展评估层和物理一致性过滤器。我们进一步将核心重建、仿真、渲染和真实性增强模块打包为京东云上的云服务,将系统转变为机器人数据生成和模型评估的可复用基础设施。

英文摘要

Generalist robot policies require trustworthy evaluation and robot-usable training data, but both are difficult to scale with physical robots alone. Real-robot trials and demonstrations remain the most faithful source of deployment signals, yet they are slow, costly, and hard to reproduce. We present DataLadder, a simulation-enabled interconversion toolchain for human-robot aligned model evaluation and data generation, denoted as Robot $\rightleftharpoons$ Simulation $\rightleftharpoons$ Human. On the one hand, the Robot $\rightarrow$ Simulation $\rightarrow$ Human pathway supports human-robot aligned model evaluation by reconstructing real-robot tabletop organization tasks as calibrated digital twins for scalable evaluation, while using human embodied feedback to inspect and refine the naturalness of simulated motions. On the other hand, the Human $\rightarrow$ Simulation $\rightarrow$ Robot pathway supports human-robot aligned data generation: it lifts ego-centric human demonstrations into simulation, checks them under robot physical constraints, and converts them into robot-centered trajectories, annotations, and visual observations. Together, these pathways use the JoySim simulator as both a scalable evaluation layer and a physical consistency filter for robot data generation. We further package the core reconstruction, simulation, rendering, and realism-augmentation modules as cloud services on JD Cloud, turning the system into reusable infrastructure for robot data generation and model evaluation.

2606.16774 2026-06-16 cs.AI cs.CL 新提交

OpenClaw-Skill: Collective Skill Tree Search for Agentic Large Language Models

OpenClaw-Skill:面向智能体大语言模型的集体技能树搜索

Tianyi Lin, Chuanyu Sun, Jingyi Zhang, Changxu Wei, Huanjin Yao, Shunyu Liu, Xikun Zhang, Liu Liu, Jiaxing Huang

发表机构 * The Hong Kong Polytechnic University(香港理工大学) Nanyang Technological University(南洋理工大学) Tsinghua University(清华大学) Royal Melbourne Institute of Technology(皇家墨尔本理工大学) Beijing University of Aeronautics and Astronautics(北京航空航天大学)

AI总结 提出集体技能树搜索(CSTS)框架,通过集体智能生成和评估技能节点,构建结构化、多样且可泛化的技能树,并引入集体技能强化学习,提升大语言模型在工具使用、多步推理和动态环境交互中的智能体能力。

Comments 13 pages, 2 figures

详情
AI中文摘要

为大型语言模型(LLM)智能体配备有效技能对于解决OpenClaw等现实世界系统中的复杂任务至关重要。在这项工作中,我们旨在开发一个自动构建此类可重用技能的框架,以增强LLM在工具使用、多步推理和动态环境交互方面的能力。为此,我们提出了集体技能树搜索(CSTS),一种新颖的基于树搜索的技能构建框架,用于构建结构化、多样且可泛化的技能树。CSTS的核心思想是利用集体智能,通过两个迭代阶段共同搜索、识别和组合有效技能:集体技能节点生成(CSN-Gen)和集体技能节点评估(CSN-Assess)。CSN-Gen利用来自多个模型的集体知识,为每个子任务探索多样化的候选技能,实现全面的技能探索。CSN-Assess使用多个模型作为评判者,通过两种评分机制评估和选择技能节点:(1)集体质量评分,聚合独立评估以产生技能有效性的稳健估计;(2)集体可迁移性评分,明确验证技能是否在不同模型间良好泛化。通过CSTS,我们构建了一套全面的技能树以及技能增强的训练数据,使模型能够有效学习和利用技能。此外,我们引入了集体技能强化学习,主动从技能树中选择多个相关技能,以拓宽解空间探索,避免陷入单一技能及其导致的同质或次优解。最终,我们训练的模型OpenClaw-Skill在长期规划、工具使用和跨挑战性基准的泛化方面展现出卓越的智能体能力。

英文摘要

Equipping Large Language Model (LLM) agents with effective skills is crucial for solving complex tasks in real-world systems like OpenClaw. In this work, we aim to develop a framework that automatically constructs such reusable skills to enhance LLMs in tool use, multi-step reasoning, and dynamic environment interaction. To this end, we propose Collective Skill Tree Search (CSTS), a novel tree-search-based skill construction framework that constructs structured, diverse and generalizable tree of skills. The core idea of CSTS is to leverage collective intelligence to jointly search, identify and compose effective skills via two iterative phases: Collective Skill Node Generation (CSN-Gen) and Collective Skill Node Assessment (CSN-Assess). CSN-Gen exploits collective knowledge from multiple models to explore diverse candidate skills for each subtask, enabling comprehensive skill exploration. CSN-Assess employs multiple models as judges to evaluate and select skill nodes with two scoring mechanisms: (1) collective quality scoring that aggregates independent evaluations to produce a robust estimate of skill effectiveness, and (2) collective transferability scoring that explicitly verifies whether a skill generalizes well across different models. With CSTS, we construct a set of comprehensive tree of skills along with skill-augmented training data, enabling models to effectively learn and utilize skills. Besides, we introduce Collective Skill Reinforcement Learning, which actively selects multiple relevant skills from the tree to broaden solution-space exploration, avoid being trapped by a single skill and its resulting homogeneous or suboptimal solutions. As a result, our trained model, OpenClaw-Skill, exhibits outstanding agentic capabilities in long-horizon planning, tool use and generalization over challenging benchmarks.

2606.16771 2026-06-16 cs.LG 新提交

GD$^2$PO: Mitigating Multi-Reward Conflicts via Group-Dynamic reward-Decoupled Policy Optimization

GD$^2$PO: 通过组动态奖励解耦策略优化缓解多奖励冲突

Haotian Liu, Yihao Liu, Jingwei Ni, Siyuan Huang, Xinpeng Liu, Pengyu Cheng, Jiajun Song, Ruijin Ding, Junfeng Li, Zhechao Yu, Mengyu Zhou, Hongteng Xu, Xiaoxi Jiang, Guanjun Jiang

发表机构 * Qwen Large Model Application Team, Alibaba(阿里巴巴通义千问大模型应用团队) Renmin University of China(中国人民大学) Peking University(北京大学) ETH Zürich(苏黎世联邦理工学院) University of Zurich(苏黎世大学) The Chinese University of Hong Kong(香港中文大学)

AI总结 提出GD$^2$PO算法,通过冲突感知过滤机制屏蔽奖励不一致的rollout,并结合查询级重加权,解决多奖励优化中的信号抵消问题,提升RL训练效率。

Comments 24 pages, 9 figures

详情
AI中文摘要

随着LLM的发展,后训练强化学习(RL)越来越依赖多维奖励来培养全面能力。这种转变需要新的算法来同时优化多样且可能相互竞争的目标。为了解决这个问题,现有方法如组奖励解耦策略优化(GDPO)将整体得分分解为独立的奖励组,然后在每个组内分别计算RL损失。然而,这种策略仍然会遇到多奖励冲突:单个rollout在某些奖励维度上可能产生正优势,但在其他维度上产生负优势,导致聚合过程中相反信号相互抵消,进一步阻碍RL训练效率。受动态采样策略优化(DAPO)的启发,DAPO通过过滤掉具有接近零优势的无效rollout来提高RL训练效率,我们提出了组动态奖励解耦策略优化(GD$^2$PO)。具体来说,GD$^2$PO采用冲突感知过滤机制来屏蔽遭受严重奖励不一致的rollout。通过防止冲突信号相互抵消,这种掩蔽策略保留并增强了有效RL优势的幅度,从而显著加速学习效率。此外,我们引入了查询级重加权,根据每个查询的整体奖励共识动态调整其更新强度。在多种多奖励场景(包括工具调用和人类偏好对齐)上的实验表明,GD$^2$PO持续且显著优于现有基线。代码可在https://github.com/Qwen-Applications/GD2PO获取。

英文摘要

As LLMs advance, post-training reinforcement learning (RL) increasingly relies on multi-dimensional rewards to cultivate comprehensive capabilities. This shift demands new algorithms capable of optimizing diverse and potentially competing objectives simultaneously. To address this, existing methods such as Group reward-Decoupled Policy Optimization (GDPO) decompose the overall score into independent reward groups, then compute the RL loss separately within each group. However, this strategy still encounters multi-reward conflicts: a single rollout can yield positive advantages on certain reward dimensions but negative ones on others, causing opposing signals to cancel each other out during aggregation, further hindering RL training efficiency. Inspired by Dynamic sAmpling Policy Optimization (DAPO), which improves RL training efficiency by filtering out ineffective rollouts with near-zero advantages, we propose Group-Dynamic reward-Decoupled Policy Optimization (GD$^2$PO). Specifically, GD$^2$PO employs a conflict-aware filtering mechanism to mask out rollouts suffering from severe reward-wise disagreement. By preventing conflicting signals from canceling each other out, this masking strategy preserves and enhances the magnitude of effective RL advantages, thereby significantly accelerating learning efficiency. Furthermore, we introduce query-level reweighting to dynamically adjust the update intensity of each query based on its overall reward consensus. Experiments on various multi-reward scenarios, including tool calling and human preference alignment, demonstrate that GD$^2$PO consistently and significantly outperforms existing baselines. The code is available at https://github.com/Qwen-Applications/GD2PO.

2606.16769 2026-06-16 cs.AI 新提交

Skill-to-LoRA: From Using Skills to Learning Behaviors for Token-Efficient LLM Agents

Skill-to-LoRA:从使用技能到学习行为以实现令牌高效的LLM智能体

Tianyi Zhang, Zhonghao Qi

发表机构 * The Chinese University of Hong Kong(香港中文大学)

AI总结 提出Skill-to-LoRA方法,将技能文本转换为LoRA适配器,替代运行时注入技能文档,在SWE-Skills-Bench上提升通过率并降低令牌成本。

Comments Preprint. 10 pages, 4 figures

详情
AI中文摘要

智能体技能通常以SKILL.md文件形式分发:描述工作流、工具、资源和领域约定的人类可读程序文档。虽然便于检查和重用,但这种设计需要将相同的可重用程序重复注入运行时上下文。我们提出Skill-to-LoRA(S2L),一种以行为为中心的技能表示,用技能特定的LoRA适配器替代运行时技能文本。S2L不是压缩技能文档本身,而是建模技能文本引起的行为变化:离线时,使用完整的SKILL.md合成技能引导的演示;在线时,省略完整文档,动态加载对应的LoRA适配器以激活学习到的技能行为。我们使用Qwen3.6-27B在SWE-Skills-Bench的21个技能子集上评估S2L。与无技能和完整技能文本基线相比,S2L的通过率分别提高2.9和5.2个百分点,同时相对于完整技能文本提示,每步令牌成本降低6.6%。S2L在18/21个技能上匹配或优于完整技能文本,在15/21个技能上匹配或优于无技能基线。控制实验进一步表明,性能提升依赖于技能特定的适配器对齐:错误LoRA和共享LoRA均降低性能。这些结果表明,许多程序性智能体技能可以从运行时指令转换为可训练、可动态加载的行为模块。代码将在接收后发布。

英文摘要

Agent skills are commonly distributed as SKILL.md files: human-readable procedural documents that describe workflows, tools, resources, and domain conventions. While convenient for inspection and reuse, this design requires the same reusable procedure to be repeatedly injected into the runtime context. We propose Skill-to-LoRA(S2L), a behavior-centric skill representation that replaces runtime skill text with skill-specific LoRA adapters. Rather than compressing the skill document itself, S2L models the behavioral change induced by the skill text: offline, the complete SKILL.md is used to synthesize skill-guided demonstrations; online, the full document is omitted and the corresponding LoRA adapter is dynamically loaded to activate the learned skill behavior. We evaluate S2L with Qwen3.6-27B on a 21-skill subset of SWE-Skills-Bench. Compared with the no-skill and Full Skill Text baselines, S2L improves pass rate by 2.9 and 5.2 percentage points, respectively, while reducing per-step token cost by 6.6% relative to Full Skill Text prompting. S2L matches or improves Full Skill Text on 18/21 skills and the no-skill baseline on 15/21 skills. Control experiments further show that the gains depend on skill-specific adapter alignment: Wrong-LoRA and Shared-LoRA both reduce performance. These results suggest that many procedural agent skills can be converted from runtime instructions into trainable, dynamically loadable behavioral modules. Code will be released upon acceptance.

2606.16768 2026-06-16 cs.LG 新提交

Taming Curvature: Architecture Warm-Up for Stable Transformer Training

驯服曲率:稳定Transformer训练的架构预热

Sameera Ramasinghe, Ajanthan Thalaiyasingam, Hadi Mohaghegh Dolatabadi, Chamin Hewa Koneputugodage, Gil Avraham, Violetta Shevchenko, Yan Zuo, Karol Pajak, Alexander Long

发表机构 * Pluralis Research

AI总结 提出基于热启动幂迭代的快速在线曲率估计方法,并发现训练不稳定性与预条件曲率激增相关,进而提出渐进增加网络深度的架构预热策略,有效稳定大模型训练。

详情
AI中文摘要

训练数十亿参数的Transformer通常很脆弱,会出现瞬时的损失尖峰和发散,浪费计算资源。尽管最近发展的边缘稳定性(EoS)理论通过(预条件)曲率提供了理解和控制优化方法稳定性的强大工具,但由于曲率估计的复杂性,这些曲率控制方法在大规模Transformer训练中并不流行。为此,我们首先引入一种基于热启动变体的快速在线估计器,用于估计最大的(预条件)Hessian特征值(即曲率),该估计器使用Hessian-向量积进行幂迭代。我们从理论上证明,并通过实验验证,所提出的方法在十亿参数规模下使每次迭代的曲率跟踪变得可行,同时更加准确。利用这一工具,我们发现训练不稳定性与预条件曲率的激增同时发生,并且曲率随深度增加而增长。基于这些观察,我们提出架构预热:逐步增加网络深度,以仔细控制预条件Hessian并稳定训练。在大规模Transformer上的实验验证了我们的方法能够实现高效的曲率跟踪,并在不减慢收敛速度的情况下,与现有最先进的稳定技术相比减少了不稳定性。

英文摘要

Training billion-parameter Transformers is often brittle, with transient loss spikes and divergence that waste compute. Even though the recently developed Edge of Stability (EoS) theory provides a powerful tool to understand and control the stability of optimization methods via the (preconditioned) curvature, these curvature-controlling methods are not popular in large-scale Transformer training due to the complexity of curvature estimation. To this end, we first introduce a fast online estimator of the largest (preconditioned) Hessian eigenvalue (i.e., curvature) based on a warm-started variant for power iteration with Hessian-vector products. We show theoretically, and verify empirically, that the proposed method makes per-iteration curvature tracking feasible at billion parameter scale while being more accurate. Using this tool, we find that training instabilities coincide with surges in preconditioned curvature and that curvature grows with depth. Motivated by these observations, we propose architecture warm-up: progressively growing network depth to carefully control the preconditioned Hessian and stabilize training. Experiments on large Transformers validate that our approach enables efficient curvature tracking and reduces instabilities compared to existing state-of-the-art stabilization techniques without slowing down convergence.

2606.16767 2026-06-16 cs.CV 新提交

Text-Vision Co-Instructed Image Editing

文本-视觉协同指导的图像编辑

Chenxi Xie, Yuhui Wu, Qiaosi Yi, Lei Zhang

发表机构 * The Hong Kong Polytechnic University(香港理工大学) OPPO Research Institute(OPPO研究院)

AI总结 提出TV-Edit框架,联合文本指令的语义表达与稀疏视觉指令的空间引导,实现精确且忠实于意图的图像编辑,显著优于现有方法。

详情
AI中文摘要

现有的图像编辑方法通常可分为基于文本指令和基于视觉提示两类。文本指令语义表达丰富,但受限于编辑结果空间控制的粗粒度。相比之下,拖拽和点等视觉提示能提供精确的空间引导,但存在语义意图固有的模糊性。为统一文本和视觉提示的优势,我们提出文本-视觉协同指导的图像编辑,将文本指令作为语义意图、稀疏视觉指令作为空间引导联合建模,旨在实现精确且忠实于意图的图像操作。为此,我们首先构建了一个包含超过23K个样本的文本-视觉指令配对数据集,这些样本源自动态视频,为跨模态指令提供对齐监督。然后,我们提出TV-Edit,一个文本-视觉指令统一编辑框架,将基于拖拽或点的视觉指令与图像-文本语义上下文化,并将其提升为语义感知的控制表示,用于预训练的编辑骨干网络。通过整合语义意图和空间约束,TV-Edit相比纯文本或纯拖拽方法实现了更精确的空间控制、更少的指令歧义和更强的结构一致性。最后,我们建立了TV-Edit-Bench,一个精心设计的基准,用于评估语义忠实度、空间对齐和视觉一致性,通过地面真实参考和受控的文本-视觉变化进行可靠评估。我们在多个编辑骨干网络上的实验表明,TV-Edit始终产生更精确且忠实于意图的编辑,显著优于最先进的基于指令和基于拖拽的基线方法。

英文摘要

Existing image editing methods can be generally categorized into textual instruction-based and visual prompt-based ones. Textual instructions are semantically expressive, but are limited by the coarse granularity of spatial control of the editing results. In contrast, visual prompts such as drag and point can provide precise spatial guidance, but are limited by the inherent ambiguity in semantic intent. To unify the strength of textual and visual prompts, we present Text-Vision Co-Instructed Image Editing, which jointly models textual instructions as semantic intent and sparse visual instructions as spatial guidance, aiming to achieve precise and intent-faithful image manipulation. To this end, we first construct a textual-visual instruction paired dataset with more than 23K samples derived from dynamic videos, enabling aligned supervision for cross-modal instruction. We then propose TV-Edit, a Textual-Visual instruction unified Editing framework to contextualize drag or point-based visual instructions with image-text semantics and lift them into semantic-aware control representations for pretrained editing backbones. By integrating semantic intent and spatial constraints, TV-Edit leads to more precise spatial control, less instruction ambiguity, and stronger structural consistency than text-only or drag-based alternatives. Finally, we establish TV-Edit-Bench, a deliberately designed benchmark to evaluate semantic faithfulness, spatial alignment, and visual consistency with ground-truth references and controlled textual-visual variations for reliable assessment. Our experiments across multiple editing backbones demonstrate that TV-Edit consistently yields more precise and intent-faithful edits, significantly outperforming state-of-the-art instruction-based and drag-based baselines.

2606.16759 2026-06-16 cs.LG 新提交

Maximum Entropy Inverse Reinforcement Learning for Mean-Field Games with Average Reward

平均奖励均值场博弈的最大熵逆强化学习

Şevket Kaan Alkır, Naci Saldı, Berkay Anahtarcı, Can Deha Karıksız

发表机构 * Bilkent University(比尔肯大学) Özyeğin University(厄齐金大学)

AI总结 针对平均奖励准则下的离散时间无限时域均值场博弈,提出基于最大因果熵的逆强化学习方法,通过占据测度框架统一处理有限维线性奖励和无限维RKHS奖励,并设计梯度上升算法实现策略恢复。

Comments 49 pages, 2 figures, 2 tables

详情
AI中文摘要

我们研究了平均奖励准则下离散时间、无限时域均值场博弈(MFGs)的逆强化学习。专家演示被认为来自未知奖励下的平稳均值场均衡,目标是通过最大因果熵原理恢复解释观察行为的策略。我们通过强制与专家均值场项和长期特征期望的一致性来制定逆问题,在统一的占据测度框架内处理两类奖励。对于有限维线性奖励,我们给出了具有显式对数配分目标的对偶凸重构,并证明了平滑性和曲率性质,从而证明了恒定步长梯度下降的合理性。对于无限维RKHS奖励,我们开发了一种拉格朗日松弛,其内最大化策略由软贝尔曼方程刻画。主要障碍是缺乏折扣因子收缩。我们通过引入基于极小化的次随机核来解决这个问题,该核产生了软贝尔曼算子的严格收缩。我们建立了对数似然得分的Fréchet可微性和Lipschitz平滑性,从而得到了具有收敛保证的梯度上升算法。两个数值例子,一个恶意软件传播MFG和一个基于RKHS的消费者选择模型,表明恢复的策略与专家行为紧密匹配。

英文摘要

We study inverse reinforcement learning for discrete-time, infinite-horizon mean-field games (MFGs) under an average-reward criterion. Expert demonstrations are assumed to arise from a stationary mean-field equilibrium under an unknown reward, and the goal is to recover a policy explaining the observed behaviour via the maximum causal entropy principle. We formulate the inverse problem by enforcing consistency with the expert mean-field term and long-run feature expectations, treating two reward classes within a unified occupation-measure framework. For finite-dimensional linear rewards, we give a convex dual reformulation with an explicit log-partition objective, and prove smoothness and curvature properties justifying constant-step-size gradient descent. For infinite-dimensional RKHS rewards, we develop a Lagrangian relaxation whose inner-maximising policy is characterised by a soft Bellman equation. The main obstacle is the absence of a discount-factor contraction. We resolve this by introducing a minorisation-based sub-stochastic kernel that yields a strict contraction of the soft Bellman operator. We establish Fréchet differentiability and Lipschitz smoothness of the log-likelihood score, leading to a gradient ascent algorithm with convergence guarantees. Two numerical examples, a malware-spread MFG and an RKHS-based consumer-choice model, show that the recovered policies closely match expert behaviour.

2606.16756 2026-06-16 cs.CV 新提交

3D Classification of Paramagnetic Rim Lesions in Multiple Sclerosis via Asymmetric QSM-FLAIR Modeling

多发性硬化症中顺磁性边缘病变的3D分类:基于非对称QSM-FLAIR建模

Veronica Pignedoli, Giacomo Boffa, Nicoletta Noceti, Matilde Inglese, Francesca Odone, Matteo Moro

发表机构 * MaLGa, DIBRIS, University of Genova(热那亚大学) DINOGMI, University of Genova(热那亚大学) IRCCS Azienda Ospedaliera Metropolitana(IRCCS大都会医院)

AI总结 提出一种3D多模态深度学习框架,利用非对称QSM-FLAIR建模对多发性硬化症中的顺磁性边缘病变进行自动分类,通过自监督预训练和对比正则化提升有限数据下的鲁棒性,在88名患者队列中验证了有效性。

Comments 10 pages, 3 figures, accepted at MICCAI 2026. Github link: https://github.com/veronicapignedoli/FRODO

详情
AI中文摘要

在磁敏感加权MRI上识别的顺磁性边缘病变(Rim$^+$)最近已成为多发性硬化症(MS)慢性活动性炎症的特异性生物标志物,并与长期残疾进展相关。然而,磁敏感成像和专家判读仍局限于专业中心,视觉评估耗时且可变,且Rim$^+$病变的低患病率给自动分析带来了严重的类别不平衡挑战。我们提出了一种3D多模态深度学习框架,用于从定量磁化率图(QSM)和FLAIR MRI中进行病变级别的Rim$^+$/Rim$^-$分类。该架构通过将QSM作为主要磁敏感驱动信号并用FLAIR衍生的结构上下文进行条件化,显式建模了模态非对称性。为了提高在有限数据下的鲁棒性,我们采用了自监督多模态预训练,随后进行带有对比正则化的监督微调。该方法在临床采集的88名MS患者队列中进行了评估,以专家病变标注作为参考标准。结果显示了相比先前架构的性能提升,支持了非对称多模态建模在自动识别慢性活动性病变中的有效性。

英文摘要

Paramagnetic rim lesions (Rim$^+$) identified on susceptibility-sensitive MRI have recently emerged as a specific biomarker of chronic active inflammation in Multiple Sclerosis (MS) and are associated with long-term disability progression. However, susceptibility imaging and expert interpretation remain limited to specialized centers, visual assessment is time-consuming and variable, and the low prevalence of Rim$^+$ lesions poses severe class imbalance challenges for automated analysis. We propose a 3D multimodal deep learning framework for lesion-level Rim$^+$/Rim$^-$ classification from Quantitative Susceptibility Mapping (QSM) and FLAIR MRI. The architecture explicitly models modality asymmetry by treating QSM as the primary susceptibility-driven signal and conditioning it with FLAIR-derived structural context. To improve robustness under limited data, we employ self-supervised multimodal pretraining followed by supervised fine-tuning with contrastive regularization. The method was evaluated on a clinically acquired cohort of 88 people with MS with expert lesion annotations as reference standard. Results highlight improved performance compared to prior architectures, supporting the effectiveness of asymmetric multimodal modeling for automated chronic active lesion identification.

2606.16753 2026-06-16 cs.CL cs.AI cs.LG 新提交

P3B3: A Multi-Turn Conversational Benchmark for Measuring European and Brazilian Portuguese Variety Bias in LLMs

P3B3:用于测量大语言模型中欧洲和巴西葡萄牙语变体偏差的多轮对话基准

Rafael Ferreira, Inês Vieira, Inês Calvo, James Furtado, Iago Paulo, Diogo Tavares, Diogo Glória-Silva, David Semedo, João Magalhães

发表机构 * NOVA University of Lisbon(新里斯本大学) NOVA LINCS(NOVA LINCS实验室)

AI总结 提出P3B3基准,通过专家策划的对话提示和评估框架,测量大语言模型在葡萄牙语变体(欧洲vs巴西)上的偏差和可控性,发现多数模型偏向巴西葡萄牙语。

Comments Accepted at MeLLM Workshop at ACL 2026

详情
AI中文摘要

随着大语言模型(LLMs)融入日常交流,捕捉区域语言变异对于可靠和公平的语言使用至关重要。在葡萄牙语中,欧洲(pt-PT)和巴西(pt-BR)变体仍然代表性不均,pt-BR在数据量上占主导地位,而LLM对葡萄牙语变体的偏好尚未得到充分探索。为弥补这一空白,我们引入了P3B3,一个由专家策划的语言变体无关的对话提示基准,以及一个用于测量变体偏差和可控性的评估框架。在多个模型上的实验表明,大多数LLM表现出对pt-BR的强烈偏差,且不同模型的可控性存在差异。这些结果凸显了需要在语言变体之间实现更平衡的多语言表示。

英文摘要

As Large Language Models (LLMs) become embedded in everyday communication, capturing regional linguistic variation is essential for reliable and equitable language use. In Portuguese, European (pt-PT) and Brazilian (pt-BR) varieties remain unevenly represented, with pt-BR dominating in data quantity, while LLM preference for Portuguese variants remains underexplored. To address this gap, we introduce P3B3, an expert-curated language variety agnostic benchmark of conversational prompts, along with an evaluation framework for measuring variety bias and controllability. Experiments on several models show that most LLMs exhibit a strong bias toward pt-BR, with variation in controllability across models. These results highlight the need for more balanced multilingual representation across language varieties.