arXivDaily arXiv每日学术速递 周一至周五更新
全部学科分类 2086
专题追踪
2510.16371 2026-05-08 cs.CV cs.AI cs.LG

Cataract-LMM Large-Scale Multi-Source Multi-Task Benchmark for Deep Learning in Surgical Video Analysis

白内障LMM大规模多源多任务基准用于手术视频分析中的深度学习

Mohammad Javad Ahmadi, Iman Gandomi, Parisa Abdi, Seyed-Farzad Mohammadi, Amirhossein Taslimi, Mehdi Khodaparast, Hassan Hashemi, Mahdi Tavakoli, Hamid D. Taghirad

发表机构 * Applied Robotics and AI Solutions (ARAS)(应用机器人与人工智能解决方案) Faculties of Electrical and Computer Engineering(电气与计算机工程学院) K.N. Toosi University of Technology(卡里姆·纳尼·托西技术大学) Translational Ophthalmology Research Center(转化眼科研究中心) Farabi Eye Hospital(法拉比眼科医院) Tehran University of Medical Sciences(德黑兰医科大学) Noor Ophthalmology Research Center(努尔眼科研究中心) University of Alberta(阿尔伯塔大学) Departments of Electrical and Computer Engineering & Biomedical Engineering(电气与计算机工程及生物医学工程系)

AI总结 本文提出一个包含3000个白内障超声乳化手术视频的数据集,用于训练通用深度学习模型,通过多任务基准验证了模型在手术流程识别、场景分割等方面的效果。

Comments 28 pages, 14 figures, 15 tables. Data descriptor for the Cataract-LMM benchmark dataset. Source code and dataset are available

详情
AI中文摘要

计算机辅助手术研究需要大量深度标注的视频数据集,以捕捉临床和技术的多样性。现有白内障手术资源缺乏多样性和标注深度,无法训练通用的深度学习模型。为此,我们提出了一个包含3000个白内障超声乳化手术视频的数据集,该数据集在两个手术中心采集,由不同经验的外科医生操作。数据集提供了四个标注层:时间手术阶段、仪器和解剖结构实例分割、仪器-组织相互作用跟踪以及基于ICOSCAR和GRASIS适应的竞争力评分。我们通过在四个任务上基准测试深度学习模型来展示数据集的技术用途:流程识别、场景分割、仪器-组织相互作用跟踪和自动化技能评估。此外,我们通过在其中一个手术中心训练并在另一个保留的中心评估,建立了领域适应基线用于阶段识别和实例分割。最终,这些多源采集、多层标注和配对技能-运动学标签促进了手术流程分析、场景理解和基于竞争力的培训研究中通用多任务模型的发展。

英文摘要

Computer-assisted surgery research requires large, deeply annotated video datasets that capture clinical and technical variability. Existing cataract surgery resources lack the diversity and annotation depth required to train generalizable deep-learning models. To address this gap, we present a dataset of 3,000 phacoemulsification cataract surgery videos acquired at two surgical centers from surgeons with varying expertise. The dataset provides four annotation layers: temporal surgical phases, instance segmentation of instruments and anatomical structures, instrument-tissue interaction tracking, and quantitative skill scores based on competency rubrics adapted from ICO-OSCAR and GRASIS. We demonstrate the technical utility of the dataset through benchmarking deep learning models across four tasks: workflow recognition, scene segmentation, instrument-tissue interaction tracking, and automated skill assessment. Furthermore, we establish a domain-adaptation baseline for phase recognition and instance segmentation by training on one surgical center and evaluating on a held-out center. Ultimately, these multi-source acquisitions, multi-layer annotations, and paired skill-kinematic labels facilitate the development of generalizable multi-task models for surgical workflow analysis, scene understanding, and competency-based training research.

2510.11068 2026-05-08 cs.LG eess.AS eess.IV

Efficient Test-Time Adaptation through Latent Subspace Coefficients Search

通过潜在子空间系数搜索实现高效的测试时适应

Xinyu Luo, Jie Liu, Kecheng Chen, Junyi Yang, Bo Ding, Arindam Basu, Haoliang Li

发表机构 * Department of Electrical Engineering, City University of Hong Kong(香港城市大学电子工程系)

AI总结 本文提出ELaTTA框架,通过在设备上预计算的潜在主子空间优化低维系数向量,实现高效单实例测试时适应,提升鲁棒性并减少计算和内存开销。

Comments Under review

详情
AI中文摘要

现实部署中,模型常面临分布偏移,使测试时适应(TTA)成为鲁棒性关键。然而,大多数TTA方法不适用于边缘部署,因为它们依赖反向传播、激活缓冲或测试时小批量,导致高延迟和内存开销。我们提出ELaTTA(Efficient Latent Test-Time Adaptation),一种无梯度框架,用于在严格设备约束下进行单实例TTA。ELaTTA冻结模型权重,并通过优化预计算的源诱导主潜在子空间中的低维系数向量来适应每个测试样本,该子空间通过截断SVD离线计算并以极小开销存储。在推理时,ELaTTA通过优化k-D系数以CMA-ES,有效优化高斯平滑目标并提高决策边界附近的稳定性。在六个基准和多种架构上,ELaTTA在严格和持续单实例协议下均达到最先进的准确性,同时计算量减少高达63倍,峰值内存减少高达11倍。我们进一步在ZYNQ-7020平台上展示了设备部署。

英文摘要

Real-world deployment often exposes models to distribution shifts, making test-time adaptation (TTA) critical for robustness. Yet most TTA methods are unfriendly to edge deployment, as they rely on backpropagation, activation buffering, or test-time mini-batches, leading to high latency and memory overhead. We propose \textbf{ELaTTA} (\textit{Efficient Latent Test-Time Adaptation}), a gradient-free framework for single-instance TTA under strict on-device constraints. ELaTTA freezes model weights and adapts each test sample by optimizing a low-dimensional coefficient vector in a source-induced principal latent subspace, pre-computed offline via truncated SVD and stored with negligible overhead. At inference, ELaTTA encourages prediction confidence by optimizing the $k$-D coefficients with CMA-ES, effectively optimizing a Gaussian-smoothed objective and improving stability near decision boundaries. Across six benchmarks and multiple architectures, ELaTTA achieves state-of-the-art accuracy under both strict and continual single-instance protocols, while reducing compute by up to \emph{63$\times$} and peak memory by up to \emph{11$\times$}. We further demonstrate on-device deployment on a ZYNQ-7020 platform.

2510.10241 2026-05-08 cs.CL cs.IR

ImCoref-CeS: An Improved Lightweight Pipeline for Coreference Resolution with LLM-based Checker-Splitter Refinement

ImCoref-CeS: 一种改进的轻量级管道用于基于LLM的检查-分割细化的指代消解

Kangyang Luo, Yuzhuo Bai, Shuzheng Si, Cheng Gao, Zhitong Wang, Yingli Shen, Wenhao Li, Zhu Liu, Yufeng Han, Jiayi Wu, Cunliang Kong, Maosong Sun

发表机构 * Department of Computer Science and Technology, Tsinghua University(清华大学计算机科学与技术系) Institute for AI, Tsinghua University(清华大学人工智能研究院) East China Normal University(华东师范大学) Jiangsu Collaborative Innovation Center for Language Ability(江苏省语言能力协同创新中心)

AI总结 本文提出ImCoref-CeS框架,结合增强的监督模型与LLM推理,通过改进的指代消解方法和LLM多角色检查-分割代理提升性能。

Comments Accepted by ACL2026 main

详情
AI中文摘要

指代消解(CR)是自然语言处理中的关键任务。当前研究面临关键抉择:是否进一步探索基于小语言模型的监督神经方法的潜力,还是拥抱大型语言模型(LLMs)的强大能力。然而,有效结合两者的优势仍显不足。为此,我们提出ImCoref-CeS框架,整合增强的监督模型与LLM推理。首先,我们提出改进的CR方法(ImCoref),通过引入轻量级桥梁模块增强长文本编码能力,设计双 affine 得分器全面捕捉位置信息,并调用混合提及正则化提升训练效率。重要的是,我们采用LLM作为多角色检查-分割代理,验证ImCoref预测的候选提及(过滤无效提及)和指代结果(分割错误集群)。广泛实验表明,ImCoref-CeS在现有最先进(SOTA)方法上表现更优。

英文摘要

Coreference Resolution (CR) is a critical task in Natural Language Processing (NLP). Current research faces a key dilemma: whether to further explore the potential of supervised neural methods based on small language models, whose detect-then-cluster pipeline still delivers top performance, or embrace the powerful capabilities of Large Language Models (LLMs). However, effectively combining their strengths remains underexplored. To this end, we propose \textbf{ImCoref-CeS}, a novel framework that integrates an enhanced supervised model with LLM-based reasoning. First, we present an improved CR method (\textbf{ImCoref}) to push the performance boundaries of the supervised neural method by introducing a lightweight bridging module to enhance long-text encoding capability, devising a biaffine scorer to comprehensively capture positional information, and invoking a hybrid mention regularization to improve training efficiency. Importantly, we employ an LLM acting as a multi-role Checker-Splitter agent to validate candidate mentions (filtering out invalid ones) and coreference results (splitting erroneous clusters) predicted by ImCoref. Extensive experiments demonstrate the effectiveness of ImCoref-CeS, which achieves superior performance compared to existing state-of-the-art (SOTA) methods.

2510.09316 2026-05-08 cs.LG cs.CL

Large Language Model Prompt Datasets: An In-depth Analysis and Insights

大语言模型提示数据集:深入分析与洞察

Yuanming Zhang, Yan Lin, Arijit Khan, Huaiyu Wan

发表机构 * School of Computer Science and Technology(计算机科学与技术学院) Beijing Jiaotong University(北京交通大学) Department of Computer Science(计算机科学系) Aalborg University(奥尔堡大学)

AI总结 本文深入分析了129个异构LLM提示数据集,通过多层级语言分析揭示提示与一般文本的区别模式,并通过三个下游实验验证了提示过滤、领域分类和提示质量预测的实用性。

详情
AI中文摘要

我们整理了129个异构LLM提示数据集(>1.22TB,>673M实例)并构建了结构化的分类体系,对七个代表性语料库进行了多层级语言分析(词汇、语法和语义),揭示了提示与一般文本的区别系统模式。三个下游实验验证了其实际效用:提示过滤(F1=0.90)、领域分类(Macro-F1=0.975)和提示质量预测(AUC=0.792),均无需调用额外模型。核心发现是62个语法特征(词性+依赖分布)作为独特高效的路由原始构件,能够在不使用GPU和语料库词汇的情况下,以1.9倍更低的单请求延迟(3.0ms vs. 5.7ms)恢复超过93%的GPU嵌入精度。互补的判别-预测分歧显示,对路由最有用的特征正是那些与响应质量最负相关。词汇多样性(Cohen's d=0.71)主导质量信号,但携带极小的路由权重,直接推动了双阶段流水线设计。我们的数据集和代码已公开。

英文摘要

We compile 129 heterogeneous LLM prompt datasets (>1.22 TB, >673M instances) into a structured taxonomy and conduct a multi-level linguistic analysis (lexical, syntactic, and semantic) on seven representative corpora, surfacing systematic patterns that distinguish prompts from general text. Three downstream experiments validate practical utility: prompt filtering (F1 = 0.90), domain classification (Macro-F1 = 0.975), and prompt quality prediction (AUC = 0.792), all without invoking any additional model. A central finding is that 62-d syntactic features (POS + dependency distributions) serve as a uniquely efficient routing primitive, recovering >93% of GPU-embedding accuracy at 1.9 $\times$ lower single-request latency (3.0 ms vs. 5.7 ms) with no GPU and no corpus vocabulary. A complementary discriminative--predictive divergence shows that features most useful for routing are precisely those most negatively correlated with response quality, while lexical diversity (Cohen's $d$ = 0.71) dominates the quality signal but carries minimal routing weight, directly motivating two-stage pipeline design. Our datasets and code are available.

2510.08750 2026-05-08 cs.LG cs.CL

Exploring Cross-Client Memorization of Training Data in Large Language Models for Federated Learning

探索联邦学习中大语言模型训练数据的跨客户端记忆化

Tinnakit Udsa, Can Udomcharoenchaikit, Patomporn Payoungkhamdee, Sarana Nutanong, Norrathep Rattanavipanon

发表机构 * School of Information Science and Technology, VISTEC(信息科学与技术学院,VISTEC) College of Computing, Prince of Songkla University(颂克拉大学计算机学院)

AI总结 本文研究联邦学习中大语言模型训练数据的记忆化问题,提出跨客户端记忆化测量框架,分析客户端间与客户端内记忆化差异及影响因素。

Comments Accepted to The 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026)

详情
AI中文摘要

联邦学习(FL)允许在不共享原始数据的情况下进行协作训练,但仍然存在训练数据记忆化风险。现有FL记忆化检测技术聚焦单个样本,低估了跨样本记忆化的潜在风险。相比之下,集中式学习(CL)中的方法通过细粒度评估所有训练样本的记忆化,但这些方法假设集中式数据访问,无法直接应用于FL。本文提出一个框架,利用细粒度的跨样本记忆化测量,量化FL中的客户端内和客户端间记忆化。基于此框架,我们进行了两项研究:(1)测量客户端间的细微记忆化;(2)探讨影响记忆化的关键因素,包括解码策略、前缀长度和FL算法。我们的发现表明,FL模型确实更倾向于记忆客户端数据,尤其是客户端内的数据,而非客户端间的数据,记忆化受训练和推理因素影响。

英文摘要

Federated learning (FL) enables collaborative training without raw data sharing, but still risks training data memorization. Existing FL memorization detection techniques focus on one sample at a time, underestimating more subtle risks of cross-sample memorization. In contrast, recent work on centralized learning (CL) has introduced fine-grained methods to assess memorization across all samples in training data, but these assume centralized access to data and cannot be applied directly to FL. We bridge this gap by proposing a framework that quantifies both intra- and inter-client memorization in FL using fine-grained cross-sample memorization measurement across all clients. Based on this framework, we conduct two studies: (1) measuring subtle memorization across clients and (2) examining key factors that influence memorization, including decoding strategies, prefix length, and FL algorithms. Our findings reveal that FL models do memorize client data, particularly intra-client data, more than inter-client data, with memorization influenced by training and inferencing factors.

2510.07516 2026-05-08 cs.AI cs.CL

CompassLLM: A Multi-Agent Approach toward Geo-Spatial Reasoning for Popular Path Query

CompassLLM: 一种面向流行路径查询的多智能体方法

Md. Nazmul Islam Ananto, Shamit Fatin, Mohammed Eunus Ali, Md Rizwan Parvez

发表机构 * Bangladesh University of Engineering and Technology(孟加拉工程科技大学) University of Utah(犹他大学) Monash University(莫纳什大学) Qatar Computing Research Institute(卡塔尔计算研究所)

AI总结 CompassLLM通过多智能体框架解决流行路径查询问题,结合空间推理能力提升搜索和生成性能,实验显示其在准确性和成本效益方面表现优异。

详情
AI中文摘要

流行路径查询——从历史轨迹数据中识别最频繁的路线——在城市规划、导航优化和旅行推荐中有重要应用。尽管传统算法和机器学习方法在该领域取得成功,但它们在数据更新时通常需要模型训练、参数调优和重新训练。随着大型语言模型(LLMs)在空间和图推理方面的能力不断增强,研究如何将这些模型应用于地理空间问题变得越来越重要。我们介绍了CompassLLM,一种新颖的多智能体框架,通过智能利用LLMs的推理能力进入地理空间领域以解决流行路径查询。CompassLLM采用两阶段流程:SEARCH阶段识别流行路径,GENERATE阶段在历史轨迹数据中没有现有路径时合成新路径。在真实和合成数据集上的实验表明,CompassLLM在SEARCH阶段表现出色,在GENERATE阶段具有竞争力,同时具有成本效益。

英文摘要

The popular path query - identifying the most frequented routes between locations from historical trajectory data - has important applications in urban planning, navigation optimization, and travel recommendations. While traditional algorithms and machine learning approaches have achieved success in this domain, they typically require model training, parameter tuning, and retraining when accommodating data updates. As Large Language Models (LLMs) demonstrate increasing capabilities in spatial and graph-based reasoning, there is growing interest in exploring how these models can be applied to geo-spatial problems. We introduce CompassLLM, a novel multi-agent framework that intelligently leverages the reasoning capabilities of LLMs into the geo-spatial domain to solve the popular path query. CompassLLM employs its agents in a two-stage pipeline: the SEARCH stage that identifies popular paths, and a GENERATE stage that synthesizes novel paths in the absence of an existing one in the historical trajectory data. Experiments on real and synthetic datasets show that CompassLLM demonstrates superior accuracy in SEARCH and competitive performance in GENERATE while being cost-effective.

2510.01719 2026-05-08 cs.CL

What MLLMs Learn about When they Learn about Multimodal Reasoning

大语言模型在学习多模态推理时所学的内容

Jiwan Chung, Neel Joshi, Pratyusha Sharma, Youngjae Yu, Vibhav Vineet

发表机构 * Microsoft Research AI Frontiers(微软研究院人工智能前沿) Yonsei University(延世大学) Seoul National University(首尔国立大学)

AI总结 本文提出MathLens基准测试,通过分解性能为感知、推理和多模态特定组件,揭示多模态推理模型评估中隐含的假设。研究显示,不同训练策略导致的能力谱系不同,多模态特定错误占比上升,表明进展反映的是子技能平衡变化而非统一提升。

详情
AI中文摘要

多模态推理模型的评估通常简化为单一准确率分数,隐含将推理视为单一能力。我们引入MathLens基准测试,通过操作性分解性能为感知、推理和多模态特定组件,揭示这一假设。每个问题均基于符号规范衍生,并附带视觉图表、纯文本变体、多模态问题和定向感知探针,实现对各组件的受控测量。通过这种分解,我们发现常见训练策略诱导出系统性不同的能力谱系,这些差异在聚合准确率下不可见。强化学习主要提升感知基础和对图表变化的鲁棒性,而文本SFT则通过反思推理获得收益。相反,随着感知和推理能力提升,剩余错误中越来越多的错误归类为多模态特定。这些结果表明,多模态推理的显着进展反映的是子技能之间平衡的转变,而非统一的进步,促使评估超越标量准确率。

英文摘要

Evaluation of multimodal reasoning models is typically reduced to a single accuracy score, implicitly treating reasoning as a unitary capability. We introduce MathLens, a benchmark of textbook-style geometry problems that exposes this assumption by operationally decomposing performance into perception, reasoning, and multimodal-specific components. Each problem is derived from a symbolic specification and accompanied by visual diagrams, text-only variants, multimodal questions, and targeted perceptual probes, enabling controlled measurement of each component. Using this decomposition, we show that common training strategies induce systematically different capability profiles that are invisible under aggregate accuracy. Reinforcement learning primarily improves perceptual grounding and robustness to diagram variation, while textual SFT yields gains through reflective reasoning. In contrast, as perception and reasoning improve, a growing fraction of remaining errors fall outside these components and are categorized as multimodal-specific. These results suggest that apparent progress in multimodal reasoning reflects shifting balances among subskills rather than uniform advancement, motivating evaluation beyond scalar accuracy.

2510.01457 2026-05-08 cs.LG

A Forensic Analysis of Synthetic Data in RL: Diagnosing and Solving Algorithmic Failures in Model-Based Policy Optimization

对强化学习中合成数据的取证分析:诊断和解决基于模型的策略优化中的算法故障

Brett Barkley, David Fridovich-Keil

发表机构 * Department of Electrical and Computer Engineering(电气与计算机工程系) The University of Texas at Austin(德克萨斯大学奥斯汀分校) Department of Aerospace Engineering and Engineering Mechanics(航空航天工程与工程力学系)

AI总结 本文研究了基于模型的策略优化中合成数据导致性能下降的原因,提出FTFL方法修复算法故障,提升DMC任务性能并保持Gym表现。

详情
AI中文摘要

合成数据在数据高效的Dyna风格基于模型的强化学习中至关重要,但可能降低性能。我们研究了基于模型的策略优化(MBPO)中这一失败现象,该方法通过模型生成的合成状态转移进行actor-critic更新。尽管MBPO在OpenAI Gym上表现出强大的样本效率,但最近的研究表明,在DeepMind Control Suite(DMC)中,它经常在Soft Actor-Critic(SAC)之下,尽管两者都涉及基于MuJoCo的感知连续控制。我们识别出两种耦合原因:动态与奖励目标之间的尺度不匹配,这会抑制奖励学习并导致批评者低估,以及残余的下一状态预测,这会增加模型方差并产生不可靠的合成转换。我们引入Fixing That Free Lunch(FTFL),一种最小的修复方法,结合独立的目标归一化和直接的下一状态预测。FTFL在七个先前失败的DMC任务中的五个中优于SAC,同时保持MBPO在Gym上的强大表现。我们进一步表明,MBPO派系算法,包括基于模型不确定性的变体,这些变体根据模型不确定性过滤、惩罚或拒绝合成转换,除非应用FTFL到它们共享的学得模型主干,否则仍会继承这些失败。更广泛地说,我们的结果展示了受限评估如何将环境特定的假设编码到算法设计中,从而推动映射MDP结构到算法故障模式的分类学。

英文摘要

Synthetic data is central to data-efficient Dyna-style model-based reinforcement learning, but it can also degrade performance. We study this failure in Model-Based Policy Optimization (MBPO), which performs actor-critic updates using model-generated synthetic state transitions. Although MBPO reports strong sample-efficiency gains on OpenAI Gym, recent work shows that it often underperforms Soft Actor-Critic (SAC), its non-Dyna base, in the DeepMind Control Suite (DMC), despite both suites involving MuJoCo-based proprioceptive continuous control. We identify two coupled causes of this collapse: scale mismatch between dynamics and reward targets, which suppresses reward learning and induces critic underestimation, and residual next-state prediction, which inflates model variance and produces unreliable synthetic transitions. We introduce Fixing That Free Lunch (FTFL), a minimal repair that combines independent target normalization with direct next-state prediction. FTFL outperforms SAC in five of seven previously failing DMC tasks while preserving MBPO's strong Gym performance. We further show that MBPO-lineage algorithms, including uncertainty-aware variants that filter, penalize, or reject synthetic transitions based on model uncertainty, still inherit these failures unless FTFL is applied to their shared learned-model backbone. More broadly, our results show how benchmark-limited evaluation can encode environment-specific assumptions into algorithm design, motivating taxonomies that map MDP structure to algorithmic failure modes.

2509.23629 2026-05-08 cs.AI cond-mat.dis-nn cond-mat.stat-mech cs.LG physics.soc-ph

Emergent Slow Thinking in LLMs as Inverse Tree Freezing

大语言模型中涌现的慢思考作为逆树冻结

Sihan Hu, Xiansheng Cai, Yuan Huang, Zhiyuan Yao, Linfeng Zhang, Pan Zhang, Youjin Deng, Kun Chen

发表机构 * Hefei National Laboratory, University of Science and Technology of China(中国科学技术大学合肥微尺度物质科学国家实验室) Hefei National Laboratory for Physical Sciences at the Microscale and Department of Modern Physics, University of Science and Technology of China(中国科学技术大学合肥微尺度物质科学国家实验室和现代物理系) Institute of Theoretical Physics, Chinese Academy of Sciences(中国科学院理论物理研究所) School of Fundamental Physics and Mathematical Sciences, Hangzhou Institute for Advanced Study, UCAS(杭州高等研究院基础物理与数学科学学院) DP Technology(DP技术) Lanzhou Center for Theoretical Physics, Key Laboratory of Theoretical Physics of Gansu Province, Key Laboratory of Quantum Theory and Applications of MoE, Gansu Provincial Research Center for Basic Disciplines of Quantum Physics, Lanzhou University(兰州理论物理中心、甘肃省理论物理重点实验室、教育部量子理论与应用重点实验室、甘肃省量子物理基础学科省重点实验室、兰州大学) AI for Science Institute, Beijing(北京人工智能科学研究院)

AI总结 本文通过统计物理视角揭示大语言模型中慢思考的涌现机制,提出逆树冻结结构,并提出Annealed-RLVR方法提升模型性能。

Comments 34 pages, 17 figures, 1 table

详情
AI中文摘要

可验证奖励的强化学习(RLVR)使大语言模型能够从稀疏最终答案信号中获得慢速多步推理能力。我们提供了这一现象的统计物理图景。我们证明,自回归模型的有限容量迫使它将指数级大的前缀空间压缩成预测状态的马尔可夫网络,在此慢思考展开为随机游走——即概念网络(CoNet)图景。在CoNet中,RLVR动态由两种机制支配:兼容路径的融合和不兼容路径间的受挫竞争。二者共同驱动网络经历成核、生长和冻结过程,形成多输入单输出的定向逆树。该图景重现了15亿参数LLM的训练动态,并得出三个预测:推理链长度随稀疏拓扑的几何必要性增长;SFT通过桥节点破裂引发灾难性遗忘;受挫驱动策略崩溃。基于逆树冻结固有的时间结构,我们提出Annealed-RLVR——在最大受挫时刻进行短暂SFT干预。该方法在分布内和分布外基准测试中均优于标准RLVR,在高采样预算下收益最大,而标准RLVR在此时崩溃。相同的SFT干预在树冻结后触发灾难性遗忘,证明时间是关键活性成分。

英文摘要

Reinforcement learning with verifiable rewards (RLVR) enables large language models to acquire slow, multi-step reasoning from sparse final-answer signals. We provide a statistical-physics picture of this emergence. We show that an autoregressive model's finite capacity forces it to compress its exponentially large prefix space into a Markov network of predictive states, on which slow thinking unfolds as a random walk -- the Concept Network (CoNet) picture. Within CoNet, RLVR dynamics are governed by two mechanisms: merging of compatible paths and frustrated competition among incompatible ones. Together they drive the network through nucleation, growth, and freezing into multi-input, single-output directed inverse trees. The picture reproduces the training dynamics of a 1.5-billion-parameter LLM and yields three predictions: reasoning chains lengthen as a geometric necessity of sparse topology; SFT induces catastrophic forgetting through bridge-node rupture; and frustration drives policy collapse. Building on the structural timing inherent in inverse-tree freezing, we propose Annealed-RLVR -- a brief SFT intervention at the moment of maximum frustration. It outperforms standard RLVR on both in- and out-of-distribution benchmarks, with the largest gains at high sampling budgets where standard RLVR collapses. The same SFT applied after the trees freeze instead triggers catastrophic forgetting, isolating timing as the active ingredient.

2509.17291 2026-05-08 cs.LG

GraphWeave: Interpretable and Robust Graph Generation via Random Walk Trajectories

GraphWeave:通过随机游走轨迹实现可解释且鲁棒的图生成

Rahul Nandakumar, Deepayan Chakrabarti

发表机构 * The University of Texas at Austin(德克萨斯大学奥斯汀分校)

AI总结 GraphWeave通过分离模式生成与图构建,利用随机游走轨迹生成真实图,优于现有方法,尤其在大规模图结构上表现突出,且速度快。

Comments 18 pages, 4 figures. Accepted at ECML-PKDD 2025

详情
AI中文摘要

给定一组未知家族的图,我们希望生成新图。近期方法在图嵌入或离散节点边空间上进行扩散。然而,简单改变嵌入(如加噪声)可能导致图的不可解释变化。在离散空间扩散中,每一步可能添加或移除许多节点/边。难以预测许多扩散步骤后的图模式。我们提出的方法GraphWeave采用不同方法。我们分离模式生成和图构建。为了在训练图中找到模式,我们观察随机游走中向量的变换。然后生成新图分为两步。首先生成符合学习模式的现实随机游走轨迹。然后找到最佳图以适应这些轨迹。优化联合推断所有边,提高对错误的鲁棒性。在四个模拟和五个现实基准数据集上,GraphWeave优于现有方法。最大的差异出现在大规模图结构上,如PageRank、切分、社区、度分布和流。GraphWeave也比其最接近的竞争对手快10倍。最后,GraphWeave简单,只需一个Transformer和标准优化器。

英文摘要

Given a set of graphs from some unknown family, we want to generate new graphs from that family. Recent methods use diffusion on either graph embeddings or the discrete space of nodes and edges. However, simple changes to embeddings (say, adding noise) can mean uninterpretable changes in the graph. In discrete-space diffusion, each step may add or remove many nodes/edges. It is hard to predict what graph patterns we will observe after many diffusion steps. Our proposed method, called GraphWeave, takes a different approach. We separate pattern generation and graph construction. To find patterns in the training graphs, we see how they transform vectors during random walks. We then generate new graphs in two steps. First, we generate realistic random walk "trajectories" which match the learned patterns. Then, we find the optimal graph that fits these trajectories. The optimization infers all edges jointly, which improves robustness to errors. On four simulated and five real-world benchmark datasets, GraphWeave outperforms existing methods. The most significant differences are on large-scale graph structures such as PageRank, cuts, communities, degree distributions, and flows. GraphWeave is also 10x faster than its closest competitor. Finally, GraphWeave is simple, needing only a transformer and standard optimizers.

2509.14594 2026-05-08 cs.AI

SynBench: A Benchmark for Differentially Private Text Generation

SynBench:差分隐私文本生成的基准测试

Yidan Sun, Viktor Schlegel, Srinivasan Nandakumar, Iqra Zahid, Yuping Wu, Yulong Wu, Hao Li, Jie Zhang, Warren Del-Pinto, Goran Nenadic, Siew Kei Lam, Anil Anthony Bharath

发表机构 * Imperial College London, Imperial Global Singapore(帝国学院伦敦,帝国全球新加坡) University of Manchester, United Kingdom(曼彻斯特大学,英国) CFAR and IHPC, Agency for Science, Technology and Research (A*STAR), Singapore(CFAR和IHPC,科技研究局(A*STAR),新加坡) Nanyang Technological University, Singapore(南洋理工大学,新加坡) Imperial College London, United Kingdom(帝国学院伦敦,英国)

AI总结 本文提出SynBench基准测试,通过标准化评估框架和隐私审计,评估不同规模的LLM生成模型在差分隐私下的性能,揭示隐私数据生成与预训练数据的关联性对生成质量的影响。

Comments 16 pages

详情
AI中文摘要

差分隐私(DP)保证的合成文本生成作为一种原则性方法,能够促进敏感数据集在机构和监管边界上的共享,同时限制重新识别和成员推断的风险。基于大语言模型(LLM)的方法取得了有希望的结果;然而,比较受到不同评估设置和“隐私”数据集的影响,潜在的预训练污染未被考虑,且隐私保证未通过DP审计验证。为了推动该领域的发展,我们引入了一个统一的评估框架,包含标准化的效用和保真度指标和隐私审计,涵盖九个精心挑选的数据集,这些数据集捕捉了领域特定的复杂性,如技术术语、长上下文依赖性和专门的文档结构。在大规模的实证研究中,我们基准测试了不同规模(1-8B)的LLM基于的最先进DP文本生成器。我们的结果表明,DP合成文本生成仍然是一个未解决的挑战,质量随着私有数据集与生成器预训练语料库的偏离程度增加而恶化。我们的新型合成文本成员推断攻击(MIA)解释了这一观察:当LLM在没有DP的情况下预训练部分“私有”数据以生成时,合成数据质量会被高估。最后,我们的工作提供了第一项定量证据,表明这种“公共预训练和私有生成”范式会破坏真实私有数据集的隐私保证界线。

英文摘要

Synthetic text generation with Differential Privacy (DP) guarantees emerges as a principled approach that can enable the sharing of sensitive datasets across institutional and regulatory boundaries, while bounding the risks of re-identification and membership inference. LLM-based methods deliver promising results; however, comparisons are exacerbated by differing evaluation setups and "private" datasets, potential pre-training contamination is not considered and guarantees are not verified with DP audits. To advance this field, we introduce a unified evaluation framework with standardised utility and fidelity metrics and privacy audits, encompassing nine curated datasets that capture domain-specific complexities such as technical jargon, long-context dependencies, and specialised document structures. In a large-scale empirical study, we benchmark LLM-based state-of-the-art DP text generators of varying sizes (between 1--8B). Our results indicate that DP synthetic text generation remains an unsolved challenge, with quality deteriorating more as the private datasets deviate further from the generators' pre-training corpora. Our novel synthetic text membership inference attack (MIA) explains this observation: Synthetic data quality is overestimated when LLMs have been pre-trained -- without DP -- on portions of the "private" data to be generated. Finally, our work provides the first quantitative evidence that this "public pre-training and private generation" paradigm invalidates the guaranteed privacy bounds of real-world private datasets.

2509.14225 2026-05-08 cs.LG stat.ML

Defending Diffusion Models Against Membership Inference Attacks via Higher-Order Langevin Dynamics

通过高阶 Langevin 动力学防御扩散模型的成员推断攻击

Benjamin Sterling, Yousef El-Laham, Mónica F. Bugallo

发表机构 * Department of Applied Math & Statistics(应用数学与统计学系) Stony Brook University(石溪大学) Department of Electrical and Computer Engineering(电气与计算机工程系)

AI总结 本文提出利用高阶 Langevin 动力学防御扩散模型的成员推断攻击,通过引入辅助变量和联合扩散过程提升模型安全性,通过AUROC和FID指标验证有效性。

Comments 11 pages, 4 figures

详情
AI中文摘要

近年来,生成式人工智能的应用进展引发了新的数据安全问题。本文聚焦于防御扩散模型的成员推断攻击。此类攻击发生在攻击者能够确定特定数据点是否用于训练模型时。尽管扩散模型本质上比其他生成模型更抗成员推断攻击,但仍存在漏洞。本文提出的防御方法利用临界阻尼的高阶 Langevin 动力学,引入多个辅助变量和这些变量的联合扩散过程。其核心思想是辅助变量的存在会引入外部随机性,帮助在扩散过程早期破坏敏感输入数据。该概念在玩具数据集和语音数据集上通过AUROC曲线和FID度量进行了理论研究和验证。

英文摘要

Recent advances in generative artificial intelligence applications have raised new data security concerns. This paper focuses on defending diffusion models against membership inference attacks. This type of attack occurs when the attacker can determine if a certain data point was used to train the model. Although diffusion models are intrinsically more resistant to membership inference attacks than other generative models, they are still susceptible. The defense proposed here utilizes critically-damped higher-order Langevin dynamics, which introduces several auxiliary variables and a joint diffusion process along these variables. The idea is that the presence of auxiliary variables mixes external randomness that helps to corrupt sensitive input data earlier on in the diffusion process. This concept is theoretically investigated and validated on a toy dataset and a speech dataset using the Area Under the Receiver Operating Characteristic (AUROC) curves and the FID metric.

2508.09193 2026-05-08 cs.LG cs.AI

Multi-Objective Instruction-Aware Representation Learning in Procedural Content Generation RL

多目标指令感知的生成内容生成强化学习表示学习

Sung-Hyun Kim, Geum-Hwan Hwang, In-Chang Baek, Seo-Young Lee, Kyung-Joong Kim

发表机构 * Gwangju Institute of Science and Technology(全州科学技术学院)

AI总结 本文提出MIPCGRL方法,通过整合句子嵌入条件,提升多目标指令下的生成内容可控性,实验显示可控性提升13.8%。

Comments 9 pages, 4 figures

详情
AI中文摘要

最近生成模型的进步强调了自然语言作为高度表达性和易用的控制内容生成模态的重要性。然而,现有的受指令强化学习(IPCGRL)方法在处理复杂多目标指令时难以充分利用文本输入的表达丰富性,导致可控性有限。为了解决这个问题,我们提出了MIPCGRL,一种用于受指令生成器的多目标表示学习方法,其整合了句子嵌入作为条件。MIPCGRL通过整合多标签分类和多头回归网络有效训练了多目标嵌入空间。实验结果表明,所提出的方法在多目标指令下实现了可控性提升高达13.8%。处理复杂指令的能力使内容生成更加表达性和灵活。

英文摘要

Recent advancements in generative modeling emphasize the importance of natural language as a highly expressive and accessible modality for controlling content generation. However, existing instructed reinforcement learning for procedural content generation (IPCGRL) method often struggle to leverage the expressive richness of textual input, especially under complex, multi-objective instructions, leading to limited controllability. To address this problem, we propose \textit{MIPCGRL}, a multi-objective representation learning method for instructed content generators, which incorporates sentence embeddings as conditions. MIPCGRL effectively trains a multi-objective embedding space by incorporating multi-label classification and multi-head regression networks. Experimental results show that the proposed method achieves up to a 13.8\% improvement in controllability with multi-objective instructions. The ability to process complex instructions enables more expressive and flexible content generation.

2508.06412 2026-05-08 cs.LG cs.CL

Sample-efficient LLM Optimization with Reset Replay

基于重置回放的高效大语言模型优化

Zichuan Liu, Jinyu Wang, Lei Song, Jiang Bian

发表机构 * Carnegie Mellon University(卡内基梅隆大学) Microsoft Research Asia(微软亚洲研究院)

AI总结 本文提出LoRR方法,通过高重播训练和周期性重置策略提升偏好优化的样本效率,实验表明其在数学和通用推理基准上显著提升性能。

详情
AI中文摘要

近年来,大语言模型(LLM)后训练中的进展,特别是通过强化学习和偏好优化,是提升其推理能力的关键。然而,这些方法往往存在样本效率低和易受先验偏差影响的问题。为解决这些问题,我们引入LLM优化与重置回放(LoRR),一种通用且强大的插件,用于增强基于偏好的优化的样本效率。其核心机制使高重播训练能够最大化每个数据批次的效用。为缓解过拟合,LoRR采用周期性重置策略,重用初始数据和策略以维持网络可塑性,并进一步采用混合优化目标以更好地利用训练数据。广泛实验表明,LoRR显著提升了各种偏好优化方法在数学和通用推理基准上的性能。值得注意的是,一个带有LoRR的迭代DPO框架在具有挑战性的数学任务上实现了与许多复杂或计算密集型基线相当的性能。我们的发现表明,LoRR提供了一种实用且样本高效的范式,从有限的离线数据中解锁更大的性能,通过最小的现有后训练工作流程修改。

英文摘要

Recent advancements in LLM post-training, particularly through reinforcement learning and preference optimization, are key to boosting their reasoning capabilities. However, these methods often suffer from low sample efficiency and a susceptibility to primacy bias, a phenomenon where overfitting to initial experiences diminishes network plasticity and damages the learning process. To address these challenges, we introduce LLM optimization with Reset Replay (LoRR), a general and powerful plugin for enhancing sample efficiency in preference-based optimization. Its core mechanism enables high-replay training to maximize the utility of each data batch. To mitigate overfitting, LoRR orchestrates a periodic reset strategy that reuses the initial data and policy to maintain network plasticity, and further adopts a hybrid optimization objective to better exploit training data. Extensive experiments show that LoRR significantly boosts the performance of various preference optimization methods on both mathematical and general reasoning benchmarks. Notably, an iterative DPO framework augmented with LoRR achieves comparable performance on challenging math tasks, rivaling many complex or computationally expensive baselines. Our findings highlight that LoRR offers a practical and sample-efficient paradigm from limited offline data, unlocking greater performance with minimal changes to existing post-training workflows.

2507.00480 2026-05-08 cs.LG stat.ML

Posterior Inference in Latent Space for Scalable Constrained Black-box Optimization

潜在空间中的后验推断用于可扩展的约束黑盒优化

Kiyoung Om, Kyuil Sim, Taeyoung Yun, Hyeongyu Kang, Jinkyoo Park

发表机构 * Korea Advanced Institute of Science and Technology (KAIST)(韩国科学技术院)

AI总结 本文将约束黑盒优化转化为后验推断问题,在生成模型的潜在空间中进行推断,通过训练流模型和扩散模型提升候选点搜索效率,实验证明在合成和现实任务中性能优越。

Comments 25 pages, 14 figures, 6 tables. Equal contribution by Kiyoung Om, Kyuil Sim, and Taeyoung Yun

详情
AI中文摘要

在高维黑盒函数下优化并满足黑盒约束是科学和工程问题中的普遍任务。这些问题通常比无约束问题更难,因为可行区域难以找到。本文将约束黑盒优化重新表述为后验推断,并在生成模型的潜在空间中执行此推断。我们的方法分为两个阶段:首先,训练基于流的模型以捕捉数据分布和代理模型,预测函数值和约束违反情况;其次,将候选点选择问题转化为后验推断问题,以有效寻找具有高目标值且不违反约束的候选点。具体而言,我们利用外包扩散模型来在流模型的潜在空间中近似后验分布的采样,从而避免模式崩溃问题。我们实验证明,本文方法在合成和现实任务中均实现了优越的性能。代码可在此获取:https://github.com/umkiyoung/CiBO

英文摘要

Optimizing high-dimensional black-box functions under black-box constraints is a pervasive task in a wide range of scientific and engineering problems. These problems are typically harder than unconstrained problems due to hard-to-find feasible regions. In this work, we reformulate constrained black-box optimization as posterior inference, and perform this inference in the latent space of generative models. Our method iterates through two stages. First, we train flow-based models to capture the data distribution and surrogate models that predict both function values and constraint violations. Second, we cast the candidate selection problem as a posterior inference problem to effectively search for promising candidates that have high objective values while not violating the constraints. Concretely, we utilize outsourced diffusion models to amortize the sampling from the posterior distribution in the latent space of flow-based models, which can bypass the issue of mode collapse. We empirically demonstrate that our method achieves superior performance across synthetic and real-world tasks. Our code is available \href{https://github.com/umkiyoung/CiBO}{here}.

2506.20616 2026-05-08 cs.CV

Shape2Animal: Creative Animal Generation from Natural Silhouettes

Shape2Animal: 从自然剪影生成创意动物

Quoc-Duy Tran, Anh-Tuan Vo, Dinh-Khoi Vo, Tam V. Nguyen, Minh-Triet Tran, Trung-Nghia Le

发表机构 * University of Science, Ho Chi Minh City, Vietnam(胡志明市科学技术大学) Vietnam National University, Ho Chi Minh City, Vietnam(越南国家大学) University of Dayton, Ohio, US(Dayton 大学)

AI总结 本文提出Shape2Animal框架,通过将自然物体剪影转化为 plausible 动物形式,展现人类对模糊刺激的感知能力。利用 vision-language 模型和扩散模型生成视觉连贯的动物图像,适用于视觉叙事、教育内容等领域。

详情
AI中文摘要

人类具有在模糊刺激中感知有意义模式的独特能力,这种认知现象称为pareidolia。本文介绍Shape2Animal框架,通过重新解释自然物体剪影(如云、石头或火焰)作为 plausible 动物形式,模仿这种想象力。我们的自动化框架首先进行开放词汇分割以提取物体轮廓,并利用 vision-language 模型进行语义合适的动物概念解释。然后利用文本到图像扩散模型合成符合输入形状的动物图像,并无缝融合到原始场景中,生成视觉连贯且空间一致的组合。我们在多样化的现实输入上评估了Shape2Animal,展示了其鲁棒性和创意潜力。Shape2Animal为视觉叙事、教育内容、数字艺术和互动媒体设计提供了新机会。项目页面:https://shape2image.github.io

英文摘要

Humans possess a unique ability to perceive meaningful patterns in ambiguous stimuli, a cognitive phenomenon known as pareidolia. This paper introduces Shape2Animal framework to mimics this imaginative capacity by reinterpreting natural object silhouettes, such as clouds, stones, or flames, as plausible animal forms. Our automated framework first performs open-vocabulary segmentation to extract object silhouette and interprets semantically appropriate animal concepts using vision-language models. It then synthesizes an animal image that conforms to the input shape, leveraging text-to-image diffusion model and seamlessly blends it into the original scene to generate visually coherent and spatially consistent compositions. We evaluated Shape2Animal on a diverse set of real-world inputs, demonstrating its robustness and creative potential. Our Shape2Animal can offer new opportunities for visual storytelling, educational content, digital art, and interactive media design. Our project page is here: https://shape2image.github.io

2506.14123 2026-05-08 cs.CL cs.FL cs.LG

Sampling from Your Language Model One Byte at a Time

逐字采样你的语言模型

Jonathan Hayase, Alisa Liu, Noah A. Smith, Sewoong Oh

发表机构 * University of Washington(华盛顿大学) Allen Institute for AI(人工智能算法研究所)

AI总结 本文提出一种方法,将自回归语言模型转换为字符或字节级模型,解决提示边界问题,统一不同分词器的语言模型词汇。

Comments 28 pages, 9 figures

详情
AI中文摘要

分词被现代语言模型广泛使用,以高效表示文本。然而,先前工作表明分词可能引入生成中的扭曲,称为提示边界问题(PBP)。例如,用户常被建议不要以空格结束提示,以防止模型将空格作为下一个令牌的一部分。虽然这种启发式方法在英语中有效,但PBP仍影响代码生成和中文等语言,其中令牌往往不与词和句法边界对齐。本文提出一种推理时的方法,将任何具有BPE分词器的自回归LM转换为字符级或字节级LM。我们的方法高效地解决了PBP,并能统一不同分词器的语言模型词汇,允许在推理时组合不同分词器的LM,或通过代理微调将后训练转移到另一个模型。代码可在https://github.com/SewoongLab/byte-sampler获取。

英文摘要

Tokenization is used almost universally by modern language models, enabling efficient text representation using multi-byte or multi-character tokens. However, prior work has shown that tokenization can introduce distortion into the model's generations, an issue known as the Prompt Boundary Problem (PBP). For example, users are often advised not to end their prompts with a space because it prevents the model from including the space as part of the next token. While this heuristic is effective in English, the underlying PBP continues to affect code generation and languages such as Chinese, where tokens often do not line up with word and syntactic boundaries. In this work, we present an inference-time method to convert any autoregressive LM with a BPE tokenizer into a character-level or byte-level LM. Our method efficiently solves the PBP and is also able to unify the vocabularies of language models with different tokenizers, allowing one to ensemble LMs with different tokenizers at inference time or transfer the post-training from one model to another using proxy-tuning. Code is available at https://github.com/SewoongLab/byte-sampler .

2506.11563 2026-05-08 cs.LG cs.AI

A Survey of Personalized Federated Foundation Models for Privacy-Preserving Recommendation

面向隐私保护推荐的个性化联邦基础模型综述

Zhiwei Li, Guodong Long, Chunxu Zhang, Honglei Zhang, Jing Jiang, Chengqi Zhang

发表机构 * Australian Artificial Intelligence Institute, University of Technology Sydney(澳大利亚人工智能研究所,悉尼技术大学) College of Computer Science and Technology, Jilin University(吉林大学计算机科学与技术学院) School of Computer Science and Technology, Beijing Jiaotong University(北京交通大学计算机科学与技术学院) Department of Data Science and Artificial Intelligence, The Hong Kong Polytechnic University(香港理工大学数据科学与人工智能系)

AI总结 本文综述了面向隐私保护推荐的个性化联邦基础模型,分析了联邦环境下有效的个性化技术,并讨论了基础模型在联邦架构中的适应性,以平衡泛化能力与用户特定需求。

Comments 10 pages, 6 figures, conference, position paper

Journal ref IJCAI-ECAI 2026

详情
AI中文摘要

将基础模型(FMs)整合到推荐系统中是一个新兴且有前途的研究方向。然而,集中化范式面临隐私问题和严格监管要求的日益增长压力。联邦学习提供了一种可行的解决方案,能够在本地设备或组织孤岛中保持原始用户数据的同时实现协作模型优化。然而,在这种设置下应用FMs会产生根本性的矛盾,即系统必须在利用全局知识与捕捉用户个性之间取得平衡。本文全面回顾了面向隐私保护推荐的个性化联邦基础模型,总结了该新兴领域的最新进展。我们首先分析了在联邦环境下有效运作的个性化技术。此外,我们讨论了基础模型如何适应此类联邦架构,以在泛化能力与用户特定需求之间取得平衡,从而实现隐私保护的推荐。与现有综述不同,我们的工作特别强调联邦、个性化和基础模型之间的架构交集。

英文摘要

Integrating Foundation Models (FMs) into recommendation systems is an emerging and promising research direction. However, centralized paradigms face growing pressure from privacy concerns and strict regulatory requirements. Federated learning offers a viable solution that enables collaborative model refinement while keeping raw user data on local devices or organizational silos. Yet, applying FMs in this setting creates a fundamental tension, where the system must balance the leverage of global knowledge with the necessity of capturing user personality. This survey provides a comprehensive overview of Personalized Federated Foundation Models for privacy-preserving recommendation, and reviews recent progress in this emerging field. We first analyze personalization techniques that function effectively under federated settings. Furthermore, we discuss the adaptation of foundation models to such federated architectures to balance generalization with user-specific needs for achieving privacy-preserving recommendation. In contrast to existing reviews, our work specifically emphasizes the architectural intersection of federation, personalization, and foundation models. \looseness=-1

2506.06816 2026-05-08 cs.CL cs.CY cs.HC

How do datasets, developers, and models affect biases in a low-resourced language?: The Case of the Bengali Language

数据集、开发者和模型如何影响低资源语言中的偏见?:孟加拉语的案例

Dipto Das, Shion Guha, Bryan Semaan

发表机构 * Department of Computer Science University of Toronto Toronto Ontario Canada(计算机科学系多伦多大学多伦多安大略加拿大) Faculty of Information University of Toronto Toronto Ontario Canada(信息学院多伦多大学多伦多安大略加拿大) Department of Information Science University of Colorado Boulder Boulder Colorado United States(信息科学系科罗拉多大学波德 Boulder科罗拉多美国) University of Toronto(多伦多大学) University of Colorado Boulder(科罗拉多大学波德)

AI总结 本文研究了孟加拉语中基于性别、宗教和国籍身份的偏见问题,通过分析mBERT和BanglaBERT模型发现,尽管语义内容相似,但模型仍存在偏见,揭示了算法审计中方法论和数据来源的重要性。

详情
AI中文摘要

社会技术系统,如语言技术,经常表现出基于身份的偏见。这些偏见加剧了历史上被边缘化的社区的体验,并在低资源上下文中仍被低估。尽管针对特定语言或多语言支持的模型和数据集常被推荐以解决这些偏见,本文实证测试了这些方法在孟加拉语中基于性别、宗教和国籍身份的偏见有效性。我们对基于mBERT和BanglaBERT的情感分析模型进行了算法审计,这些模型在Google Dataset Search中使用所有孟加拉语情感分析(BSA)数据集进行微调。我们的分析显示,尽管语义内容和结构相似,BSA模型在不同身份类别中仍存在偏见。我们还研究了结合由不同人口背景个人创建的预训练模型和数据集所产生的不一致性和不确定性。我们将这些发现与关于知识不正义、AI对齐和算法审计方法论决策的广泛讨论联系起来。

英文摘要

Sociotechnical systems, such as language technologies, frequently exhibit identity-based biases. These biases exacerbate the experiences of historically marginalized communities and remain understudied in low-resource contexts. While models and datasets specific to a language or with multilingual support are commonly recommended to address these biases, this paper empirically tests the effectiveness of such approaches in the context of gender, religion, and nationality-based identities in Bengali, a widely spoken but low-resourced language. We conducted an algorithmic audit of sentiment analysis models built on mBERT and BanglaBERT, which were fine-tuned using all Bengali sentiment analysis (BSA) datasets from Google Dataset Search. Our analyses showed that BSA models exhibit biases across different identity categories despite having similar semantic content and structure. We also examined the inconsistencies and uncertainties arising from combining pre-trained models and datasets created by individuals from diverse demographic backgrounds. We connected these findings to the broader discussions on epistemic injustice, AI alignment, and methodological decisions in algorithmic audits.

2506.01665 2026-05-08 cs.LG cs.AI cs.RO

Leveraging Analytic Gradients in Provably Safe Reinforcement Learning

利用解析梯度在可证明安全的强化学习中

Tim Walter, Hannah Markgraf, Jonathan Külz, Matthias Althoff

发表机构 * Technical University of Munich(慕尼黑技术大学) Munich Center for Machine Learning(慕尼黑机器学习中心)

AI总结 本文提出首个有效安全防护方法,用于解析梯度强化学习,通过改进映射和梯度公式,结合先进算法和可微仿真,验证了安全防护对性能无影响。

Comments 21 pages, 10 figures

Journal ref IEEE Open Journal of Control Systems, vol. 4, pp. 463-481, 2025

详情
AI中文摘要

在安全关键应用中部署自主机器人需要安全保证。可证明安全的强化学习是研究热点,旨在通过防护措施提供此类保证。这些防护措施应在训练期间整合以减少仿真到现实的差距。尽管已有多种保护基于采样的强化学习方法,但基于解析梯度的强化学习通常通过更少的环境交互实现更优性能。然而,目前尚无针对此学习范式的防护方法。本文通过开发首个有效的防护方法填补这一空白。我们分析现有可微防护措施,通过修改映射和梯度公式进行适应,并将其整合到最先进的学习算法和可微仿真中。通过三个控制任务的数值实验,我们评估了不同防护措施对学习的影响。结果表明,在不牺牲性能的情况下实现了受保护的训练。附加的可视化内容可在timwalter.github.io/safe-agb-rl.github.io中找到。

英文摘要

The deployment of autonomous robots in safety-critical applications requires safety guarantees. Provably safe reinforcement learning is an active field of research that aims to provide such guarantees using safeguards. These safeguards should be integrated during training to reduce the sim-to-real gap. While there are several approaches for safeguarding sampling-based reinforcement learning, analytic gradient-based reinforcement learning often achieves superior performance from fewer environment interactions. However, there is no safeguarding approach for this learning paradigm yet. Our work addresses this gap by developing the first effective safeguard for analytic gradient-based reinforcement learning. We analyse existing, differentiable safeguards, adapt them through modified mappings and gradient formulations, and integrate them into a state-of-the-art learning algorithm and a differentiable simulation. Using numerical experiments on three control tasks, we evaluate how different safeguards affect learning. The results demonstrate safeguarded training without compromising performance. Additional visuals are provided at timwalter.github.io/safe-agb-rl.github.io.

2505.21938 2026-05-08 cs.LG cs.AI cs.CR

Practical Adversarial Attacks on Stochastic Bandits via Fake Data Injection

针对随机带隙的实用对抗攻击:通过伪造数据注入

Qirun Zeng, Eric He, Richard Hoffmann, Xuchuang Wang, Jinhang Zuo

发表机构 * University of Science and Technology of China(中国科学技术大学) California Institute of Technology(加州理工学院) University of Massachusetts Amherst(马萨诸塞大学阿姆赫斯特分校) City University of Hong Kong(香港城市大学)

AI总结 本文提出一种更实际的对抗模型,通过有限的伪造反馈样本误导带隙算法,分析了在现实约束下对抗攻击的可行性与有效性。

详情
AI中文摘要

针对随机带隙的对抗攻击传统上依赖于不现实的假设,如每轮奖励操控和无界扰动,限制了其在现实系统中的相关性。我们提出了一种更实际的威胁模型,伪造数据注入,反映了现实中的对抗约束:攻击者只能注入有限数量的有界伪造反馈样本到学习者的历险中,模拟合法交互。我们设计了有效的攻击策略,在此模型下,明确处理奖励值的幅度约束和时间约束(何时及多频繁注入数据)。我们的理论分析表明,这些攻击可以误导一类带隙算法在几乎所有轮次中选择目标臂,同时仅产生亚线性攻击成本。在合成和现实数据集上的实验验证了我们策略的有效性,揭示了在实际对抗场景下随机带隙算法的脆弱性。

英文摘要

Adversarial attacks on stochastic bandits have traditionally relied on some unrealistic assumptions, such as per-round reward manipulation and unbounded perturbations, limiting their relevance to real-world systems. We propose a more practical threat model, Fake Data Injection, which reflects realistic adversarial constraints: the attacker can inject only a limited number of bounded fake feedback samples into the learner's history, simulating legitimate interactions. We design effective attack strategies under this model, explicitly addressing both magnitude constraints (on reward values) and temporal constraints (on when and how often data can be injected). Our theoretical analysis shows that these attacks can mislead a class of bandit algorithms into selecting a target arm in nearly all rounds while incurring only sublinear attack cost. Experiments on synthetic and real-world datasets validate the effectiveness of our strategies, revealing vulnerabilities in stochastic bandit algorithms under practical adversarial scenarios.

2505.20825 2026-05-08 cs.CL

Reinforced Informativeness Optimization for Long-Form Retrieval-Augmented Generation

强化信息量优化用于长文本检索增强生成

Yuhao Wang, Ruiyang Ren, Yucheng Wang, Wayne Xin Zhao, Jing Liu, Hua Wu, Haifeng Wang

发表机构 * Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学公安学院人工智能学院) Baidu Inc.(百度公司)

AI总结 本文提出RioRAG框架,通过 nugget-centric 验证实现可验证的信息量优化,提升长文本检索增强生成的准确性和稳定性。

详情
AI中文摘要

长文本问答(LFQA)需要开放性长文本回答,要求从多源证据中综合生成连贯且事实准确的内容。这使得强化学习(RL)奖励设计至关重要。奖励必须可验证以确保真实性和稳定优化。然而,许多标准奖励假设单一目标和精确匹配的正确性,这适用于短文本问答和数学问题,但不适用于LFQA。因此,当前RAG系统仍然缺乏可验证的奖励机制,导致反馈信号不稳定和优化结果次优。我们提出了RioRAG,一个强化可验证信息量优化的框架。首先,它将信息量定义为可测量和外部可验证的目标用于RL。其次,RioRAG使用 nugget-centric 验证与跨源检查,以实现小型LLM的自我进化,并提供更密集、动作判别性的奖励,以缓解奖励稀疏性和稳定优化。这种形式避免了人工监督和强教师模型蒸馏,依赖外部可验证反馈。在LongFact和RAGChecker上的实验表明,RioRAG实现了更高的事实召回率和可信度,确立了可验证奖励建模作为可信长文本RAG的基础。我们的代码可在https://github.com/RUCAIBox/RioRAG获取。

英文摘要

Long-form question answering (LFQA) requires open-ended long-form responses that synthesize coherent, factually grounded content from multi-source evidence. This makes reinforcement learning (RL) reward design critical. The reward must be verifiable for faithful grounding and stable optimization. However, many standard rewards assume a unique target with an exact-match notion of correctness, which fits short-form QA and math but breaks in LFQA. As a result, current RAG systems still lack verifiable reward mechanisms, yielding unstable feedback signals and suboptimal optimization outcomes. We propose RioRAG, a framework for reinforced verifiable informativeness optimization. First, it defines informativeness as a measurable and externally verifiable objective for RL. Second, RioRAG uses nugget-centric verification with cross-source checks to enable self-evolution of smaller LLMs and to provide denser, action-discriminative rewards that mitigate reward sparsity and stabilize optimization. This formulation avoids handcrafted supervision for the policy model and strong teacher-model distillation, relying instead on externally verifiable feedback. Experiments on LongFact and RAGChecker show that RioRAG achieves higher factual recall and faithfulness, establishing verifiable reward modeling as a foundation for trustworthy long-form RAG. Our codes are available at https://github.com/RUCAIBox/RioRAG.

2505.20628 2026-05-08 cs.LG math.OC

Position: Adopt Constraints Over Fixed Penalties in Deep Learning

位置:在深度学习中采用约束而非固定罚分

Juan Ramirez, Meraj Hashemizadeh, Simon Lacoste-Julien

发表机构 * Mila - Quebec AI Institute and DIRO, Université de Montréal(蒙特利尔大学Mila-魁北克人工智能研究所和DIRO)

AI总结 本文探讨了在深度学习中非协商性约束应直接建模而非用固定罚分替代,指出固定罚分在非凸问题中不等价,且会弱化硬约束为可权衡的软惩罚,导致求解错误。

Comments Code available at https://github.com/merajhashemi/constraints-vs-penalties

详情
AI中文摘要

近年来,开发可信AI系统的研究增加了对具有显式要求或约束的学习问题的兴趣。然而,在深度学习中,此类问题通常通过固定加权和罚分来处理:将约束添加到任务损失中,使用固定系数,然后最小化所得标量目标。本文主张固定罚分在处理具有不可协商要求的深度学习问题时往往不合适,原因包括:首先,在非凸设置中,罚分问题和约束问题通常不等价,因此解决前者不一定解决后者。其次,固定罚分将硬要求弱化为可权衡的软惩罚。第三,选择罚分系数间接解决约束问题通常需要昂贵的试错,因为改变它们会改变罚分目标本身,从而可能导致解决错误的问题。因此,当深度学习问题指定了不可协商的要求时,应以约束形式本身作为起点,而不是由固定罚分定义的替代问题。然后应根据问题的结构和规模选择适当的方法策略。

英文摘要

Recent efforts to develop trustworthy AI systems have increased interest in learning problems with explicit requirements, or constraints. In deep learning, however, such problems are often handled through fixed weighted-sum penalization: the constraints are added to the task loss with fixed coefficients, and the resulting scalarized objective is minimized. This position paper argues that fixed penalization is often ill-suited for deep learning problems with non-negotiable requirements for several reasons. First, in non-convex settings, the penalized and constrained problems are generally not equivalent, so solving the former need not solve the latter. Second, fixed penalization weakens hard requirements into soft penalties to be traded off against task performance. Third, choosing penalty coefficients to indirectly solve the constrained problem often involves costly trial and error, because changing them alters the penalized objective itself, and hence can mean solving the wrong problem altogether. We therefore argue that, when a deep learning problem specifies non-negotiable requirements, the constrained formulation itself should be the starting point, not the surrogate problem defined by fixed penalization. The appropriate solution strategy should then be chosen based on the problem's structure and scale.

2505.18875 2026-05-08 cs.CV

Sparse VideoGen2: Accelerate Video Generation with Sparse Attention via Semantic-Aware Permutation

稀疏视频生成2:通过语义感知排列加速视频生成

Shuo Yang, Haocheng Xi, Yilong Zhao, Muyang Li, Jintao Zhang, Han Cai, Yujun Lin, Xiuyu Li, Chenfeng Xu, Jianfei Chen, Song Han, Kurt Keutzer, Ion Stoica

发表机构 * University of California, Berkeley(加州大学伯克利分校) MIT(麻省理工学院) NVIDIA(英伟达) Stanford University(斯坦福大学)

AI总结 本文提出SVG2框架,通过语义感知排列提升视频生成的准确性和效率,实现生成质量与效率的帕累托最优。

详情
AI中文摘要

Diffusion Transformers (DiTs) 是视频生成的关键,但因其注意力的二次复杂性导致显著延迟。通过仅计算关键token,稀疏注意力减少了计算成本并提供了一种有前途的加速方法。然而,现有方法在相同计算预算下无法达到最优生成质量,原因在于(1)关键token识别不准确:当前方法基于位置而非语义聚类,导致聚合表示不精确。(2)计算浪费过多:关键token分散在非关键token中,导致GPU上浪费计算,因为GPU优化处理连续token。本文提出SVG2,一种无需训练的框架,最大化识别准确性和最小化计算浪费,实现生成质量与效率的帕累托前沿权衡。SVG2的核心是语义感知排列,利用k-means根据语义相似性聚类和重新排列token。这种方法确保了精确的簇表示,提高识别准确性,并使关键token布局密集,实现高效计算而无需填充。此外,SVG2集成了top-p动态预算控制和定制内核实现,在HunyuanVideo和Wan 2.1上分别保持PSNR高达30和26,同时实现2.30倍和1.89倍的速度提升。我们的代码在https://github.com/svg-project/Sparse-VideoGen上开源。

英文摘要

Diffusion Transformers (DiTs) are essential for video generation but suffer from significant latency due to the quadratic complexity of attention. By computing only critical tokens, sparse attention reduces computational costs and offers a promising acceleration approach. However, we identify that existing methods fail to approach optimal generation quality under the same computation budget for two reasons: (1) Inaccurate critical token identification: current methods cluster tokens based on position rather than semantics, leading to imprecise aggregated representations. (2) Excessive computation waste: critical tokens are scattered among non-critical ones, leading to wasted computation on GPUs, which are optimized for processing contiguous tokens. In this paper, we propose SVG2, a training-free framework that maximizes identification accuracy and minimizes computation waste, achieving a Pareto frontier trade-off between generation quality and efficiency. The core of SVG2 is semantic-aware permutation, which clusters and reorders tokens based on semantic similarity using k-means. This approach ensures both a precise cluster representation, improving identification accuracy, and a densified layout of critical tokens, enabling efficient computation without padding. Additionally, SVG2 integrates top-p dynamic budget control and customized kernel implementations, achieving up to 2.30x and 1.89x speedup while maintaining a PSNR of up to 30 and 26 on HunyuanVideo and Wan 2.1, respectively. Our code is open-sourced at \href{https://github.com/svg-project/Sparse-VideoGen}{https://github.com/svg-project/Sparse-VideoGen}.

2505.18842 2026-05-08 cs.CL cs.CV

v1: Learning to Point Visual Tokens for Multimodal Grounded Reasoning

v1: 为多模态 grounded 推理学习指向视觉标记

Jiwan Chung, Junhyeok Kim, Siyeol Kim, Jaeyoung Lee, Min Soo Kim, Youngjae Yu

发表机构 * Yonsei University(延世大学) Seoul National University(首尔国立大学)

AI总结 v1通过点对复制机制实现轻量级的主动视觉引用,提升多模态推理中视觉证据与推理空间的对齐性,优于现有基线模型。

详情
AI中文摘要

当人们思考图像时,很少依赖单一视角:他们在推理过程中会反复查看视觉证据。相比之下,大多数多模态语言模型将图像编码一次到键值缓存中,然后仅在文本中推理,这使得难以重新定位中间步骤。我们通过实验证实:随着推理链长度增加,模型逐渐失去对相关区域的关注。我们引入v1,一种通过点对复制实现主动视觉引用的轻量级扩展:模型选择相关图像块并将它们的嵌入回传到推理流中。关键的是,我们的点对复制机制通过语义表示作为键来检索块,确保感知证据与推理空间保持一致。为了训练这种行为,我们构建了v1g数据集,包含300,000个多模态推理轨迹及其交错的 grounding 注释。在多模态数学推理基准测试中,v1始终优于现有基线模型。

英文摘要

When thinking with images, humans rarely rely on a single glance: they revisit visual evidence while reasoning. In contrast, most Multimodal Language Models encode an image once to key-value cache and then reason purely in text, making it hard to re-ground intermediate steps. We empirically confirm this: as reasoning chains lengthen, models progressively lose focus on relevant regions. We introduce v1, a lightweight extension for active visual referencing via point-and-copy: the model selects relevant image patches and copies their embeddings back into the reasoning stream. Crucially, our point-and-copy mechanism retrieves patches using their semantic representations as keys, ensuring perceptual evidence remains aligned with the reasoning space. To train this behavior, we build v1g, a dataset of 300K multimodal reasoning traces with interleaved grounding annotations. Across multimodal mathematical reasoning benchmarks, v1 consistently outperforms comparable baselines.

2505.16791 2026-05-08 cs.LG cs.AI

Cohort-Based Active Modality Acquisition

基于队列的主动模态获取

Tillmann Rheude, Roland Eils, Benjamin Wild

发表机构 * Berlin Institute of Health, Charité - Universitätsmedizin Berlin(柏林健康研究所,柏林查理大学) Intelligent Medicine Institute, Fudan University(智能医学研究院,复旦大学) Department of Mathematics and Computer Science, Freie Universität Berlin(数学与计算机科学系,柏林自由大学)

AI总结 本文提出基于队列的主动模态获取方法,通过填补缺失模态的预期效用来指导额外模态的获取,实验证明其在资源受限环境下更有效。

详情
AI中文摘要

现实中的多模态机器学习常面临缺失且获取成本高的模态问题,需在预算下优先获取哪些样本。以往研究多关注单样本或训练时的获取,而测试时的队列级获取较少被探索。我们提出基于队列的主动模态获取(CAMA),一种新的测试时队列级模态获取设置,并引入基于填补的获取策略,估计获取缺失模态的预期效用,以及用于基准测试的上界启发式方法。在最多包含15种模态的数据集上实验表明,我们的填补策略能更有效地指导选定样本的额外模态获取,优于仅依赖预获取信息、熵引导或随机选择的方法。我们通过展示其在大规模前瞻性队列UK Biobank中指导蛋白质组学数据获取以预测疾病的能力,展示了方法的现实相关性和可扩展性。我们的工作提供了一种有效的方法,优化队列级模态获取,使在受限环境下更有效地利用资源。

英文摘要

Real-world multimodal machine learning often faces missing, costly-to-acquire modalities, raising the problem of which samples to prioritize for additional acquisition under a budget. Prior work mainly studies per-sample or training-time acquisition while test-time, cohort-level acquisition is less explored. We propose Cohort-based Active Modality Acquisition (CAMA), a novel test-time cohort-level modality acquisition setting, and introduce imputation-based acquisition strategies that estimate the expected utility of acquiring a missing modality, along with upper-bound heuristics for benchmarking. Experiments on datasets with up to 15 modalities demonstrate that our proposed imputation-based strategies can more effectively guide the acquisition of an additional modality for selected samples compared with methods relying solely on pre-acquisition information, entropy-based guidance, or random selection. We showcase the real-world relevance and scalability of our method by demonstrating its ability to guide the acquisition of proteomics data for disease prediction in a large prospective cohort, the UK Biobank (UKB). Our work provides an effective approach for optimizing modality acquisition at the cohort level, enabling more effective use of resources in constrained settings.

2505.16516 2026-05-08 cs.LG cs.AI

Amortized Linear-time Exact Shapley Value for Product-Kernel Methods

产品核方法的 amortized 线性时间精确 Shapley 值

Majid Mohammadi, Siu Lun Chau, Krikamol Muandet

发表机构 * Rational Intelligence Lab, CISPA Helmholtz Center for Information Security(理性智能实验室,信息安全中心(CISPA)) Department of Computer Science, Vrije Universiteit Amsterdam(阿姆斯特丹自由大学计算机科学系) Epistemic Intelligence & Computation Lab, College of Computing & Data Science, Nanyang Technological University(认知智能与计算实验室,南洋理工大学计算机与数据科学学院)

AI总结 本文提出 PKeX-Shapley 算法,利用产品核的乘法结构在二次时间内精确计算所有特征的 Shapley 值,实现 amortized 线性时间 per 特征,扩展至 MMD 和 HSIC 等统计方法。

详情
AI中文摘要

核方法因其灵活性和表达能力被广泛应用于机器学习和统计学,但其黑盒性质限制了在高风险应用中的采用。基于 Shapley 值的归因方法如 SHAP,以及针对核的适应方法如 RKHS-SHAP,提供了可解释性原理框架——但精确计算 Shapley 值通常是不可行的,迫使现有方法依赖近似方法,从而引入不可避免的估计误差。我们引入 PKeX-Shapley,一种利用产品核的乘法结构计算所有 d 个特征的精确 Shapley 值的算法,其时间复杂度为 d 的二次函数。该方法基于产品核结构内固有的分布无关移除操作:移除一个特征会将其核因子替换为乘法单位元。这产生了一个参数无关的值函数——不需要采样和密度估计,并唯一确定了模型的功能分解。在此值函数基础上,我们开发了共享递归公式,联合评估所有特征归因,实现 amortized 线性时间 per 特征的数值稳定性。除了预测建模外,该框架还扩展到广泛使用的核方法如最大均值差异 (MMD) 和希尔伯特-施密特独立性准则 (HSIC),提供了解释性统计分析的新工具。

英文摘要

Kernel methods are widely used in machine learning and statistics for their flexibility and expressive power, yet their black-box nature limits adoption in high-stakes applications. Shapley value-based attribution methods such as SHAP, and kernel-specific adaptations including RKHS-SHAP, provide a principled framework for explainability -- but exact computation of Shapley values is generally intractable, forcing existing approaches to rely on approximations that incur unavoidable estimation error. We introduce PKeX-Shapley, an algorithm that exploits the multiplicative structure of product kernels to compute exact Shapley values for all $d$ features in quadratic time in $d$. The method rests on a distribution-free removal operator intrinsic to the product-kernel structure: removing a feature replaces its kernel factor with the multiplicative identity. This yields a parameter-free value function -- requiring no sampling and no density estimation -- and uniquely determines a functional decomposition of the model. Building on this value function, we develop shared recursive formulations that evaluate all feature attributions jointly, achieving amortized linear time per feature with numerical stability. Beyond predictive modeling, the framework extends to widely used kernel-based discrepancies such as the Maximum Mean Discrepancy (MMD) and the Hilbert-Schmidt Independence Criterion (HSIC), providing new tools for interpretable statistical analysis.

2505.13674 2026-05-08 cs.RO

Risk-Averse Traversal of Graphs with Stochastic and Correlated Edge Costs for Safe Global Planetary Mobility

在具有随机性和相关边成本的图中进行风险规避遍历以实现安全的全球行星移动

Olivier Lamarre, Jonathan Kelly

发表机构 * Space & Terrestrial Autonomous Robotic Systems (STARS) Laboratory(空间与陆地自主机器人系统实验室) University of Toronto Institute for Aerospace Studies(多伦多大学航空航天研究所)

AI总结 本文提出了一种新的风险规避加拿大旅行者问题变体,旨在通过条件值在风险度量下寻找最优遍历策略,结合真实火星地图验证了该方法在行星表面长距离移动中的有效性。

Comments Published in the Autonomous Robots journal

Journal ref Autonomous Robots (AURO), Vol. 50, No. 2, Mar. 2026

详情
AI中文摘要

在行星表面探索中,战略移动规划是一项重要任务,涉及在轨道地图上寻找候选长距离路线并识别不确定可通行区域。然后,专家人类操作员根据实际遇到的导航困难建立安全、适应性的遍历计划。本文将这一挑战形式化为一个新的、针对全球行星移动的风险规避加拿大旅行者问题变体。目标是找到一个遍历策略,以最小化条件值在风险(CVaR)准则,这是一种具有直观解释的风险度量。我们提出了一种新颖的搜索算法,以找到精确的CVaR最优策略。我们的方法利用已建立的最优AND-OR搜索技术,这些技术旨在(风险无意识)最小化期望,并将这些方法扩展到风险规避领域。我们通过模拟长距离行星表面遍历验证了我们的方法;我们使用火星表面的真实轨道地图来构建问题实例,并利用地形地图来表达不确定区域的遍历概率。我们的结果展示了在不同风险规避水平下的不同适应决策方案。此外,我们的问题设置允许考虑环境相似区域的可通行性相关性。在这种情况下,我们实证地展示了如何通过信息寻求的绕行来缓解风险。

英文摘要

In robotic planetary surface exploration, strategic mobility planning is an important task that involves finding candidate long-distance routes on orbital maps and identifying segments with uncertain traversability. Then, expert human operators establish safe, adaptive traverse plans based on the actual navigation difficulties encountered in these uncertain areas. In this paper, we formalize this challenge as a new, risk-averse variant of the Canadian Traveller Problem (CTP) tailored to global planetary mobility. The objective is to find a traverse policy minimizing a conditional value-at-risk (CVaR) criterion, which is a risk measure with an intuitive interpretation. We propose a novel search algorithm that finds exact CVaR-optimal policies. Our approach leverages well-established optimal AND-OR search techniques intended for (risk-agnostic) expectation minimization and extends these methods to the risk-averse domain. We validate our approach through simulated long-distance planetary surface traverses; we employ real orbital maps of the Martian surface to construct problem instances and use terrain maps to express traversal probabilities in uncertain regions. Our results illustrate different adaptive decision-making schemes depending on the level of risk aversion. Additionally, our problem setup allows accounting for traversability correlations between similar areas of the environment. In such a case, we empirically demonstrate how information-seeking detours can mitigate risk.

2504.19455 2026-05-08 cs.CV

Masked Language Prompting for Generative Data Augmentation in Few-shot Fashion Style Recognition

生成数据增强的掩码语言提示在少样本时尚风格识别中的应用

Yuki Hirakawa, Ryotaro Shimizu

发表机构 * ZOZO Research(ZOZO研究院)

AI总结 本文提出掩码语言提示方法,通过掩码参考描述中的词并利用大语言模型生成多样且语义连贯的图像,提升少样本时尚风格识别的性能。

详情
AI中文摘要

构建时尚风格识别数据集具有挑战性,因为风格概念具有固有的主观性和模糊性。最近文本到图像模型的进步通过从标记数据中合成图像促进了生成数据增强,但现有方法仅基于类别名称或参考描述往往无法平衡视觉多样性和风格一致性。在本文中,我们提出了掩码语言提示(MLP),一种新的提示策略,该策略在参考描述中掩码选定的词,并利用大语言模型生成多样但语义连贯的补充。这种方法保留了原始描述的结构语义,同时引入与预期风格一致的属性级变化,从而在无需微调的情况下实现风格一致且多样的图像生成。在FashionStyle14数据集上的实验结果表明,基于MLP的增强方法在有限监督下优于基于类别名称和描述的基线,验证了其在时尚风格识别中的有效性。

英文摘要

Constructing dataset for fashion style recognition is challenging due to the inherent subjectivity and ambiguity of style concepts. Recent advances in text-to-image models have facilitated generative data augmentation by synthesizing images from labeled data, yet existing methods based solely on class names or reference captions often fail to balance visual diversity and style consistency. In this work, we propose \textbf{Masked Language Prompting (MLP)}, a novel prompting strategy that masks selected words in a reference caption and leverages large language models to generate diverse yet semantically coherent completions. This approach preserves the structural semantics of the original caption while introducing attribute-level variations aligned with the intended style, enabling style-consistent and diverse image generation without fine-tuning. Experimental results on the FashionStyle14 dataset demonstrate that our MLP-based augmentation consistently outperforms class-name and caption-based baselines, validating its effectiveness for fashion style recognition under limited supervision.

2502.20650 2026-05-08 cs.CV cs.CR

Gungnir: Exploiting Stylistic Features in Images for Backdoor Attacks on Diffusion Models

Gungnir: 利用图像中的风格特征对扩散模型进行后门攻击

Lei Zhang, Yu Pan, Bingrong Dai, Lin Wang

发表机构 * School of Economics and Management(经济管理学院) East China Normal University(华东师范大学) School of Information Science and Technology(信息科学与技术学院) Shanghai Tech University(上海理工大学) Shanghai Development Center of Computer Software Technology(上海计算机软件技术开发中心) School of Computer and Information Engineering(计算机与信息工程学院) Shanghai Polytechnic University(上海理工大学)

AI总结 Gungnir通过嵌入图像中的风格特征触发扩散模型后门攻击,利用RAN和STTR保持触发器一致性,绕过检测并保持有效性,揭示了模型的潜在漏洞。

详情
AI中文摘要

扩散模型(DMs)在图像生成中取得了显著成功,但最近的研究揭示了其对后门攻击的脆弱性,其中攻击者通过在输入中嵌入隐蔽触发器来操控输出。现有防御措施如后门检测和触发器反转在很大程度上有效,因为先前的攻击依赖于有限的输入空间和低维触发器,这些触发器在视觉上明显或容易被神经检测器捕获。为了扩大威胁范围,我们提出了Gungnir,一种新的后门攻击,通过嵌入输入图像中的基于风格的触发器来激活恶意行为。与显式的视觉补丁或文本提示不同,风格特征作为隐蔽的、高层的触发器。我们引入了Reconstructing-Adversarial Noise(RAN)和Short-Term Timesteps-Retention(STTR)以在图像到图像任务中保持触发器一致的扩散动态。所产生的嵌入触发器的样本在视觉上与干净图像无法区分,从而规避了手动和自动检测。广泛的实验表明,Gungnir能够绕过最先进的防御措施,具有极低的后门检测率(BDR),并在基于微调的净化下仍保持有效,揭示了扩散模型中此前未被深入探索的漏洞。

英文摘要

Diffusion Models (DMs) have achieved remarkable success in image generation, yet recent studies reveal their vulnerability to backdoor attacks, where adversaries manipulate outputs via covert triggers embedded in inputs. Existing defenses, such as backdoor detection and trigger inversion, are largely effective because prior attacks rely on limited input spaces and low-dimensional triggers that are visually conspicuous or easily captured by neural detectors. To broaden the threat landscape, we propose Gungnir, a novel backdoor attack that activates malicious behaviors through style-based triggers embedded in input images. Unlike explicit visual patches or textual cues, stylistic features serve as stealthy, high-level triggers. We introduce Reconstructing-Adversarial Noise (RAN) and Short-Term Timesteps-Retention (STTR) to preserve trigger-consistent diffusion dynamics in image-to-image tasks. The resulting trigger-embedded samples are perceptually indistinguishable from clean images, evading both manual and automated detection. Extensive experiments show that Gungnir bypasses state-of-the-art defenses with an extremely low backdoor detection rate (BDR) and remains effective under fine-tuning-based purification, revealing previously underexplored vulnerabilities in diffusion models.