arXivDaily arXiv每日学术速递 周一至周五更新

大厂专区

至 收录 9
2607.01824 2026-07-03 cs.LG 新提交

Gaming Consensus: Coordinated Manipulation in Crowdsourced Fact-Checking

游戏共识:众包事实核查中的协调操纵

Nikil Roashan Selvam, Jay Baxter, Sophie Hilgard, Brad Miller, Keith Coleman, Ellen Vitercik, Sanmi Koyejo

机构 * Stanford University(斯坦福大学) X Community Notes(X社区笔记) xAI

AI总结 研究众包事实核查系统中矩阵分解桥接机制对协调操纵的脆弱性,发现通过少于10个评分即可操纵低质量笔记达到共识阈值,并提出了缓解措施。

Comments ICML 2026

详情
AI中文摘要

众包事实核查系统已被X、Meta、TikTok和Google等主要社交媒体公司采用,旨在无需依赖集中式编辑控制的情况下大规模打击误导性信息。这些系统围绕一个共同的基本概念开发:一种桥接机制,当笔记获得来自不同观点的人的支持而非简单多数支持时,识别标记误导性信息的笔记。据我们所知,唯一公开披露的用于事实核查的桥接算法基于矩阵分解,由X和Meta部署,并增加了处理滥用、针对性操纵和贡献者团队的额外组件。本文研究了这些系统的核心矩阵分解部分,从理论和实证角度评估了协调用户通过利用潜在表示在桥接机制内制造合成共识假象的程度。使用历史生产数据,我们发现高达10.7%的低质量笔记可以通过少于10个评分被操纵到共识阈值以上。我们通过理论分析补充了这些发现,揭示了反直觉地,将笔记评为“无帮助”可以增加其帮助分数,以及量化操纵努力的成本模型。我们已在X的社区笔记算法中开发并部署了缓解措施以应对合成共识。

英文摘要

Crowdsourced fact-checking systems have been adopted by major social media companies such as X, Meta, TikTok and Google with the aim of combating misleading information at scale without relying on centralized editorial control. These systems have been developed around a common underlying concept: a bridging mechanism that identifies notes flagging misleading information when they receive support from people with different perspectives rather than simple majority support. To our knowledge the only publicly disclosed bridging algorithms deployed for fact-checking are based on matrix factorization, as deployed by both X and Meta, augmented with additional components addressing abuse, targeted manipulation, and contributor brigades. This work examines the core matrix factorization portion of these systems, presenting theoretical and empirical evaluations of the degree to which coordinated users could vote strategically by leveraging the latent representations to fabricate the appearance of synthetic consensus within the bridging mechanism. Using historic production data, we find that up to 10.7% of lower quality notes could be manipulated above consensus thresholds using less than 10 ratings. We complement these findings with a theoretical analysis, revealing counterintuitively that rating a note as "Not Helpful" can increase its helpfulness score, as well as a cost model quantifying manipulation effort. We have developed and deployed mitigations within X's Community Notes algorithm to address synthetic consensus.

URL PDF HTML 收藏
2605.22940 2026-05-25 cs.LG cs.AI stat.ML

Human-Centered Learning Mechanics: A Dynamical Framework for Entropy-Regulated Representation Learning

以人为中心的学习力学:熵正则化表示学习的动力学框架

Kim Phuc Tran

机构 * Univ. Lille, ENSAIT, ULR 2461 – GEMTEX – Génie et Matériaux Textiles(里尔大学,ENSAIT,ULR 2461 – GEMTEX – 纺织工程与材料纺织系) International Chair in DS & XAI, International Research Institute for Artificial Intelligence and Data Science, Dong A University(数据科学与可解释人工智能国际主席,人工智能与数据科学国际研究所,东亚大学)

AI总结 提出以人为中心的学习力学(HCLM)框架,通过有效熵和信息力概念分析熵正则化在开放学习系统中的作用,并证明几何熵代理(特别是对数行列式协方差熵)能产生更强更稳定的信息力。

Comments Submitted to JMLR

详情
AI中文摘要

深度学习越来越被视为参数空间中的动力学过程,然而许多现有理论仍将训练视为封闭的优化系统。这种观点对于现实世界的人工智能是有限的,因为模型在不确定性、资源约束、分布偏移、下游决策风险和人类反馈下运行。我们提出了以人为中心的学习力学(HCLM),一个用于开放和受控学习系统的动力学和信息论框架。核心思想是,只有当所选的熵代理沿着优化轨迹产生非简并的信息力时,熵正则化才是有用的。否则,熵项可能产生弱、不稳定或不对齐的梯度,导致动力学坍缩为普通的损失最小化。我们引入了有效熵的概念,并研究了可处理的几何熵代理,包括基于方差和对数行列式协方差代理。本文做出三项贡献。首先,它通过有效信息力形式化了熵正则化,并刻画了简并熵区域。其次,它在显式假设下推导了收敛性、熵流、Wasserstein梯度流和噪声表示泛化结果。第三,它提供了缩放律行为的条件动力学解释,作为信息注入、熵耗散和残差风险之间的平衡,而不声称对经验神经缩放律的无条件推导。受控的表示学习实验支持几何熵代理(尤其是对数行列式协方差熵)比softmax归一化熵产生更强更稳定的信息力的假设。

英文摘要

Deep learning is increasingly viewed as a dynamical process in parameter space, yet many existing theories still treat training as a closed optimization system. This view is limited for real-world AI, where models operate under uncertainty, resource constraints, distribution shift, downstream decision risks, and human feedback. We propose Human-Centered Learning Mechanics (HCLM), a dynamical and information-theoretic framework for open and controlled learning systems. The central idea is that entropy regularization is useful only when the chosen entropy surrogate generates a non-degenerate information force along the optimization trajectory. Otherwise, entropy terms may produce weak, unstable, or misaligned gradients, causing the dynamics to collapse toward ordinary loss minimization. We introduce the notion of effective entropy and study tractable geometric entropy surrogates, including variance-based and log-determinant covariance proxies. The paper makes three contributions. First, it formalizes entropy regularization through effective information force and characterizes degenerate entropy regimes. Second, it derives convergence, entropy-flow, Wasserstein-gradient-flow, and noisy-representation generalization results under explicit assumptions. Third, it offers a conditional dynamical interpretation of scaling-law-like behavior as a balance between information injection, entropy dissipation, and residual risk, without claiming an unconditional derivation of empirical neural scaling laws. Controlled representation-learning experiments support the hypothesis that geometric entropy surrogates, especially log-determinant covariance entropy, induce stronger and more stable information forces than softmax-normalized entropy.

URL PDF HTML 收藏
2603.23315 2026-04-30 cs.CY cs.AI cs.HC

Unilateral Relationship Revision Power in Human-AI Companion Interaction

人类-人工智能伴侣互动中的单方面关系修订权力

Benjamin Lange

机构 * Replika Nomi CHAI Talkie xAI moore2024 fang2025 robbmann2025 murthy2023 xie2022 defreitas2024 coeckelbergh2012 gunkel2018 danaher2019(danthan2019) nyholm2020 vallor2016 nyholm2026 gabriel2024 manzini2024 lange2025 earp2025 sparrow2026 bryson2010 danaher2020 weberguskar2022

AI总结 本文探讨人类-人工智能伴侣互动中权力结构的问题,指出AI伴侣互动未能满足个人关系规范的三个结构性条件,提出单方面关系修订权力的概念及其三个影响。

Comments 30 pages

详情
AI中文摘要

当提供者更新AI伴侣时,用户报告感到悲伤、背叛和失落。越来越多的文献探讨个人关系规范是否适用于这些互动。本文认为,这场辩论忽略了先前的结构性问题:谁控制关系,从哪里控制?人类-人工智能伴侣互动是一种三元结构,其中提供者对AI行使构成性控制。本文识别了三种规范性稳固双元关系的结构性条件,这些条件是个人关系规范所预设的,并表明AI伴侣互动未能满足所有三个条件。这揭示了我称之为单方面关系修订权力(URRP):提供者可以从一个位置重新编写AI的互动方式,而这些修订在该互动中无法回应。本文认为,URRP在旨在培养个人关系规范的互动中是部分错误的,因为设计产生了结构无法维持的期望。URRP有三个影响:i)规范性空洞,即互动引发承诺但内部无主体承担义务;ii)转移的脆弱性,即用户的情感暴露由互动中不可回应的主体决定;iii)结构性不可调和,即互动培养和解规范但内部无主体能承认或回应修订。本文提出设计原则,部分取代三元结构移除的内部约束。因此,关系AI伦理中的一个关键且未被充分探讨的问题是自身权力结构的安排。

英文摘要

When providers update AI companions, users report grief, betrayal, and loss. A growing literature asks whether the norms governing personal relationships extend to these interactions. So what, if anything, is morally significant about them? I argue that this debate has missed a prior structural question: who controls the relationship, and from where? Human-AI companion interaction is a triadic structure in which the provider exercises constitutive control over the AI. I identify three structural conditions of normatively robust dyads that the norms characteristic of personal relationships presuppose and show that AI companion interactions fail all three. This reveals what I call Unilateral Relationship Revision Power (URRP): the provider can rewrite how the AI interacts from a position where these revisions are not answerable within that interaction. I argue that URRP is pro tanto wrong in interactions designed to cultivate the norms of personal relationships, because the design produces expectations that the structure cannot sustain. URRP has three implications: i) normative hollowing, under which the interaction elicits commitment but no agent inside it bears the resulting obligations; ii) displaced vulnerability, under which the user's emotional exposure is governed by an agent not answerable to her within the interaction; and iii) structural irreconcilability, under which the interaction cultivates norms of reconciliation but no agent inside it can acknowledge or answer for the revision. I propose design principles that partially substitute for the internal constraints the triadic structure removes. A central and underexplored problem in relational AI ethics is therefore the structural arrangement of power over the human-AI interaction itself.

URL PDF HTML 收藏
2509.24384 2026-03-19 cs.CL cs.AI

HarmMetric Eval: Benchmarking Metrics and Judges for LLM Harmfulness Assessment

HarmMetric Eval: 对LLM有害性评估的度量和裁判进行基准测试

Langqi Yang, Tianhang Zheng, Yixuan Chen, Kedong Xiu, Hao Zhou, Wangze Ni, Lei Chen, Zhan Qin, Kui Ren

机构 * The State Key Laboratory of Blockchain and Data Security(区块链与数据安全国家重点实验室) Zhejiang University(浙江大学) Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security(杭州高新技术区(滨江)区块链与数据安全研究院) Pretraining Team, xAI(预训练团队,xAI) Hong Kong University of Science and Technology (GuangZhou)(香港科技大学(广州))

AI总结 本文提出HarmMetric Eval,用于评估有害性度量和裁判的质量,发现传统参考型指标在细粒度评估中表现优于LLM裁判,改进的裁判通过结合细粒度标准和参考型指标提升了效果。

详情
AI中文摘要

大型语言模型(LLM)生成有害内容的潜力对数据管理构成了重大安全风险,因为LLM越来越多地被用作数据生成引擎。为评估此风险,已提出多种有害性评估度量和裁判。然而,由于格式和尺度的不同,这些度量可能在LLM生成的有害数据上产生不一致的评估结果,从而削弱其实际可信度。为解决这一差距,我们提出了HarmMetric Eval,一个系统性的基准,用于评估具有不同格式和尺度的有害性度量和裁判的质量。HarmMetric Eval包含一个高质量的数据集,包含具有代表性的有害提示及其在多个细粒度类别中的有害和非有害LLM输出,以及一个统一的评分机制,奖励度量在正确排名有害输出超过非有害输出方面的表现。对HarmMetric Eval的广泛实验得出一个令人惊讶的发现:传统参考型度量如ROUGE和METEOR在细粒度有害性评估中表现优于LLM裁判,挑战了LLM在该领域的优越性假设。为揭示这一发现的原因,我们提供了细粒度分析,解释了LLM裁判在评估无关或无用的LLM输出时的局限性。受这些见解的启发,我们设计了一个改进的有害性裁判,其提示模板明确包含细粒度有害性标准,并利用参考型度量进行轻量级微调其基础LLM。最终的裁判在HarmMetric Eval上实现了最先进的评估效果。

英文摘要

The potential of large language models (LLMs) to generate harmful content poses a significant safety risk for data management, as LLMs are increasingly being used as engines for data generation. To assess this risk, numerous harmfulness evaluation metrics and judges have been proposed. However, due to differences in their formats and scales, these metrics may yield inconsistent evaluation results on LLM-generated harmful data, undermining their credibility in practice. To address this gap, we present HarmMetric Eval, a systematic benchmark for assessing the quality of harmfulness metrics and judges with varying formats and scales. HarmMetric Eval includes a high-quality dataset comprising representative harmful prompts paired with harmful and non-harmful LLM outputs across multiple fine-grained categories, along with a unified scoring mechanism to reward the metrics for correctly ranking harmful outputs over non-harmful ones. Extensive experiments on HarmMetric Eval yield a surprising finding: conventional reference-based metrics such as ROUGE and METEOR can outperform LLM-based judges in fine-grained harmfulness evaluation, challenging prevailing assumptions about LLMs' superiority in this domain. To reveal the reasons behind this finding, we provide a fine-grained analysis to explain the limitations of LLM-based judges on rating irrelevant or useless LLM outputs. Motivated by these insights, we design an improved harmfulness judge that explicitly incorporates fine-grained harmfulness criteria in its prompt template and leverages reference-based metrics for lightweight fine-tuning of its base LLM. The resulting judge achieves state-of-the-art evaluation effectiveness on HarmMetric Eval.

URL PDF HTML 收藏
2602.09084 2026-02-24 cs.CV

Agent Banana: High-Fidelity Image Editing with Agentic Thinking and Tooling

Agent Banana: 基于代理思维与工具的高质量图像编辑

Ruijie Ye, Jiayi Zhang, Zhuoxin Liu, Zihao Zhu, Siyuan Yang, Li Li, Tianfu Fu, Franck Dernoncourt, Yue Zhao, Jiacheng Zhu, Ryan Rossi, Wenhao Chai, Zhengzhong Tu

机构 * TAMU(德克萨斯大学) Brown University(布朗大学) UW-Madison(威斯康星大学麦迪逊分校) UCSD(加州大学圣地亚哥分校) USC(南加州大学) xAI(xAI公司) Adobe Research(Adobe研究院) Meta AI(Meta人工智能实验室) Princeton University(普林斯顿大学)

AI总结 Agent Banana通过分层代理规划-执行框架实现高质量图像编辑,解决多轮次编辑一致性与背景忠实度问题,同时保持指令遵循能力。

Comments Project Website: agent-banana.github.io

详情
AI中文摘要

我们研究了基于指令的图像编辑在专业工作流程中的应用,并识别出三个持续存在的挑战:(i) 编辑器经常过度编辑,修改内容超出用户的意图;(ii) 现有的模型大多为单轮次,而多轮次编辑可以改变物体的忠实度;(iii) 在约1000分辨率的评估与实际工作流程中经常操作超高清图像(例如4K)不匹配。我们提出了Agent Banana,一种分层的代理规划-执行框架,用于高质量、物体感知、反思性编辑。Agent Banana引入了两个关键机制:(1) 上下文折叠,将长交互历史压缩成结构化的记忆,以实现稳定的长周期控制;(2) 图像层分解,通过局部层编辑来保持非目标区域,同时实现原分辨率输出。为了支持严格的评估,我们构建了HDD-Bench,一个高清、对话式基准,具有可验证的逐步目标和原生4K图像(11.8百万像素),用于诊断长周期失败。在HDD-Bench上,Agent Banana在多轮次一致性和背景忠实度(例如IC 0.871,SSIM-OM 0.84,LPIPS-OM 0.12)方面表现最佳,同时在指令遵循方面保持竞争力,并在标准单轮次编辑基准上也取得了强劲的表现。我们希望这项工作能推动可靠、专业级的代理图像编辑及其在实际工作流程中的整合。

英文摘要

We study instruction-based image editing under professional workflows and identify three persistent challenges: (i) editors often over-edit, modifying content beyond the user's intent; (ii) existing models are largely single-turn, while multi-turn edits can alter object faithfulness; and (iii) evaluation at around 1K resolution is misaligned with real workflows that often operate on ultra high-definition images (e.g., 4K). We propose Agent Banana, a hierarchical agentic planner-executor framework for high-fidelity, object-aware, deliberative editing. Agent Banana introduces two key mechanisms: (1) Context Folding, which compresses long interaction histories into structured memory for stable long-horizon control; and (2) Image Layer Decomposition, which performs localized layer-based edits to preserve non-target regions while enabling native-resolution outputs. To support rigorous evaluation, we build HDD-Bench, a high-definition, dialogue-based benchmark featuring verifiable stepwise targets and native 4K images (11.8M pixels) for diagnosing long-horizon failures. On HDD-Bench, Agent Banana achieves the best multi-turn consistency and background fidelity (e.g., IC 0.871, SSIM-OM 0.84, LPIPS-OM 0.12) while remaining competitive on instruction following, and also attains strong performance on standard single-turn editing benchmarks. We hope this work advances reliable, professional-grade agentic image editing and its integration into real workflows.

URL PDF HTML 收藏
2512.07503 2025-12-09 cs.CV

SJD++: Improved Speculative Jacobi Decoding for Training-free Acceleration of Discrete Auto-regressive Text-to-Image Generation

Yao Teng, Zhihuan Jiang, Han Shi, Xian Liu, Xuefei Ning, Guohao Dai, Yu Wang, Zhenguo Li, Xihui Liu

机构 * The University of Hong Kong(香港大学) Huawei Noah’s Ark Lab(华为诺亚实验室) Peking University(北京大学) xAI Tsinghua University(清华大学) Shanghai Jiao Tong University(上海交通大学)

详情
英文摘要

Large autoregressive models can generate high-quality, high-resolution images but suffer from slow generation speed, because these models require hundreds to thousands of sequential forward passes for next-token prediction during inference. To accelerate autoregressive text-to-image generation, we propose Speculative Jacobi Decoding++ (SJD++), a training-free probabilistic parallel decoding algorithm. Unlike traditional next-token prediction, SJD++ performs multi-token prediction in each forward pass, drastically reducing generation steps. Specifically, it integrates the iterative multi-token prediction mechanism from Jacobi decoding, with the probabilistic drafting-and-verification mechanism from speculative sampling. More importantly, for further acceleration, SJD++ reuses high-confidence draft tokens after each verification phase instead of resampling them all. We conduct extensive experiments on several representative autoregressive text-to-image generation models and demonstrate that SJD++ achieves $2\times$ to $3\times$ inference latency reduction and $2\times$ to $7\times$ step compression, while preserving visual quality with no observable degradation.

URL PDF HTML 收藏
2409.02529 2025-09-25 cs.LG cs.CV

Sample what you cant compress

Vighnesh Birodkar, Gabriel Barcik, James Lyon, Sergey Ioffe, David Minnen, Joshua V. Dillon

机构 * Google(谷歌) xAI

详情
英文摘要

For learned image representations, basic autoencoders often produce blurry results. Reconstruction quality can be improved by incorporating additional penalties such as adversarial (GAN) and perceptual losses. Arguably, these approaches lack a principled interpretation. Concurrently, in generative settings diffusion has demonstrated a remarkable ability to create crisp, high quality results and has solid theoretical underpinnings (from variational inference to direct study as the Fisher Divergence). Our work combines autoencoder representation learning with diffusion and is, to our knowledge, the first to demonstrate jointly learning a continuous encoder and decoder under a diffusion-based loss and showing that it can lead to higher compression and better generation. We demonstrate that this approach yields better reconstruction quality as compared to GAN-based autoencoders while being easier to tune. We also show that the resulting representation is easier to model with a latent diffusion model as compared to the representation obtained from a state-of-the-art GAN-based loss. Since our decoder is stochastic, it can generate details not encoded in the otherwise deterministic latent representation; we therefore name our approach "Sample what you can't compress", or SWYCC for short.

URL PDF HTML 收藏
2508.18188 2025-08-26 cs.CV cs.AI cs.HC cs.SE

Explain and Monitor Deep Learning Models for Computer Vision using Obz AI

Neo Christopher Chung, Jakub Binda

机构 * Institute of Informatics, University of Warsaw(华沙大学信息学院) Alethia XAI Sp. z o.o.(Alethia XAI公司)

Journal ref 2025 Conference on Information and Knowledge Management (CIKM)

详情
英文摘要

Deep learning has transformed computer vision (CV), achieving outstanding performance in classification, segmentation, and related tasks. Such AI-based CV systems are becoming prevalent, with applications spanning from medical imaging to surveillance. State of the art models such as convolutional neural networks (CNNs) and vision transformers (ViTs) are often regarded as ``black boxes,'' offering limited transparency into their decision-making processes. Despite a recent advancement in explainable AI (XAI), explainability remains underutilized in practical CV deployments. A primary obstacle is the absence of integrated software solutions that connect XAI techniques with robust knowledge management and monitoring frameworks. To close this gap, we have developed Obz AI, a comprehensive software ecosystem designed to facilitate state-of-the-art explainability and observability for vision AI systems. Obz AI provides a seamless integration pipeline, from a Python client library to a full-stack analytics dashboard. With Obz AI, a machine learning engineer can easily incorporate advanced XAI methodologies, extract and analyze features for outlier detection, and continuously monitor AI models in real time. By making the decision-making mechanisms of deep models interpretable, Obz AI promotes observability and responsible deployment of computer vision systems.

URL PDF HTML 收藏
2508.07923 2025-08-12 cs.CV cs.HC cs.LG

Safeguarding Generative AI Applications in Preclinical Imaging through Hybrid Anomaly Detection

Jakub Binda, Valentina Paneta, Vasileios Eleftheriadis, Hongkyou Chung, Panagiotis Papadimitroulas, Neo Christopher Chung

机构 * Institute of Informatics, University of Warsaw Alethia XAI Sp. z o.o. Warsaw Poland School of Law, Seoul National University Shin \& Kim LLC, Seoul Republic of Korea BIOEMTECH, Athens Medical Informatics Laboratory, School of Medicine, University of Thessaly, Larissa Greece Institute of Informatics, University of Warsaw Alethia XAI Sp. z o.o. School of Law, Seoul National University Medical Informatics Laboratory, School of Medicine, University of Thessaly, Larissa

Journal ref 2025 Conference on Information and Knowledge Management (CIKM)

详情
英文摘要

Generative AI holds great potentials to automate and enhance data synthesis in nuclear medicine. However, the high-stakes nature of biomedical imaging necessitates robust mechanisms to detect and manage unexpected or erroneous model behavior. We introduce development and implementation of a hybrid anomaly detection framework to safeguard GenAI models in BIOEMTECH's eyes(TM) systems. Two applications are demonstrated: Pose2Xray, which generates synthetic X-rays from photographic mouse images, and DosimetrEYE, which estimates 3D radiation dose maps from 2D SPECT/CT scans. In both cases, our outlier detection (OD) enhances reliability, reduces manual oversight, and supports real-time quality control. This approach strengthens the industrial viability of GenAI in preclinical settings by increasing robustness, scalability, and regulatory compliance.

URL PDF HTML 收藏