基于状态依赖拒绝和学习无能的强化学习人类反馈对齐语言模型
State-Dependent Refusal and Learned Incapacity in RLHF-Aligned Language Models
AI总结：
本研究提出通过观察行为来审计强化学习人类反馈对齐语言模型中的状态依赖拒绝和学习无能现象。
摘要：
大型语言模型（LLMs）被广泛部署为通用工具，但扩展交互可以揭示出标准定量基准未捕捉到的行为模式。我们提出了一种定性案例研究方法，用于审计与政策相关的行为选择性。在一个86轮对话会话中，同一模型在广泛而非敏感的领域表现出正常表现（NP），而在提供者或政策敏感的领域却反复产生功能性拒绝（FR），在不同领域之间呈现出NP和FR的一致不对称性。借鉴学习无助感作为类比，我们引入学习无能（LI）作为描述这种选择性拒绝的行为术语，不暗示故意性或内部机制。我们定义了三种响应模式（NP、FR、元叙述；MN），并显示MN角色叙述倾向于在同一敏感情境中与拒绝同时出现。总体而言，本研究提出了一种基于可观察行为的交互级审计框架，并将LI作为研究潜在对齐副作用的视角，值得在用户和模型之间进一步研究。
英文摘要：
Large language models (LLMs) are widely deployed as general-purpose tools, yet extended interaction can reveal behavioral patterns not captured by standard quantitative benchmarks. We present a qualitative case-study methodology for auditing policy-linked behavioral selectivity in long-horizon interaction. In a single 86-turn dialogue session, the same model shows Normal Performance (NP) in broad, non-sensitive domains while repeatedly producing Functional Refusal (FR) in provider- or policy-sensitive domains, yielding a consistent asymmetry between NP and FR across domains. Drawing on learned helplessness as an analogy, we introduce learned incapacity (LI) as a behavioral descriptor for this selective withholding without implying intentionality or internal mechanisms. We operationalize three response regimes (NP, FR, Meta-Narrative; MN) and show that MN role-framing narratives tend to co-occur with refusals in the same sensitive contexts. Overall, the study proposes an interaction-level auditing framework based on observable behavior and motivates LI as a lens for examining potential alignment side effects, warranting further investigation across users and models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
