arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.14531cs.AI
收藏

情感成本函数用于人工智能安全:教导代理感受不可逆后果的重量

Emotional Cost Functions for AI Safety: Teaching Agents to Feel the Weight of Irreversible Consequences

Pandurang Mopgar

AI总结:

本文提出情感成本函数框架,使代理产生质性痛苦状态,通过丰富的故事表征不可逆后果,从而塑造代理的性格。实验表明,质性痛苦产生特定智慧而非一般性瘫痪。

摘要:

人类通过灾难性错误的学习并非通过数值惩罚,而是通过质性痛苦重塑自我。当前AI安全方法未能复制此过程。奖励塑造捕捉的是量而非意义,基于规则的对齐仅约束行为而不改变它。我们提出情感成本函数框架,使代理发展质性痛苦状态,即对不可逆后果的丰富叙事表征,这些状态持续向前并主动塑造性格。不同于数值惩罚,质性痛苦状态捕捉失去之物的意义、它所创造的具体空缺,以及它如何改变代理对相似未来情境的关系。我们的四组件架构——后果处理器、性格状态、前瞻性扫描和故事更新,基于一个原则:行为不可撤销,代理必须活在所造成的事物中。前瞻性恐惧通过两条路径运作。经验性恐惧源于代理自身经历的后果。预经验性恐惧是通过训练或跨代理传输获得的,无需直接经验。它们共同反映了人类智慧如何在经验和文化中积累。十项实验涵盖金融交易、危机支持和内容审核,显示质性痛苦产生特定智慧而非一般性瘫痪。代理在适度机会上正确参与率为90-100%,而数值基线过度拒绝率为90%。架构消减确认机制是必要的。完整系统生成每个探测器十句个人接地短语,而普通LLM则为零。统计验证(N=10)确认可重复性在80-100%一致性范围内。

英文摘要:

Humans learn from catastrophic mistakes not through numerical penalties, but through qualitative suffering that reshapes who they are. Current AI safety approaches replicate none of this. Reward shaping captures magnitude, not meaning. Rule-based alignment constrains behaviour, but does not change it. We propose Emotional Cost Functions, a framework in which agents develop Qualitative Suffering States, rich narrative representations of irreversible consequences that persist forward and actively reshape character. Unlike numerical penalties, qualitative suffering states capture the meaning of what was lost, the specific void it creates, and how it changes the agent's relationship to similar future situations. Our four-component architecture - Consequence Processor, Character State, Anticipatory Scan, and Story Update is grounded in one principle. Actions cannot be undone and agents must live with what they have caused. Anticipatory dread operates through two pathways. Experiential dread arises from the agent's own lived consequences. Pre-experiential dread is acquired without direct experience, through training or inter-agent transmission. Together they mirror how human wisdom accumulates across experience and culture. Ten experiments across financial trading, crisis support, and content moderation show that qualitative suffering produces specific wisdom rather than generalised paralysis. Agents correctly engage with moderate opportunities at 90-100% while numerical baselines over-refuse at 90%. Architecture ablation confirms the mechanism is necessary. The full system generates ten personal grounding phrases per probe vs. zero for a vanilla LLM. Statistical validation (N=10) confirms reproducibility at 80-100% consistency.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑