arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.03344cs.LGcs.AIcs.CL
收藏

鲁棒性作为任务性能的涌现属性

Robustness as an Emergent Property of Task Performance

IBM Research(IBM研究院);MIT(麻省理工学院)

Shir Ashury-Tahan, Ariel Gera, Elron Bandel, Michal Shmueli-Scheuer, Leshem Choshen

AI总结:

本文提出鲁棒性是任务性能的涌现属性,通过实证分析发现鲁棒性主要由任务特定能力驱动,而非模型固有属性,挑战了传统观点。

摘要:

鲁棒性常被视为现实应用中的关键未来挑战,其中稳定性至关重要。然而,由于模型往往以相似的顺序学习任务,我们假设更容易的任务无论以何种方式呈现给模型都会更容易。事实上,在本文中,我们证明了当模型在任务上接近高性能时,鲁棒性实际上已被有效实现。通过在多个模型和多样本集上进行实证分析(例如,同义词、不同温度),我们发现存在强正相关性。此外,我们发现鲁棒性主要由任务特定能力驱动,而不是模型层面的固有属性,这挑战了当前将鲁棒性视为独立能力的方法。从高层视角来看,我们可能期望随着新任务饱和,模型在这些任务上的鲁棒性将相应出现。对于研究人员来说,这表明显式努力测量和提高鲁棒性可能值得减少关注,因为这种鲁棒性很可能随着性能提升而发展。对于实践者来说,这表明文献中处理的任务确实不可靠,但在过去较简单的任务上,模型是可靠的,已准备好应用于现实世界。

英文摘要:

Robustness is widely viewed as a key challenge for real-world applications. However, because current research focuses only on difficult tasks, it partially captures real-world readiness. In this paper, we argue and verify that robustness, defined as consistency across semantically equivalent inputs, closely follows task difficulty: once models master a task, robustness emerges naturally. Through an empirical analysis of multiple models across diverse datasets and configurations (e.g., paraphrases, temperature changes), we observe a strong positive correlation between task performance and robustness. Furthermore, our findings indicate that robustness is driven primarily by task-specific competence rather than inherent model attributes, challenging the common view of robustness as an independent capability. This perspective implies that as tasks mature and model performance saturates, robustness on those tasks will similarly emerge. For researchers, this suggests that explicit efforts to measure robustness may deserve reduced emphasis, as robustness is likely to improve alongside performance. For practitioners, it signals that while many existing benchmarks are still unstable, models are already reliable on earlier tasks and suitable for deployment.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑