Interactive-agent evaluations commonly rely on standardized LLM-based user simulators that exhibit broadly cooperative behavior, potentially overlooking failures that arise under diverse but realistic user interactions. This can lead to overly optimistic estimates of agent robustness and provide limited failure signals for training agents that generalize across users. Existing approaches largely rely on predefined user behaviors, which may miss agent- or domain-specific weaknesses, while failure-oriented search can reduce task success by invalidating the underlying task rather than exposing genuine robustness failures. To fill this gap, we introduce TIPER, a task-integrity-preserving persona evolution method that automatically discovers agent-specific user policies exposing robustness weaknesses while keeping the underlying task valid and solvable. TIPER evolves persistent user personas through reflection-guided behavioral mutations and task-wise Pareto selection under a task-integrity constraint. Across four interactive-agent benchmarks, TIPER yields lower held-out avg@3 scores in all 12 GPT-5.4 settings, with observed reductions of 5.8 to 33.7 percentage points, and in all 28 agent-domain pairs spanning seven agents, with a median observed reduction of 22.9 points. Further analysis yields a behavior-to-failure taxonomy that connects discovered user behaviors to recurring agent failure modes. Together, these results show that TIPER enables more diagnostic robustness evaluation of interactive agents and provides informative failure analysis for future agent robustness improvement. Code repository: https://github.com/UCSB-NLP-Chang/TIPER.
Uncovering Logit Suppression Vulnerabilities in LLM Safety Alignment
Huazhong University of Science and Technology(华中科技大学);Nanyang Technological University(南洋理工大学);University of New South Wales(新南威尔士大学);Nankai University(南开大学)
Identifying the Achilles' Heel: An Iterative Method for Dynamically Uncovering Factual Errors in Large Language Models
Renmin University of China(中国人民大学);The Chinese University of Hong Kong(香港中文大学);The Chinese University of Hong Kong, Shenzhen(香港中文大学深圳校区);Johns Hopkins University(约翰霍普金斯大学);Nanyang Technological University(南洋理工大学);Xiaohongshu Inc.(小红书公司);Tencent Inc.(腾讯公司)
Digital Skin, Digital Bias: Uncovering Tone-Based Biases in LLMs and Emoji Embeddings
University of North Texas(北德克萨斯大学);International Center for Artificial Intelligence Research and Ethics(国际人工智能研究与伦理中心);Southern Methodist University(南卫理公会大学);The University of Arizona(亚利桑那大学);The University of Alabama(阿拉巴马大学)
Agent4Edu: Generating Learner Response Data by Generative Agents for Intelligent Education Systems
State Key Laboratory of Cognitive Intelligence(认知智能国家重点实验室);University of Science and Technology of China(中国科学技术大学);Institute of Artificial Intelligence(人工智能研究院);Hefei Comprehensive National Science Center(合肥综合性国家科学中心)