Agentic large language models (LLMs) are increasingly evaluated on cybersecurity tasks using capture-the-flag (CTF) benchmarks, yet existing pointwise benchmarks offer limited insight into agent robustness and generalisation across alternative versions of the source code. We introduce CTF challenge families, whereby a single CTF is used to generate a family of semantically-equivalent challenges via semantics-preserving program transformations, enabling controlled evaluation of robustness while keeping the underlying exploit strategy fixed. We present Evolve-CTF, a tool that generates CTF families from Python challenges using a range of transformations. Using Evolve-CTF to derive families from Cybench and Intercode challenges, we evaluate 13 agentic LLM configurations with tool access. We find that models are remarkably robust to renaming and code insertion, but that composed transformations and deeper obfuscation degrade performance by requiring more sophisticated tool use. Enabling explicit reasoning has little effect on success rates. Our work contributes a technique and tool for future LLM evaluations, and a large dataset characterising the capabilities of current state-of-the-art models in this domain.
How Do LLM Agents Actually Get the Flag? Trace-Level Provenance for Agentic Offensive Security Evaluation
NYU Tandon School of Engineering(纽约大学坦登工程学院);NYU Abu Dhabi(纽约大学阿布扎比分校);CISPA - Helmholtz Center for Information Security(CISPA亥姆霍兹信息安全中心);International Institute of Information Technology Hyderabad(海得拉巴国际信息技术学院);Indian Institute of Technology Tirupati(蒂鲁帕蒂印度理工学院)
AgenticSum: An Agentic Inference-Time Framework for Faithful Clinical Text Summarization
Department of XXX, University of YYY, Location, Country(XXX系,YYY大学,地点,国家);School of ZZZ, Institute of WWW, Location, Country(ZZZ学院,WWW研究所,地点,国家);University of Delaware, Newark, DE, USA(特拉华大学)
GeoCloak: An Agentic Framework for Semantics-Preserving Geolocation Privacy via Peripheral Clue Manipulation
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));The Hong Kong University of Science and Technology(香港科技大学);CISPA Helmholtz Center for Information Security(CISPA赫尔姆霍兹信息安全中心)
Coding-Free and Privacy-Preserving Agentic Framework for Data-Driven Clinical Research
Infmedix, Co., Ltd.(Infmedix公司);Department of Transdisciplinary Studies, Seoul National University(首尔国立大学跨学科研究系);Healthcare AI Research Institute, Seoul National University Hospital(首尔国立大学医院医疗人工智能研究所);Department of Medicine, Seoul National University College of Medicine(首尔国立大学医学院医学系);Department of Transdisciplinary Medicine, Seoul National University Hospital(首尔国立大学医院跨学科医学系);Department of Radiology, Massachusetts General Hospital and Harvard Medical School(麻省总医院放射科及哈佛医学院)
FLAG-Trader: Fusion LLM-Agent with Gradient-based Reinforcement Learning for Financial Trading
Harvard University(哈佛大学);Stevens Institute of Technology(史蒂文斯理工学院);Columbia University(哥伦比亚大学);University of Minnesota(明尼苏达大学);NVIDIA(英伟达公司);Rensselaer Polytechnic Institute(伦斯勒理工学院);TheFinAI;University of Manchester(曼彻斯特大学)
Agentic Explainable Artificial Intelligence (Agentic XAI) Approach To Explore Better Explanation: A Case Study in Decision Support for Rice Cultivation in Japan
Less Interaction But More Explanation: A Communication Perspective on Agentic AI Interfaces
Media Effects Research Laboratory, Pennsylvania State University(媒体影响研究实验室,宾夕法尼亚州立大学);Center for Socially Responsible AI, Pennsylvania State University(负责任人工智能中心,宾夕法尼亚州立大学)
From Privacy to Trust in the Agentic Era: A Taxonomy of Challenges in Trustworthy Federated Learning Through the Lens of Trust Report 2.0
Department of Computer Science and Artificial Intelligence, Andalusian Research Institute in Data Science and Computational Intelligence (DaSCI) University of Granada(计算机科学与人工智能系,数据科学与计算智能安达卢西亚研究 institute,格拉纳达大学);Department of Software Engineering, Andalusian Research Institute in Data Science and Computational Intelligence (DaSCI) University of Granada(软件工程系,数据科学与计算智能安达卢西亚研究 institute,格拉纳达大学)
RefGrader: Automated Grading of Mathematical Competition Proofs using Agentic Workflows
Pennsylvania State University(宾夕法尼亚州立大学);City University of New York(纽约市立大学);New York University(纽约大学);Amirkabir University of Technology(阿米尔卡比尔理工大学);Autodesk(欧特克公司);Carnegie Mellon University(卡内基梅隆大学)
City Editing: Hierarchical Agentic Execution for Dependency-Aware Urban Geospatial Modification
University of Kansas, Lawrence, KS, USA(堪萨斯大学);Seoul National University, Seoul, South Korea(首尔国立大学);University of Florida, Gainesville, FL, USA(佛罗里达大学);NEC Laboratories America, Princeton, NJ, USA(NEC美国实验室);Clemson University, Clemson, SC, USA(克莱姆森大学);Arizona State University, Tempe, AZ, USA(亚利桑那州立大学)
Agentic Learner with Grow-and-Refine Multimodal Semantic Memory
Nanjing University of Science and Technology(南京理工大学);Baidu Inc(百度公司);AIML, Adelaide University(AIML,阿德莱德大学);Singapore University of Technology and Design(新加坡科技设计大学)