InfiGUIAgent: A Multimodal Generalist GUI Agent with Native Reasoning and Reflection
Zhejiang University（浙江大学）；Dalian University of Technology（大连理工大学）；Reallm Labs（Reallm实验室）；ByteDance Inc（字节跳动公司）；The Hong Kong Polytechnic University（香港理工大学）
Graphical User Interface (GUI) Agents, powered by multimodal large language models (MLLMs), have shown great potential for task automation on computing devices such as computers and mobile phones. However, existing agents face challenges in multi-step reasoning and reliance on textual annotations, limiting their effectiveness. We introduce \textit{InfiGUIAgent}, an MLLM-based GUI Agent trained with a two-stage supervised fine-tuning pipeline. Stage 1 enhances fundamental skills such as GUI understanding and grounding, while Stage 2 integrates hierarchical reasoning and expectation-reflection reasoning skills using synthesized data to enable native reasoning abilities of the agents. \textit{InfiGUIAgent} achieves competitive performance on several GUI benchmarks, highlighting the impact of native reasoning skills in enhancing GUI interaction for automation tasks. Resources are available at \url{https://github.com/Reallm-Labs/InfiGUIAgent}.
Specialist or Generalist? Instruction Tuning for Specific NLP Tasks
Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；Language Technology Lab, University of Cambridge（剑桥大学语言技术实验室）；Cohere；Tencent AI Lab（腾讯AI实验室）
Virginia Tech（弗吉尼亚理工大学）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；University of California, Berkeley（加州大学伯克利分校）
TAGS: A Test-Time Generalist-Specialist Framework with Retrieval-Augmented Reasoning and Verification
Mohamed bin Zayed University of Artificial Intelligence（莫莫德·本·扎耶德人工智能大学）；Monash University（墨尔本大学）；Xi’an Jiaotong-Liverpool University（西安交通大学利物浦大学）；University of Southampton（南安普顿大学）
Generalist Large Language Models Outperform Clinical Tools on Medical Benchmarks
NYU Langone Health（纽约大学朗格尼医学中心）；The University of Texas at Austin（得克萨斯大学奥斯汀分校）；Washington University School of Medicine in St. Louis（圣路易斯华盛顿大学医学院）；New York University（纽约大学）
Inference-Time Scaling for Generalist Reward Modeling
DeepSeek-AI；Dept. of Computer Sci. & Tech., Tsinghua University（计算机科学与技术系，清华大学）；Institute for AI Industry Research (AIR), Tsinghua University（人工智能产业研究院（AIR），清华大学）
One Filters All: A Generalist Filter for State Estimation
School of Vehicle and Mobility, Tsinghua University（清华大学车辆与移动系统学院）；College of Engineering, Peking University（北京大学工程学院）；College of AI, Tsinghua University（清华大学人工智能学院）
HalluClear: Diagnosing, Evaluating and Mitigating Hallucinations in GUI Agents
MAIS&NLPR, Institute of Automation, Chinese Academy of Sciences（自动化研究所人工智能与自然语言处理实验室，中国科学院）；School of Artificial Intelligence, UCAS（中国科学院大学人工智能学院）；Meituan（美团）
Large language models require a new form of oversight: capability-based monitoring
MIT Sloan School of Management（麻省理工学院斯隆管理学院）；Mass General Brigham（麻省总医院布里格姆）；Harvard Medical School（哈佛医学院）；Brigham and Women’s Hospital（布莱根妇女医院）；Dana-Farber Cancer Institute（丹娜-法伯癌症研究所）；Brandeis University（布兰迪斯大学）；Harvard John A. Paulson School Of Engineering And Applied Sciences（哈佛大学约翰·A·保尔森工程与应用科学学院）；Boston Children’s Hospital（波士顿儿童医院）；Northeastern University（东北大学）
CityRiSE: Reasoning Urban Socio-Economic Status in Large Vision-Language Models via Reinforcement Learning
Information Hub, The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州）信息中心）；Department of Electronic Engineering, BNRist, Tsinghua University（清华大学电子工程系）
OS-Themis: A Scalable Critic Framework for Generalist GUI Rewards
University of Science and Technology of China（中国科学技术大学）；Shanghai AI Laboratory（上海人工智能实验室）；CUHK MMLab（香港中文大学多模态实验室）；The Hong Kong University of Science and Technology（香港科学大学）；National University of Singapore（新加坡国立大学）
Caution for the Environment: Multimodal LLM Agents are Susceptible to Environmental Distractions
School of Computer Science（计算机学院）；Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering（智能交互与认知工程重点实验室）；Shanghai Jiao Tong University（上海交通大学）；Shanghai Key Laboratory of Trusted Data Circulation and Governance in Web3（Web3可信数据流通与治理上海市重点实验室）；GenAI, Meta（Meta GenAI）
ScreenAgent: A Vision Language Model-driven Computer Control Agent
School of Artificial Intelligence, Jilin University（吉林大学人工智能学院）；Engineering Research Center of Knowledge-Driven Human-Machine Intelligence, Ministry of Education, China（教育部知识驱动人机智能工程研究中心）
OR-VSKC: Resolving Visual-Semantic Knowledge Conflicts in Operating Rooms with Synthetic Data-Guided Alignment
Shanghai University of Engineering Science（上海工程技术大学）；Tencent YouTu Lab（腾讯优图实验室）；Clinical Research Unit, Zhongshan Hospital of Fudan University（复旦大学中山医院临床研究部）
Radiology's Last Exam (RadLE): Benchmarking Frontier Multimodal AI Against Human Experts and a Taxonomy of Visual Reasoning Errors in Radiology
Centre for Responsible Autonomous Systems in Healthcare (CRASH) Lab, Koita Centre for Digital Health（负责任的自主医疗系统中心（CRASH）实验室、Koita数字健康中心）；Ashoka University（阿什oka大学）；Independent Researcher（独立研究者）；Koita Centre for Digital Health（Koita数字健康中心）
Read to Play (R2-Play): Decision Transformer with Multimodal Game Instruction
Multimodal Art Projection Research Community（多模态艺术投影研究社区）；Beijing University of Posts and Telecommunications（北京邮电大学）；HKUST（香港科技大学）；University of Waterloo（滑铁卢大学）；Vector Institute（矢量研究所）
UI-Oceanus: Scaling GUI Agents with Synthetic Environmental Dynamics
Key Lab of HCST (PKU), MOE（高可信软件技术教育部重点实验室（北京大学）；北京大学计算机学院）；School of Computer Science, Peking University（北京通明湖信息技术应用创新中心）；Beijing Tongming Lake Information Technology Application Innovation Center（腾讯微信）；WeChat, Tencent Inc.（西蒙菲莎大学）；Simon Fraser University（德克萨斯大学达拉斯分校）；University of Texas at Dallas（复旦大学系统与先进计算研究所）；Fudan University Institute of Systems for Advanced Computing（上海开放计算系统研究所）；Shanghai Institute of Systems for Open Computing
