VLM-Guided Iterative Refinement for Surgical Image Segmentation with Foundation Models
School of Automation Science and Engineering, South China University of Technology, Guangzhou, China（自动化科学与工程学院，华南理工大学，广州，中国）；Vanderbilt University, Nashville, TN, USA（范德比尔特大学，纳什维尔，田纳西州，美国）；The Pennsylvania State University, University Park, PA , USA（宾夕法尼亚州立大学，大学园，宾夕法尼亚州，美国）；Peking University, Beijing , China（北京大学，北京，中国）；Virginia Commonwealth University, Richmond, VA, USA（弗吉尼亚共同wealth大学，里士满，弗吉尼亚州，美国）；University of California San Diego, San Diego, CA, USA（加州大学圣地亚哥分校，圣地亚哥，加利福尼亚州，美国）
Surgical image segmentation is essential for robot-assisted surgery and intraoperative guidance. However, existing methods are constrained to predefined categories, produce one-shot predictions without adaptive refinement, and lack mechanisms for clinician interaction. We propose IR-SIS, an iterative refinement system for surgical image segmentation that accepts natural language descriptions. IR-SIS leverages a fine-tuned SAM3 for initial segmentation, employs a Vision-Language Model to detect instruments and assess segmentation quality, and applies an agentic workflow that adaptively selects refinement strategies. The system supports clinician-in-the-loop interaction through natural language feedback. We also construct a multi-granularity language-annotated dataset from EndoVis2017 and EndoVis2018 benchmarks. Experiments demonstrate state-of-the-art performance on both in-domain and out-of-distribution data, with clinician interaction providing additional improvements. Our work establishes the first language-based surgical segmentation framework with adaptive self-refinement capabilities.
RefineRank: Joint Box Refinement and Ranking for Surgical Spatio-Temporal Grounding
UCL Hawkes Institute, University College London（伦敦大学学院霍克斯研究所）；King’s College London（伦敦国王学院）；School of Medicine, Nankai University（南开大学医学院）；University of Manchester（曼彻斯特大学）
VLM-R$^3$: Region Recognition, Reasoning, and Refinement for Enhanced Multimodal Chain-of-Thought
National Engineering Research Center for Software Engineering, Peking University（软件工程国家工程研究中心，北京大学）；Alibaba Group（阿里巴巴集团）；ZEEKR Intelligent Technology Holding Limited（ZEKR智能科技控股有限公司）
MedClaw: Heuristic Agent Harness for Long-Horizon Surgical Video Reasoning
School of Automation and Intelligence, Beijing Jiaotong University（北京交通大学自动化与智能学院）；University of Maryland（马里兰大学）；Suzhou Institute for Advanced Research, University of Science and Technology of China（中国科学技术大学苏州高等研究院）；University of British Columbia（不列颠哥伦比亚大学）；Beijing Tiantan Hospital, Capital Medical University（首都医科大学附属北京天坛医院）
Distilling Expert Surgical Knowledge: How to train local surgical VLMs for anatomy explanation in Complete Mesocolic Excision
1 Institute of Medical Technology；Intelligent Systems, Hamburg University of Technology, Hamburg, Germany 2 Department of General, Visceral；Thoracic Surgery, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
SurGo-R1: Benchmarking and Modeling Contextual Reasoning for Operative Zone in Surgical Video
National University of Singapore, Singapore（新加坡国立大学）；Southern Medical University, China（南方医科大学）；Guangzhou Research Translation and Innovation Institute, National University of Singapore, China（广州研究翻译与创新研究院，新加坡国立大学，中国）
Your Vision-Language Model Can't Even Count to 20: Exposing the Failures of VLMs in Compositional Counting
Guilin University of Electronic Technology（桂林电子科技大学）；The Ohio State University（俄亥俄州立大学）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；University of California, Berkeley（加州大学伯克利分校）
VLM-PAR: A Vision Language Model for Pedestrian Attribute Recognition
Department of Innovation Engineering（创新工程系）；University of Salento, Italy（意大利萨伦托大学）；Institute of Applied Sciences and Intelligent Systems - CNR（应用科学与智能系统研究所 - CNR）；University of the Basque Country UPV/EHU（巴斯克国家大学UPV/EHU）；IKERBASQUE, Basque Foundation for Science（伊基塔斯克巴塞克基金会）；Sorbonne University Abu Dhabi（索邦大学阿布扎比分校）
TAVR-VLM: Risk-Conditioned Causal Grounding for Hallucination-Resistant Report Generation
Xi’an Jiaotong-Liverpool University（西交利物浦大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Shanghai Jiao Tong University（上海交通大学）；University of Liverpool（利物浦大学）；Kunming University of Science and Technology（昆明理工大学）
MaxSAT-Based Feedback for Guiding Vision-Language Models in Sudoku
Artificial Intelligence Research Institute (IIIA), Consejo Superior de Investigaciones Científicas (CSIC)（人工智能研究所（IIIA），西班牙科学研究高级理事会（CSIC））；Institut de Robòtica i Informàtica Industrial (IRI-CSIC-UPC)（工业机器人与信息学研究所（IRI-CSIC-UPC））
Surgical Anatomy Recognition with Context Learning using Foundation Representations
Department of Biomedical Engineering, Medical Image Analysis, Eindhoven University of Technology（埃因霍温理工大学，生物医学工程系，医学图像分析）；Department of Electrical Engineering, Architectures for Reliable Image Analysis, Eindhoven University of Technology（埃因霍温理工大学，电气工程系，可靠图像分析架构）；Department of Surgery, University Medical Center Utrecht（乌得勒支大学医学中心，外科）；Department of Oncological Urology, University Medical Center Utrecht（乌得勒支大学医学中心，肿瘤泌尿外科）；Department of Urology, Catharina Hospital（凯瑟琳娜医院，泌尿外科）；Department of Electrical Engineering, Mobile Perception Systems Lab, Eindhoven University of Technology（埃因霍温理工大学，电气工程系，移动感知系统实验室）
SpatialAct: Probing Spatial Reasoning-to-Action Capabilities of VLM Agents in 3D Scenes
The Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学（广州））；Zhongguancun Academy（中关村学院）；Tsinghua University（清华大学）；Helsinki University（赫尔辛基大学）
SurgiSR4K: A High-Resolution Endoscopic Video Dataset for Robotic-Assisted Minimally Invasive Procedures
Intuitive Surgical, Inc.（Intuitive Surgical公司）；Johns Hopkins Medicine Neurosurgery（约翰霍普金斯医学神经外科）；Johns Hopkins University Electrical and Computer Engineering（约翰霍普金斯大学电气与计算机工程）；University of British Columbia Electrical and Computer Engineering（不列颠哥伦比亚大学电气与计算机工程）；Wilford & Kate Bailey Small Animal Teaching Hospital（威尔福德与凯蒂·贝利小动物教学医院）
Challenging Vision-Language Models with Surgical Data: A New Dataset and Broad Benchmarking Study
organization= German Cancer Research Center (DKFZ) Heidelberg, Div. Intelligent Medical Systems , country= Germany；organization= National Center for Tumor Diseases (NCT), NCT Heidelberg , country= Germany；Data Science School for Health , country= Germany；organization= Medical Faculty, Heidelberg University , country= Germany；organization= Faculty of Mathematics；Computer Science, Heidelberg University , country= Germany；organization= Weldon School of Biomedical Engineering, Purdue University , city= West Lafayette , state= IN , country= USA；organization= Regenstrief Center for Healthcare Engineering (RCHE), Purdue University , city= West Lafayette , state= IN , country= USA；organization= Department of Biostatistics；Health Data Science, Richard M. Fairbanks School of Public Health, Indiana University School of Medicine , city= Indianapolis , state= IN , country= USA；organization= Department of Surgery, Indiana University School of Medicine , city= Indianapolis , state= IN , country= USA；organization= Department of Visceral, Thoracic；Vascular Surgery, University Hospital；Faculty of Medicine Carl Gustav Carus, TUD Dresden University of Technology , city= Dresden , country= Germany
SurgAtt-Tracker: Online Surgical Attention Tracking via Temporal Proposal Reranking and Motion-Aware Refinement
Department of Electronic Engineering, The Chinese University of Hong Kong, Hong Kong SAR, China（香港中文大学电子工程系）；Division of Gastrointestinal Surgery, Shenzhen People's Hospital, China（深圳人民医院胃肠外科）；College of Mechatronics and Control Engineering, Shenzhen University, China（深圳大学机电与控制工程学院）；Department of Mechanical Engineering, The University of Hong Kong, Hong Kong SAR, China（香港大学机械工程系）；Department of Electronic and Electrical Engineering, Southern University of Science and Technology（南方科技大学电子与电气工程系）
DiSa: Saliency-Aware Foreground-Background Disentangled Framework for Open-Vocabulary Semantic Segmentation
Department of Computer Science and Engineering, Lehigh University（计算机科学与工程系，莱恩大学）；Department of Computer Science（计算机科学系）；Engineering, Lehigh University（工程系，莱恩大学）；Qualcomm AI Research（高通人工智能研究）
Zero-Effort Image-to-Music Generation: An Interpretable RAG-based VLM Approach
The Hong Kong University of Science and Technology（香港科学与技术大学）；The Hong Kong Polytechnic University（香港理工大学）；The University of Hong Kong（香港大学）；The Hong Kong University of Science（香港科学大学）；The Hong Kong Polytechnic University Hong Kong China（香港理工大学香港中国）；The University of Hong Kong Hong Kong China（香港大学香港中国）
