GRASP: Geospatial pixel Reasoning viA Structured Policy learning
Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；School of Information, Central University of Finance and Economics（中央财经大学信息学院）；Key Laboratory of Geographic Information Science (Ministry of Education), East China Normal University（教育部地理信息科学重点实验室，华东师范大学）；School of Electronic and Computer Engineering, Peking University（北京大学电子工程学院）；Department of Geography and Environmental Management, University of Waterloo（滑铁卢大学地理与环境管理系）
Chengjie Jiang, Yunqi Zhou, Jiafeng Yan, Jing Li, Jiayang Li, Yue Zhou, Hongjie He, Jonathan Li
Geospatial pixel reasoning aims to generate segmentation masks in remote sensing imagery directly from natural-language instructions. Most existing approaches follow a paradigm that fine-tunes multimodal large language models under supervision with dense pixel-level masks as ground truth. While effective within the training data distribution, this design suffers from two main drawbacks: (1) the high cost of large-scale dense mask annotation, and (2) the limited generalization capability of supervised fine-tuning in out-of-domain scenarios. To address these issues, we propose GRASP, a structured policy-learning framework that integrates a multimodal large language model with a pretrained segmentation model in a cascaded manner. To enhance generalization, we introduce PRIME, a training paradigm that replaces supervised fine-tuning with reinforcement learning to better align reasoning and grounding behaviors with task objectives. To reduce annotation costs, we design BoP-Rewards, which substitutes dense mask labels with bounding box and positive points. It further verifies outputs through two complementary signals: format, which constrains the reasoning and grounding structure to remain syntactically parsable, and accuracy, which evaluates the quality of predicted boxes and points. For evaluation, we train our method and all baselines on EarthReason and GeoPixInstruct, constructing an in-domain benchmark by merging their test sets. We further release GRASP-1k, a fully out-of-domain benchmark with reasoning-intensive queries, reasoning traces, and fine-grained masks. Experimental results demonstrate state-of-the-art (SOTA) in-domain performance and up to 54\% improvement in out-of-domain scenarios, confirming that reinforcement learning with cost-aware rewards provides a robust and scalable paradigm for geospatial pixel reasoning. All code and datasets will be released publicly.
Building age estimation based on widely available multi-source1 geospatial data
The University of Hong Kong（香港大学）；Institute for Climate and Carbon Neutrality, the University of Hong Kong（香港大学气候与碳中和研究所）；The University of Hong Kong Shenzhen Institute of Research and Innovation（香港大学深圳研究院）；School of Information Engineering, Guangdong University of Technology（广东工业大学信息工程学院）；Institute for Smart Cities, School of Architecture and Urban Planning, Shenzhen University（深圳大学建筑与城市规划学院智慧城市研究院）；College of Urban and Environmental Sciences, Academy of Frontier Interdisciplinary Research, Central China Normal University（华中师范大学城市与环境科学学院前沿交叉研究院）；University of North Carolina（北卡罗来纳大学）；Chulalongkorn University（朱拉隆功大学）
HyBiomass: Global Hyperspectral Imagery Benchmark Dataset for Evaluating Geospatial Foundation Models in Forest Aboveground Biomass Estimation
Remote Sensing Technology Institute, German Aerospace Center (DLR)（远程传感技术研究所，德国航天中心）；Global Land Monitoring Group, Helmholtz German Research Centre for Geosciences (GFZ)（全球土地监测组，海德堡德国地理科学研究中心）；Department of Aerospace Engineering, University of the Bundeswehr Munich（航空航天工程系，联邦国防军 Munich 大学）；Chair of Data Science in Earth Observation (SiPEO), Technical University of Munich (TUM)（地球观测数据科学教授职位（SiPEO），慕尼黑技术大学）
From Pixels to Places: A Systematic Benchmark for Evaluating Image Geolocalization Ability in Large Language Models
University of South Florida Tampa USA；University of Alabama Tuscaloosa USA；University of Michigan Ann Arbor USA；Texas Tech University Lubbock USA；Texas A \& M University College Station USA；University of Pittsburgh Pittsburgh USA；University of South Florida；University of Alabama；University of Michigan；Texas Tech University；Texas A \& M University；University of Pittsburgh
Instrumenting and collecting annotated visual grasping datasets to train modern machine learning algorithms can be extremely time-consuming and expensive. An appealing alternative is to use off-the-shelf simulators to render synthetic data for which ground-truth annotations are generated automatically. Unfortunately, models trained purely on simulated data often fail to generalize to the real world. We study how randomized simulated environments and domain adaptation methods can be extended to train a grasping system to grasp novel objects from raw monocular RGB images. We extensively evaluate…
This paper presents Densely Supervised Grasp Detector (DSGD), a deep learning framework which combines CNN structures with layer-wise feature fusion and produces grasps and their confidence scores at different levels of the image hierarchy (i.e., global-, region-, and pixel-levels). % Specifically, at the global-level, DSGD uses the entire image information to predict a grasp. At the region-level, DSGD uses a region proposal network to identify salient regions in the image and predicts a grasp for each salient region. At the pixel-level, DSGD uses a fully convolutional network and predicts a g…
Humans learn to recognize and manipulate new objects in lifelong settings without forgetting the previously gained knowledge under non-stationary and sequential conditions. In autonomous systems, the agents also need to mitigate similar behavior to continually learn the new object categories and adapt to new environments. In most conventional deep neural networks, this is not possible due to the problem of catastrophic forgetting, where the newly gained knowledge overwrites existing representations. Furthermore, most state-of-the-art models excel either in recognizing the objects or in grasp p…
PEGG-Net: Pixel-Wise Efficient Grasp Generation in Complex Scenes
National University of Singapore Graduate School（新加坡国立大学研究生院）；Hong Kong University of Science and Technology（香港科技大学）；National University of Singapore（新加坡国立大学）
Extracting hand regions and their grasp information from images robustly in real-time is critical for occupants' safety and in-vehicular infotainment applications. It must however, be noted that naturalistic driving scenes suffer from rapidly changing illumination and occlusion. This is aggravated by the fact that hands are highly deformable objects, and change in appearance frequently. This work addresses the task of accurately localizing driver hands and classifying the grasp state of each hand. We use a fast ConvNet to first detect likely hand regions. Next, a pixel-based skin classifier th…
Training robust supervised deep learning models for many geospatial applications of computer vision is difficult due to dearth of class-balanced and diverse training data. Conversely, obtaining enough training data for many applications is financially prohibitive or may be infeasible, especially when the application involves modeling rare or extreme events. Synthetically generating data (and labels) using a generative model that can sample from a target distribution and exploit the multi-scale nature of images can be an inexpensive solution to address scarcity of labeled data. Towards this goa…
Training robust supervised deep learning models for many geospatial applications of computer vision is difficult due to dearth of class-balanced and diverse training data. Conversely, obtaining enough training data for many applications is financially prohibitive or may be infeasible, especially when the application involves modeling rare or extreme events. Synthetically generating data (and labels) using a generative model that can sample from a target distribution and exploit the multi-scale nature of images can be an inexpensive solution to address scarcity of labeled data. Towards this goa…
GeoRefer-Bench: A Benchmark from Referring Pixels to Verifiable Geospatial Reasoning
Central South University（中南大学）；Jiangxi Normal University（江西师范大学）；Hohai University（河海大学）；Renmin University of China（中国人民大学）；The Chinese University of Hong Kong（香港中文大学）
The University of Tokyo（东京大学）；Carnegie Mellon University（卡内基梅隆大学）
We aim to improve the performance of regressing hand keypoints and segmenting pixel-level hand masks under new imaging conditions (e.g., outdoors) when we only have labeled images taken under very different conditions (e.g., indoors). In the real world, it is important that the model trained for both tasks works under various imaging conditions. However, their variation covered by existing labeled hand datasets is limited. Thus, it is necessary to adapt the model trained on the labeled images (source) to unlabeled images (target) with unseen imaging conditions. While self-training domain adapt…
Pixel-Wise Recognition for Holistic Surgical Scene Understanding
Center for Research and Formation in Artificial Intelligence (CinfonIA)（人工智能研究与形成中心（CinfonIA））；Universidad de los Andes（安第斯大学）；Fundación Santa Fe de Bogotá（圣菲波哥大基金会）
GeoSeg-OV: Bridging Geospatial Gaps with Structural Guidance for Open-Vocabulary Remote Sensing Segmentation
School of Computer Science, China University of Geosciences（中国地质大学计算机学院）；Systems Hub, The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州）系统中心）；Department of Computer Science, City University of Hong Kong（香港城市大学计算机系）；School of Geodesy and Geomatics, Wuhan University（武汉大学测绘学院）；Data Science and AI Innovation Research Promotion Center, Shiga University（滋贺大学数据科学与人工智能创新研究促进中心）
To aid humans in everyday tasks, robots need to know which objects exist in the scene, where they are, and how to grasp and manipulate them in different situations. Therefore, object recognition and grasping are two key functionalities for autonomous robots. Most state-of-the-art approaches treat object recognition and grasping as two separate problems, even though both use visual input. Furthermore, the knowledge of the robot is fixed after the training phase. In such cases, if the robot encounters new object categories, it must be retrained to incorporate new information without catastrophic…
Grasp Any Region: Towards Precise, Contextual Pixel Understanding for Multimodal LLMs
New Laboratory of Pattern Recognition (NLPR), State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS), Institute of Automation, Chinese Academy of Sciences (CASIA)（模式识别新实验室、多模态人工智能系统国家重点实验室、自动化研究所、中国科学院（CASIA））；University of Chinese Academy of Sciences（中国科学院大学）；Peking University（北京大学）；Wuhan University（武汉大学）；ByteDance（字节跳动）
SegAssess: Panoramic quality mapping for robust and transferable unsupervised segmentation assessment
School of Remote Sensing and Information Engineering, Wuhan University（武汉大学遥感与信息工程学院）；College of Electronic Science and Technology, National University of Defense Technology（国防科技大学电子科学学院）；Hubei Luojia Laboratory（湖北珞珈实验室）
