Landuse characterization is important for urban planning. It is traditionally performed with field surveys or manual photo interpretation, two practices that are time-consuming and labor-intensive. Therefore, we aim to automate landuse mapping at the urban-object level with a deep learning approach based on data from multiple sources (or modalities). We consider two image modalities: overhead imagery from Google Maps and ensembles of ground-based pictures (side-views) per urban-object from Google Street View (GSV). These modalities bring complementary visual information pertaining to the urban-objects. We propose an end-to-end trainable model, which uses OpenStreetMap annotations as labels. The model can accommodate a variable number of GSV pictures for the ground-based branch and can also function in the absence of ground pictures at prediction time. We test the effectiveness of our model over the area of Île-de-France, France, and test its generalization abilities on a set of urban-objects from the city of Nantes, France. Our proposed multimodal Convolutional Neural Network achieves considerably higher accuracies than methods that use a single image modality, making it suitable for automatic landuse map updates. Additionally, our approach could be easily scaled to multiple cities, because it is based on data sources available for many cities worldwide.
Multimodal Urban Tree Detection from Satellite and Street-Level Imagery via Annotation-Efficient Deep Learning Strategies
Department of Computer Science, University of California, Davis（加州大学戴维斯分校计算机科学系）；Department of Biological and Agricultural Engineering, University of California, Davis（加州大学戴维斯分校生物与农业工程系）
UrbanGraphEmbeddings: Learning and Evaluating Spatially Grounded Multimodal Embeddings for Urban Science
National University of Singapore Department of Architecture Singapore；The Chinese University of Hong Kong Dept of Systems Eng. \& Eng. Mgmt. China；Singapore Management University School of Computing \& Info. Systems Singapore；National University of Singapore；Department of Architecture；The Chinese University of Hong Kong；Singapore Management University
BuildingView: Constructing Urban Building Exteriors Databases with Street View Imagery and Multimodal Large Language Mode
Thrust of Urban Governance and Design, The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州）城市治理与设计中心）；Spatial Sciences Institute, University of Southern California（南加州大学空间科学学院）
Urban Boundaries, Social Barriers: A Benchmark and Vision-Centric Framework for Mapping Gated Communities and Equity Implications
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；School of Public Administration and Policy, Renmin University of China（中国人民大学公共管理学院）；Sony AI（索尼人工智能）；University of South China（南华大学）
Planning for Cooler Cities: A Multimodal AI Framework for Predicting and Mitigating Urban Heat Stress through Urban Landscape Transformation
Department of City and Regional Planning, University of Pennsylvania（城市与区域规划系，宾夕法尼亚大学）；Guangdong Key Laboratory for Urban Informatics, Guangdong-Hong Kong-Macao Joint Laboratory for Smart Cities（广东城市信息关键实验室、粤港澳大湾区智慧城市联合实验室）；Shenzhen Key Laboratory of Spatial Information Smart Sensing（深圳空间信息智能感知关键实验室）；Department of Urban Informatics, School of Architecture and Urban Planning, Shenzhen University（城市信息系，深圳大学）；College of Big Data and Internet, Shenzhen Technology University（大数据与互联网学院，深圳科技大学）
A Multimodal Physics-Informed Neural Network Approach for Mean Radiant Temperature Modeling
School of Computing and Augmented Intelligence, Arizona State University（亚利桑那州立大学计算与增强智能学院）；College of Architecture, Kuwait University（科威特大学建筑学院）；School of Arts, Media and Engineering, Arizona State University（亚利桑那州立大学艺术、媒体与工程学院）
MUSON: A Reasoning-oriented Multimodal Dataset for Socially Compliant Navigation in Urban Environments
Graduate School of Information Science and Technology, Hokkaido University（北海道大学信息科学研究生院）；Graduate School of Computer Science, George Mason University（乔治·马歇尔大学计算机科学研究生院）
TrajGANR: Trajectory-Centric Urban Multimodal Learning via Geospatially Aligned Neural Representations
Google Research, Mountain View, CA（谷歌研究，山景城，加利福尼亚州）；Google LLC, Mountain View, CA（谷歌公司，山景城，加利福尼亚州）；Dept. of Computer Science, University of Southern California, Los Angeles, CA（计算机科学系，南加州大学，洛杉矶，加利福尼亚州）；SEAI Lab, Dept. of Geography and the Environment, The University of Texas at Austin, Austin, TX（SEAI实验室，地理与环境系，德克萨斯大学奥斯汀分校，奥斯汀，德克萨斯州）
CarbonCLIP: Enhance Carbon Prediction from Satellite Imagery via Integrated Street-View Semantics and Temporal Context Training
Energy Research Institute at NTU（国立理工学院能源研究所）；Interdisciplinary Graduate Programme（跨学科研究生项目）；School of Electrical and Electronic Engineering（电气电子工程学院）；School of Public Administration and Policy（公共管理与政策学院）；Asian School of the Environment（亚洲环境学院）；Center for Climate Change and Environmental Health（气候变化与环境健康中心）
A Modality-Tailored Graph Modeling Framework for Urban Region Representation via Contrastive Learning
Center for Applied Statistics, School of Statistics, Innovation Platform, Renmin University of China（应用统计中心、统计学院、创新平台、中国人民大学）；Harbin Institute of Technology, Shenzhen（哈尔滨工业大学深圳学院）
RoadBench: Benchmarking MLLMs on Fine-Grained Spatial Understanding and Reasoning under Urban Road Scenarios
Department of Electronic Engineering, BNRist, Tsinghua University（清华大学电子工程系、北京信息科学与技术国家研究中心）；Zhongguancun Academy（中关村学院）；Amap, Alibaba Group（阿里巴巴集团高德地图）
Fusion and Grouping Strategies in Deep Learning for Local Climate Zone Classification of Multimodal Remote Sensing Data
Graphics-Visualization-Computing Lab, International Institute of Information Technology Bangalore, Karnataka 560100, India（图形可视化计算实验室，国际信息学院班加罗尔，卡纳塔克邦560100，印度）
Semantic Segmentation of Earth Observation Data Using Multimodal and Multi-scale Deep Networks
Université Paris Diderot, Sorbonne Paris Cité, Laboratoire Matière et Systèmes Complexes (MSC), UMR 7057 CNRS（巴黎狄德罗大学，索邦巴黎西岱大学，复杂物质与系统实验室（MSC），CNRS UMR 7057）
