Understanding intrinsic patterns and predicting spatiotemporal characteristics of cities require a comprehensive representation of urban neighborhoods. Existing works relied on either inter- or intra-region connectivities to generate neighborhood representations but failed to fully utilize the informative yet heterogeneous data within neighborhoods. In this work, we propose Urban2Vec, an unsupervised multi-modal framework which incorporates both street view imagery and point-of-interest (POI) data to learn neighborhood embeddings. Specifically, we use a convolutional neural network to extract visual features from street view images while preserving geospatial similarity. Furthermore, we model each POI as a bag-of-words containing its category, rating, and review information. Analog to document embedding in natural language processing, we establish the semantic similarity between neighborhood ("document") and the words from its surrounding POIs in the vector space. By jointly encoding visual, textual, and geospatial information into the neighborhood representation, Urban2Vec can achieve performances better than baseline models and comparable to fully-supervised methods in downstream prediction tasks. Extensive experiments on three U.S. metropolitan areas also demonstrate the model interpretability, generalization capability, and its value in neighborhood similarity analysis.
Semantic4Safety: Causal Insights from Zero-shot Street View Imagery Segmentation for Urban Road Safety
School of Geospatial Engineering and Science, Sun Yat-sen University（地理空间工程与科学学院，中山大学）；School of Geographical and Earth Sciences, University of Glasgow（地理与地球科学学院，格拉斯哥大学）；School of Economics and Management, Shanxi University（经济学与管理学院，山西大学）
Cloud to Street；Rutgers University（罗格斯大学）；European Space Agency（欧洲空间局）
To address the mounting destruction caused by floods in climate-vulnerable regions, we propose Street to Cloud, a machine learning pipeline for incorporating crowdsourced ground truth data into the segmentation of satellite imagery of floods. We propose this approach as a solution to the labor-intensive task of generating high-quality, hand-labeled training data, and demonstrate successes and failures of different plausible crowdsourcing approaches in our model. Street to Cloud leverages community reporting and machine learning to generate novel, near-real time insights into the extent of floo…
The ability to navigate from visual observations in unfamiliar environments is a core component of intelligent agents and an ongoing challenge for Deep Reinforcement Learning (RL). Street View can be a sensible testbed for such RL agents, because it provides real-world photographic imagery at ground level, with diverse street appearances; it has been made into an interactive environment called StreetLearn and used for research on navigation. However, goal-driven street navigation agents have not so far been able to transfer to unseen areas without extensive retraining, and relying on simulatio…
Department of Information and Computer Science（信息与计算机科学系）；King Fahd University of Petroleum and Minerals（国王法赫德石油与矿物大学）；SDAIA-KFUPM Joint Research Center of Artificial Intelligence（SDAIA-KFUPM人工智能联合研究中心）；Ontario Tech University（安大略技术大学）；University of Canberra（堪培拉大学）；University of Regina（里贾纳大学）
Walkable to Whom? Capturing Subjective Variability in Walkability Perception Using Multimodal Deep Learning
School of Civil and Environmental Engineering, University of New South Wales (UNSW)（新南威尔士大学土木与环境工程学院）；Research Centre for Integrated Transport Innovation (rCITI)（综合交通创新研究中心）
TrajGANR: Trajectory-Centric Urban Multimodal Learning via Geospatially Aligned Neural Representations
Google Research, Mountain View, CA（谷歌研究，山景城，加利福尼亚州）；Google LLC, Mountain View, CA（谷歌公司，山景城，加利福尼亚州）；Dept. of Computer Science, University of Southern California, Los Angeles, CA（计算机科学系，南加州大学，洛杉矶，加利福尼亚州）；SEAI Lab, Dept. of Geography and the Environment, The University of Texas at Austin, Austin, TX（SEAI实验室，地理与环境系，德克萨斯大学奥斯汀分校，奥斯汀，德克萨斯州）
