Urban air pollution has become a major environmental problem that threatens public health. It has become increasingly important to infer fine-grained urban air quality based on existing monitoring stations. One of the challenges is how to effectively select some relevant stations for air quality inference. In this paper, we propose a novel model based on reinforcement learning for urban air quality inference. The model consists of two modules: a station selector and an air quality regressor. The station selector dynamically selects the most relevant monitoring stations when inferring air quality. The air quality regressor takes in the selected stations and makes air quality inference with deep neural network. We conduct experiments on a real-world air quality dataset and our approach achieves the highest performance compared with several popular solutions, and the experiments show significant effectiveness of proposed model in tackling problems of air quality inference.
University of Maryland（马里兰大学）；University of Tehran（德黑兰大学）；Sharif University of Technology（谢里夫理工大学）
Urban air mobility is the new mode of transportation aiming to provide a fast and secure way of travel by utilizing the low-altitude airspace. This goal cannot be achieved without the implementation of new flight regulations which can assure safe and efficient allocation of flight paths to a large number of vertical takeoff/landing aerial vehicles. Such rules should also allow estimating the effective capacity of the low-altitude airspace for planning purposes. Path planning is a vital subject in urban air mobility which could enable a large number of UAVs to fly simultaneously in the airspace…
Self-organized free-flight arrival for urban air mobility
Technische Universität Dresden（德累斯顿工业大学）；Center for Scalable Data Analytics and Artificial Intelligence (ScaDS.AI)（可扩展数据分析与人工智能中心（ScaDS.AI））；Institute of Flight Systems, Universität der Bundeswehr München（慕尼黑联邦国防军大学飞行系统研究所）
Inspired by expert evaluation policy for urban perception, we proposed a novel inverse reinforcement learning (IRL) based framework for predicting urban safety and recovering the corresponding reward function. We also presented a scalable state representation method to model the prediction problem as a Markov decision process (MDP) and use reinforcement learning (RL) to solve the problem. Additionally, we built a dataset called SmallCity based on the crowdsourcing method to conduct the research. As far as we know, this is the first time the IRL approach has been introduced to the urban safety …
Airspace models have played an important role in the development and evaluation of aircraft collision avoidance systems for both manned and unmanned aircraft. As Urban Air Mobility (UAM) systems are being developed, we need new encounter models that are representative of their operational environment. Developing such models is challenging due to the lack of data on UAM behavior in the airspace. While previous encounter models for other aircraft types rely on large datasets to produce realistic trajectories, this paper presents an approach to encounter modeling that instead relies on expert kno…
We explore the use of deep learning and deep reinforcement learning for optimization problems in transportation. Many transportation system analysis tasks are formulated as an optimization problem - such as optimal control problems in intelligent transportation systems and long term urban planning. Often transportation models used to represent dynamics of a transportation system involve large data sets with complex input-output interactions and are difficult to use in the context of optimization. Use of deep learning metamodels can produce a lower dimensional representation of those relations …
MINES ParisTech（巴黎高等矿业学院）；PSL（巴黎文理研究大学）；Valeo Driving Assistance Research Valeo.ai（法雷奥驾驶辅助研究法雷奥人工智能）
Reinforcement Learning (RL) aims at learning an optimal behavior policy from its own experiments and not rule-based control methods. However, there is no RL algorithm yet capable of handling a task as difficult as urban driving. We present a novel technique, coined implicit affordances, to effectively leverage RL for urban driving thus including lane keeping, pedestrians and vehicles avoidance, and traffic light detection. To our knowledge we are the first to present a successful RL agent handling such a complex task especially regarding the traffic light detection. Furthermore, we have demons…
University of Illinois, Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；NASA Langley Research Center（美国国家航空航天局兰利研究中心）；Skygrid LLC（Skygrid有限责任公司）
This work considers the scheduling problem for Urban Air Mobility (UAM) vehicles travelling between origin-destination pairs with both hard and soft trip deadlines. Each route is described by a discrete probability distribution over trip completion times (or delay) and over inter-arrival times of requests (or demand) for the route along with a fixed hard or soft deadline. Soft deadlines carry a cost that is incurred when the deadline is missed. An online, safe scheduler is developed that ensures that hard deadlines are never missed, and that average cost of missing soft deadlines is minimized.…
Deep Reinforcement Learning for Urban Air Quality Management: Multi-Objective Optimization of Pollution Mitigation Booth Placement in Metropolitan Environments
School of Computer Science（计算机科学学院）；Engineering, Vellore Institute of Technology（工程学院，维洛雷理工学院）；Chennai Campus（钦奈校区）；Chennai, Tamil Nadu 600127, India（钦奈，泰米尔纳德邦600127，印度）
Stanford University（斯坦福大学）；Honda Research Institute（本田研究院）
Navigating urban environments represents a complex task for automated vehicles. They must reach their goal safely and efficiently while considering a multitude of traffic participants. We propose a modular decision making algorithm to autonomously navigate intersections, addressing challenges of existing rule-based and reinforcement learning (RL) approaches. We first present a safe RL algorithm relying on a model-checker to ensure safety guarantees. To make the decision strategy robust to perception errors and occlusions, we introduce a belief update technique using a learning based approach. …
Urban autonomous driving decision making is challenging due to complex road geometry and multi-agent interactions. Current decision making methods are mostly manually designing the driving policy, which might result in sub-optimal solutions and is expensive to develop, generalize and maintain at scale. On the other hand, with reinforcement learning (RL), a policy can be learned and improved automatically without any manual designs. However, current RL methods generally do not work well on complex urban scenarios. In this paper, we propose a framework to enable model-free deep reinforcement lea…
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Tsinghua University（清华大学）；Qingdao Academy of Intelligent Industries（青岛智能产业技术研究院）
Urban Traffic Control (UTC) plays an essential role in Intelligent Transportation System (ITS) but remains difficult. Since model-based UTC methods may not accurately describe the complex nature of traffic dynamics in all situations, model-free data-driven UTC methods, especially reinforcement learning (RL) based UTC methods, received increasing interests in the last decade. However, existing DL approaches did not propose an efficient algorithm to solve the complicated multiple intersections control problems whose state-action spaces are vast. To solve this problem, we propose a Deep Reinforce…
Real-Time Communication-Aware Ride-Sharing Route Planning for Urban Air Mobility: A Multi-Source Hybrid Attention Reinforcement Learning Approach
School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen（科学与工程学院，香港中文大学（深圳））；Xidian University State Key Laboratory of Integrated Services Network（西安电子科技大学集成服务网络国家重点实验室）；Department of Computer Science and Engineering, Kyung Hee University（庆北大学计算机科学与工程系）
Most traffic flow control algorithms address switching cycle adaptation of traffic signals and lights. This work addresses traffic flow optimisation by self-organising micro-level control combining Reinforcement Learning and rule-based agents for action selection performing long-range navigation in urban environments. I.e., vehicles represented by agents adapt their decision making for re-routing based on local environmental sensors. Agent-based modelling and simulation is used to study emergence effects on urban city traffic flows. An unified agent programming model enables simulation and dis…
It has been considered that urban air mobility (UAM), also known as drone-taxi or electrical vertical takeoff and landing (eVTOL), will play a key role in future transportation. By putting UAM into practical future transportation, several benefits can be realized, i.e., (i) the total travel time of passengers can be reduced compared to traditional transportation and (ii) there is no environmental pollution and no special labor costs to operate the system because electric batteries will be used in UAM system. However, there are various dynamic and uncertain factors in the flight environment, i.…
The University of Texas at Austin（德克萨斯大学奥斯汀分校）；University of Colorado Boulder（科罗拉多大学博尔德分校）
Rapid urbanization, increasing integration of distributed renewable energy resources, energy storage, and electric vehicles introduce new challenges for the power grid. In the US, buildings represent about 70% of the total electricity demand and demand response has the potential for reducing peaks of electricity by about 20%. Unlocking this potential requires control systems that operate on distributed systems, ideally data-driven and model-free. For this, reinforcement learning (RL) algorithms have gained increased interest in the past years. However, research in RL for demand response has be…
Autonomous driving is a challenging domain that entails multiple aspects: a vehicle should be able to drive to its destination as fast as possible while avoiding collision, obeying traffic rules and ensuring the comfort of passengers. In this paper, we present a deep learning variant of thresholded lexicographic Q-learning for the task of urban driving. Our multi-objective DQN agent learns to drive on multi-lane roads and intersections, yielding and changing lanes according to traffic rules. We also propose an extension for factored Markov Decision Processes to the DQN architecture that provid…
The multiagent-based participatory simulation features prominently in urban planning as the acquired model is considered as the hybrid system of the domain and the local knowledge. However, the key problem of generating realistic agents for particular social phenomena invariably remains. The existing models have attempted to dictate the factors involving human behavior, which appeared to be intractable. In this paper, Inverse Reinforcement Learning (IRL) is introduced to address this problem. IRL is developed for computational modeling of human behavior and has achieved great successes in robo…
Deep reinforcement learning is actively used for training autonomous car policies in a simulated driving environment. Due to the large availability of various reinforcement learning algorithms and the lack of their systematic comparison across different driving scenarios, we are unsure of which ones are more effective for training autonomous car software in single-agent as well as multi-agent driving environments. A benchmarking framework for the comparison of deep reinforcement learning in a vision-based autonomous driving will open up the possibilities for training better autonomous car driv…
Urban-Focused Multi-Task Offline Reinforcement Learning with Contrastive Data Sharing
Binghamton University（宾汉姆顿大学）；San Diego State University（圣地亚哥州立大学）；Lehigh University（里海大学）；University of Maryland, College Park（马里兰大学学院公园分校）；Worcester Polytechnic Institute（伍斯特理工学院）；Logistics and Supply Chain MultiTech R&D Centre（物流与供应链多技术研发中心）
University of California, Berkeley（加州大学伯克利分校）；Tsinghua University（清华大学）
Unlike popular modularized framework, end-to-end autonomous driving seeks to solve the perception, decision and control problems in an integrated way, which can be more adapting to new scenarios and easier to generalize at scale. However, existing end-to-end approaches are often lack of interpretability, and can only deal with simple driving tasks like lane keeping. In this paper, we propose an interpretable deep reinforcement learning method for end-to-end autonomous driving, which is able to handle complex urban scenarios. A sequential latent environment model is introduced and learned joint…
Navigation in a Three-Dimensional Urban Flow using Deep Reinforcement Learning
Department of Aerospace Engineering, University of Michigan（航空航天工程系，密歇根大学）；FLOW, Engineering Mechanics, KTH Royal Institute of Technology（FLOW，工程力学，皇家理工学院）
Dublin City University（都柏林城市大学）；University College Dublin（都柏林大学学院）
Calculating an Air Quality Index (AQI) typically uses data streams from air quality sensors deployed at fixed locations and the calculation is a real time process. If one or a number of sensors are broken or offline, then the real time AQI value cannot be computed. Estimating AQI values for some point in the future is a predictive process and uses historical AQI values to train and build models. In this work we focus on gap filling in air quality data where the task is to predict the AQI at 1, 5 and 7 days into the future. The scenario is where one or a number of air, weather and traffic senso…
Universidad Nacional de Educación a Distancia — UNED（西班牙国立远程教育大学）；Inverence
Air quality has become one of the main issues in public health and urban planning management, due to the proven adverse effects of high pollutant concentrations. Considering the mitigation measures that cities all over the world are taking in order to face frequent low air quality episodes, the capability of foreseeing future pollutant concentrations is of great importance. Through this paper, we present SOCAIRE, an operational tool based on a Bayesian and spatiotemporal ensemble of neural and statistical nested models. SOCAIRE integrates endogenous and exogenous information in order to predic…
This research focuses on predicting the demand for air taxi urban air mobility (UAM) services during different times of the day in various geographic regions of New York City using machine learning algorithms (MLAs). Several ride-related factors (such as month of the year, day of the week and time of the day) and weather-related variables (such as temperature, weather conditions and visibility) are used as predictors for four popular MLAs, namely, logistic regression, artificial neural networks, random forests, and gradient boosting. Experimental results suggest gradient boosting to consistent…
This is a relevant problem because the design of most cities prioritizes the use of motorized vehicles, which has degraded air quality in recent years, having a negative effect on urban health. Modeling, predicting, and forecasting ambient air pollution is an important way to deal with this issue because it would be helpful for decision-makers and urban city planners to understand the phenomena and to take solutions. In general, data-driven methods for modeling, predicting, and forecasting outdoor pollution requires an important amount of data, which may limit their accuracy. In order to deal …
University of Pennsylvania（宾夕法尼亚大学）；University of California, Berkeley（加州大学伯克利分校）；Oregon State University（俄勒冈州立大学）
With increasing urban population, there is global interest in Urban Air Mobility (UAM), where hundreds of autonomous Unmanned Aircraft Systems (UAS) execute missions in the airspace above cities. Unlike traditional human-in-the-loop air traffic management, UAM requires decentralized autonomous approaches that scale for an order of magnitude higher aircraft densities and are applicable to urban settings. We present Learning-to-Fly (L2F), a decentralized on-demand airborne collision avoidance framework for multiple UAS that allows them to independently plan and safely execute missions with spati…
Air pollution is a major global environmental health threat, in particular for people who live or work near pollution sources. Areas adjacent to pollution sources often have high ambient pollution concentrations, and those areas are commonly referred to as air pollution hotspots. Detecting and characterizing pollution hotspots are of great importance for air quality management, but are challenging due to the high spatial and temporal variability of air pollutants. In this work, we explore the use of mobile sensing data (i.e., air quality sensors installed on vehicles) to detect pollution hotsp…
How many stations are sufficient? Exploring the effect of urban weather station density reduction on imputation accuracy of air temperature and humidity
Biometry and Environmental System Analysis, University of Freiburg, Germany（弗赖堡大学生物计量与环境系统分析系）；Chair of Environmental Meteorology, University of Freiburg, Germany（弗赖堡大学环境气象学教授职位）
Imperial College London（帝国理工学院）；Southwest Jiaotong University（西南交通大学）；Department of Civil and Environmental Engineering, Imperial College London, UK（帝国理工学院土木与环境工程系）；School of Transportation and Logistics, Southwest Jiaotong University, China（西南交通大学交通运输与物流学院）
Mobile and ubiquitous sensing of urban air quality has received increased attention as an economically and operationally viable means to survey atmospheric environment with high spatial-temporal resolution. This paper proposes a machine learning based mobile air pollution sensing framework, called Deep-MAPS, and demonstrates its scientific and financial values in the following aspects. (1) Based on a network of fixed and mobile air quality sensors, we perform spatial inference of PM2.5 concentrations in Beijing (3,025 km2, 19 Jun-16 Jul 2018) for a spatial-temporal resolution of 1km-by-1km and…
Data Science Institute（数据科学研究所）；Imperial College London（帝国理工学院）
This paper presents an approach to improve computational fluid dynamics simulations forecasts of air pollution using deep learning. Our method, which integrates Principal Components Analysis (PCA) and adversarial training, is a way to improve the forecast skill of reduced order models obtained from the original model solution. Once the reduced-order model (ROM) is obtained via PCA, a Long Short-Term Memory network (LSTM) is adversarially trained on the ROM to make forecasts. Once trained, the adversarially trained LSTM outperforms a LSTM trained in a classical way. The study area is in London,…
Data Science Institute（数据科学研究所）；Imperial College London（伦敦帝国理工学院）；Leonardo Centre, Imperial Business School（帝国理工商学院莱昂纳多中心）；Department of Earth Science & Engineering（地球科学与工程系）
This paper presents an approach to improve the forecast of computational fluid dynamics (CFD) simulations of urban air pollution using deep learning, and most specifically adversarial training. This adversarial approach aims to reduce the divergence of the forecasts from the underlying physical model. Our two-step method integrates a Principal Components Analysis (PCA) based adversarial autoencoder (PC-AAE) with adversarial Long short-term memory (LSTM) networks. Once the reduced-order model (ROM) of the CFD solution is obtained via PCA, an adversarial autoencoder is used on the principal comp…
FuXi-Air: Urban Air Quality Forecasting Based on Emission-Meteorology-Pollutant multimodal Machine Learning
Shanghai Key Laboratory of Atmospheric Particle Pollution and Prevention（上海大气颗粒污染与防治重点实验室）；National Observations and Research Station for Wetland Ecosystems of the Yangtze Estuary（长江入海口湿地生态系统观测与研究站）；Department of Environmental Science and Engineering（环境科学与工程学院）；Shanghai Academy of AI for Science (SAIS)（上海人工智能科学研究院）；MOE Laboratory for National Development and Intelligent Governance（教育部国家发展与智能治理实验室）；Shanghai Institute for Energy and Carbon Neutrality Strategy（上海能源与碳中和战略研究院）；IRDR ICoE on Risk Interconnectivity and Governance on Weather/Climate Extremes Impact and Public Health（国际减灾与发展研究院风险互联与治理国际联合实验室）；Shanghai Institute of Eco-Chongming (SIEC)（上海生态崇明研究院）；Shanghai Environmental Monitoring Center (SEMC)（上海市环境监测中心）
COVID-19 which has spread in Iran from February 19, 2020, has infected 202,584 people and killed 9,507 people until June 20, 2020. The immediate suggested solution to prevent the spread of this virus was to avoid traveling around. In this study, the correlation between traveling between cities with new confirmed cases of COVID-19 in Iran is demonstrated. The data, used in the study, consisted of the daily inter-state traffic, air traffic data, and daily new COVID-19 confirmed cases. The data is used to train a regression model and voting was used to show the highest correlation between travels…
Low-Cost High-Order Singular Value Decomposition for Tensor-Based Reconstruction from Sparse Sensor Measurements: Urban Flow and Air-Quality Applications
ETSI Aeronáutica y del Espacio, Universidad Politécnica de Madrid（马德里理工大学航空与航天工程学院）；Ove Arup（奥雅纳）；Department of Aerospace Engineering, University of Michigan（密歇根大学航空航天工程系）
CityAQVis: Integrated ML-Visualization Sandbox Tool for Pollutant Estimation in Urban Regions Using Multi-Source Data (Software Article)
Graphics-Visualization-Computing Lab, International Institute of Information Technology Bangalore, Karnataka 560100, India（图形可视化计算实验室，国际信息学院班加罗尔，卡纳塔克邦560100，印度）
