无人机在户外环境中实时单目场景分析
Real-Time Monocular Scene Analysis for UAV in Outdoor Environments
AI总结：
本文提出Co-SemDepth模型，用于无人机在低空非结构化环境中实时预测深度和语义图。通过合成数据集TopAir和图像风格迁移技术提升泛化能力，并在海洋领域验证模型性能。
摘要：
本文利用无人机上的单目相机，在低空非结构化环境中预测深度和语义图。我们提出Co-SemDepth联合深度学习架构，能快速准确完成两项任务，并在多种数据集上验证有效性。神经网络训练需要大量标注数据，但在无人机领域此类数据有限。本文引入新的合成数据集TopAir，包含不同海拔户外环境的俯视图像，以弥补数据缺口。虽然使用合成数据训练方便，但转换到真实领域测试时会出现问题。本文进行了广泛分析研究，评估多个因素对合成到真实泛化的影响。Co-SemDepth和TaskPrompter模型用于比较，结果表明Co-SemDepth在深度估计上表现优异，TaskPrompter在语义分割上表现更好。此外，我们分析了哪些训练数据集能带来更好的泛化效果。为缩小合成与真实领域差距，本文探索了图像风格迁移技术，使用Cycle-GAN和扩散模型进行合成到真实风格转换。结果表明扩散模型在合成到真实风格迁移中表现更好。最后，本文聚焦海洋领域，训练Co-SemDepth于收集的合成海洋数据MidSea，并在合成和真实数据上测试。结果表明，当在SMD数据集上的真实数据测试时，Co-SemDepth表现出良好的泛化性能，但在MIT数据集上仍需进一步改进。
英文摘要：
In this thesis, we leverage monocular cameras on aerial robots to predict depth and semantic maps in low-altitude unstructured environments. We propose a joint deep-learning architecture, named Co-SemDepth, that can perform the two tasks accurately and rapidly, and validate its effectiveness on a variety of datasets. The training of neural networks requires an abundance of annotated data, and in the UAV field, the availability of such data is limited. We introduce a new synthetic dataset in this thesis, TopAir that contains images captured with a nadir view in outdoor environments at different altitudes, helping to fill the gap. While using synthetic data for the training is convenient, it raises issues when shifting to the real domain for testing. We conduct an extensive analytical study to assess the effect of several factors on the synthetic-to-real generalization. Co-SemDepth and TaskPrompter models are used for comparison in this study. The results reveal a superior generalization performance for Co-SemDepth in depth estimation and for TaskPrompter in semantic segmentation. Also, our analysis allows us to determine which training datasets lead to a better generalization. Moreover, to help attenuate the gap between the synthetic and real domains, image style transfer techniques are explored on aerial images to convert from the synthetic to the realistic style. Cycle-GAN and Diffusion models are employed. The results reveal that diffusion models are better in the synthetic to real style transfer. In the end, we focus on the marine domain and address its challenges. Co-SemDepth is trained on a collected synthetic marine data, called MidSea, and tested on both synthetic and real data. The results reveal good generalization performance of Co-SemDepth when tested on real data from the SMD dataset while further enhancement is needed on the MIT dataset.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
