Advancing ALS Applications with Large-Scale Pre-training: Dataset Development and Downstream Assessment
organization= National Institute of Advanced Industrial Science;Technology (AIST) Artificial Intelligence Research Center , city= Tokyo , country= Japan
The pre-training and fine-tuning paradigm has revolutionized satellite remote sensing applications. However, this approach remains largely underexplored for airborne laser scanning (ALS), an important technology for applications such as forest management and urban planning. In this study, we address this gap by constructing a large-scale ALS point cloud dataset and evaluating its impact on downstream applications. Our dataset comprises ALS point clouds collected across the contiguous United States, provided by the United States Geological Survey's 3D Elevation Program. To ensure efficient data collection while capturing diverse land cover and terrain types, we introduce a geospatial sampling method that selects point cloud tiles based on land cover maps and digital elevation models. As a baseline self-supervised learning model, we adopt BEV-MAE, a state-of-the-art masked autoencoder for 3D outdoor point clouds, and pre-train it on the constructed dataset. The pre-trained models are subsequently fine-tuned for downstream tasks, including tree species classification, terrain scene recognition, and point cloud semantic segmentation. Our results show that the pre-trained models significantly outperform their scratch counterparts across all downstream tasks, demonstrating the transferability of the representations learned from the proposed dataset. Furthermore, we observe that scaling the dataset using our geospatial sampling method consistently enhances performance, whereas pre-training on datasets constructed with random sampling fails to achieve similar improvements. These findings highlight the utility of the constructed dataset and the effectiveness of our sampling strategy in the pre-training and fine-tuning paradigm. The source code and pre-trained models will be made publicly available at \url{https://github.com/martianxiu/ALS_pretraining}.
CoRe-ECG: Advancing Self-Supervised Representation Learning for 12-Lead ECG via Contrastive and Reconstructive Synergy
Department of Electronic Engineering, Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院电子工程系);Department of Electronic Engineering, Tsinghua University(清华大学电子工程系);Shanghai AI Laboratory(上海人工智能实验室);Department of Cardiology, Beijing Tsinghua Changgung Hospital, School of Clinical Medicine, Tsinghua Medicine, Tsinghua University(清华大学临床医学院清华大学医学部北京清华长庚医院心内科);Department of Anesthesiology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College(中国医学科学院北京协和医学院国家癌症中心国家肿瘤临床医学研究中心肿瘤医院麻醉科);Department of Electrocardiographic Diagnosis, Fuzhou University Affiliated Provincial Hospital(福州大学附属省立医院心电图诊断科)
Advancing Prompt Learning through an External Layer
Shanghai Jiaotong University(上海交通大学);University of Science and Technology of China(中国科学技术大学);Zhejiang University(浙江大学);Defense Innovation Institute(国防科技创新研究院)
ClinicalFMamba: Advancing Clinical Assessment using Mamba-based Multimodal Neuroimaging Fusion
TD Bank Group, Toronto, Canada(TD银行集团,加拿大);Department of Computer Science, University of Toronto, Toronto, Canada(多伦多大学计算机科学系,加拿大);Department of Medical Imaging, University of Toronto, Toronto, Canada(多伦多大学医学影像系,加拿大)
OmniQuality-R: Advancing Reward Models Through All-Encompassing Quality Assessment
University of Science and Technology of China(中国科学技术大学);Nanyang Technological University(南洋理工大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Peking University(北京大学);Fudan University(复旦大学)
T3D: Advancing 3D Medical Vision-Language Pre-training by Learning Multi-View Visual Consistency
Imperial College London(帝国理工学院);University of Oxford(牛津大学);Hong Kong University of Science and Technology(香港科技大学);Peking University(北京大学);University of Science and Technology of China(中国科学技术大学)
CMEdataset Advancing China Map Detection and Standardization with Digital Image Resources
Shandong Computer Science Center (National Supercomputer Center in Jinan), Qilu University of Technology (Shandong Academy of Sciences)(齐鲁工业大学(山东省科学院)山东省计算中心(国家超级计算济南中心));Shandong Fundamental Research Center for Computer Science(山东省计算机科学基础研究中心)
ASAP: Advancing Medical Volumetric Representation Learning with Anatomy-aware Semantically-adaptive Pre-training
School of Biomedical Engineering, Division of Life Sciences and Medicine, University of Science and Technology of China(生物医学工程学院,生命科学与医学系,中国科学技术大学);Medical Imaging, Robotics, Analytic Computing & Learning (MIRACLE) Lab, YRD-RIGHT, USTC Suzhou Institute for Advanced Research(医学影像、机器人、分析计算与学习(MIRACLE)实验室,YRD-RIGHT,中国科学技术大学苏州研究院);Jiangsu Provincial Key Laboratory of Multimodal Digital Twin Technology(江苏省多模态数字孪生技术重点实验室);Biomedical Basic Research Center (BBRC) of Jiangsu Province(江苏省生物医学基础研究中心);Department of Radiology, The First Affiliated Hospital of USTC, Division of Life Sciences and Medicine, USTC(放射科,中国科学技术大学第一附属医院,生命科学与医学系,中国科学技术大学);Anhui IFLYTEK CO., Ltd(安徽科大讯飞股份有限公司);School of Medicine, Stanford University(医学院,斯坦福大学);State Key Laboratory of Precision and Intelligent Chemistry, Hefei, Anhui, China(安徽省精密与智能化学重点实验室,合肥,安徽,中国)