Ultrasound (US) is widely used for its advantages of real-time imaging, radiation-free and portability. In clinical practice, analysis and diagnosis often rely on US sequences rather than a single image to obtain dynamic anatomical information. This is challenging for novices to learn because practicing with adequate videos from patients is clinically unpractical. In this paper, we propose a novel framework to synthesize high-fidelity US videos. Specifically, the synthesis videos are generated by animating source content images based on the motion of given driving videos. Our highlights are three-fold. First, leveraging the advantages of self- and fully-supervised learning, our proposed system is trained in weakly-supervised manner for keypoint detection. These keypoints then provide vital information for handling complex high dynamic motions in US videos. Second, we decouple content and texture learning using the dual decoders to effectively reduce the model learning difficulty. Last, we adopt the adversarial training strategy with GAN losses for further improving the sharpness of the generated videos, narrowing the gap between real and synthesis videos. We validate our method on a large in-house pelvic dataset with high dynamic motion. Extensive evaluation metrics and user study prove the effectiveness of our proposed method.
UltraDfeGAN: Detail-Enhancing Generative Adversarial Networks for High-Fidelity Functional Ultrasound Synthesis
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Department of Information and Communication, Institute of Technology of Cambodia（柬埔寨技术学院信息与通信系）；Department of Telecommunication and Network Engineering, Institute of Technology of Cambodia（柬埔寨技术学院电信与网络工程系）
Carnegie Mellon University（卡内基梅隆大学）；Louisiana State University（路易斯安那州立大学）；Cosmeticsurg.net, LLC（Cosmeticsurg.net有限责任公司）
Contemporary artificial neural networks (ANN) are trained end-to-end, jointly learning both features and classifiers for the task of interest. Though enormously effective, this paradigm imposes significant costs in assembling annotated task-specific datasets and training large-scale networks. We propose to decouple feature learning from downstream lung ultrasound tasks by introducing an auxiliary pre-task of visual biomarker classification. We demonstrate that one can learn an informative, concise, and interpretable feature space from ultrasound videos by training models for predicting biomark…
Robustly Optimized Deep Feature Decoupling Network for Fatty Liver Diseases Detection
School of Computing and Artificial Intelligence, Southwest Jiaotong University（西南交通大学计算机与人工智能学院）；Department of Computer and Information Technology, Purdue University（普渡大学计算机与信息技术系）；School of Computer Science, Chengdu University of Information Technology（成都信息工程大学计算机学院）；University at Albany, State University of New York（纽约州立大学奥尔巴尼分校）
High-Fidelity Functional Ultrasound Reconstruction via A Visual Auto-Regressive Framework
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（深圳先进技术研究院，中国科学院）；Department of Computer and Information Science, University of Macau（计算机与信息科学系，澳门大学）
Massachusetts General Hospital（麻省总医院）；Massachusetts Institute of Technology（麻省理工学院）
Modern deep learning algorithms geared towards clinical adaption rely on a significant amount of high fidelity labeled data. Low-resource settings pose challenges like acquiring high fidelity data and becomes the bottleneck for developing artificial intelligence applications. Ultrasound images, stored in Digital Imaging and Communication in Medicine (DICOM) format, have additional metadata data corresponding to ultrasound image parameters and medical exams. In this work, we leverage DICOM metadata from ultrasound images to help learn representations of the ultrasound image. We demonstrate that…
MoSA: Motion-Coherent Human Video Generation via Structure-Appearance Decoupling
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University（多媒体信息处理国家重点实验室，计算机学院，北京大学）；School of Arts（艺术学院）；Li Auto（力汽车）；Harbin Institute of Technology（哈尔滨工业大学）
MultiCrafter: High-Fidelity Multi-Subject Generation via Disentangled Attention and Identity-Aware Preference Alignment
College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；School of Software Technology, Zhejiang University（浙江大学软件技术学院）；Huawei Technologies Ltd（华为技术有限公司）
Convolutional neural networks (CNNs) have attracted a rapidly growing interest in a variety of different processing tasks in the medical ultrasound community. However, the performance of CNNs is highly reliant on both the amount and fidelity of the training data. Therefore, scarce data is almost always a concern, particularly in the medical field, where clinical data is not easily accessible. The utilization of synthetic data is a popular approach to address this challenge. However, but simulating a large number of images using packages such as Field II is time-consuming, and the distribution …
School of Computer and Communication Sciences, EPFL（洛桑联邦理工学院计算机与通信科学学院）
Classic image-restoration algorithms use a variety of priors, either implicitly or explicitly. Their priors are hand-designed and their corresponding weights are heuristically assigned. Hence, deep learning methods often produce superior image restoration quality. Deep networks are, however, capable of inducing strong and hardly predictable hallucinations. Networks implicitly learn to be jointly faithful to the observed data while learning an image prior; and the separation of original data and hallucinated data downstream is then not possible. This limits their wide-spread adoption in image r…
Pyramid Self-Contrastive Learning for Single-shot Test-time Ultrasound Image Denoising
Department of Electrical and Computer Engineering, The University of Hong Kong（香港大学电子与计算机工程系）；Department of Biomedical Engineering, Duke University（达特茅斯大学生物医学工程系）
CDSA-Net:Collaborative Decoupling of Vascular Structure and Background for High-Fidelity Coronary Digital Subtraction Angiography
School of Artificial Intelligence and Digital Economy Industry, Guangzhou Institute of Science and Technology（人工智能与数字经济学院，广州科学技术研究院）；Department of Cardiology, Second Affiliated Hospital of Nanchang University（南昌大学第二附属医院心内科）；School of Artificial Intelligence, Guangzhou Institute of Science and Technology（人工智能学院，广州科学技术研究院）
Automated Fetal Biometry Assessment with Deep Ensembles using Sparse-Sampling of 2D Intrapartum Ultrasound Images
Oxford Machine Learning in NeuroImaging Lab, Department of Computer Science, University of Oxford（牛津大学计算机科学系）；Visual Geometry Group, Department of Engineering Science, University of Oxford（牛津大学工程科学系）；College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算科学与数据科学学院）
Zero-shot High-fidelity and Pose-controllable Character Animation
Fudan University（复旦大学）；Shanghai Key Lab of Intell. Info. Processing（上海智能信息处理重点实验室）；School of CS, Fudan University（复旦大学计算机科学技术学院）；Shanghai Collaborative Innovation Center of Intelligent Visual Computing（上海智能视觉计算协同创新中心）；OPPO AI Center（OPPO人工智能中心）；South China University Of Technology（华南理工大学）；Westlake University（西湖大学）
We present a novel face reconstruction method capable of reconstructing detailed face geometry, spatially varying face reflectance from a single monocular image. We build our work upon the recent advances of DNN-based auto-encoders with differentiable ray tracing image formation, trained in self-supervised manner. While providing the advantage of learning-based approaches and real-time reconstruction, the latter methods lacked fidelity. In this work, we achieve, for the first time, high fidelity face reconstruction using self-supervised learning only. Our novel coarse-to-fine deep architecture…
Hierarchical Corpus-View-Category Refinement for Carotid Plaque Risk Grading in Ultrasound
National-Regional Key Technology Engineering Laboratory for Medical Ultrasound（医学超声关键技术工程实验室）；School of Biomedical Engineering, Medical School, Shenzhen University（生物医学工程学院）；College of Computer Science and Software Engineering, Shenzhen University（计算机科学与软件工程学院）；Fuwai Shenzhen Hospital, Chinese Academy of Medical Sciences（阜外深圳医院，中国医学科学院）；Shenzhen RayShape Medical Technology Co., Ltd（深圳RayShape医疗科技有限公司）；Faculty of Applied Sciences, Macao Polytechnic University（应用科学学院）
Ultrasound Tomography of Musculoskeletal Tissues with Generative Neural Physics
College of Future Technology, Peking University（未来技术学院，北京大学）；National Biomedical Imaging Center, Peking University（北京大学国家生物医学成像中心）；Department of Mathematical Sciences, Tsinghua University（清华大学数学科学系）；School of Physical Sciences, University of Chinese Academy of Sciences（中国科学院大学物理科学学院）；School of EECE, University of Chinese Academy of Sciences（中国科学院大学电子工程与计算机科学学院）；Department of Orthopedics, Peking University Third Hospital（北京大学第三医院骨科部）；Department of Ultrasound, Peking University Third Hospital（北京大学第三医院超声科）；Department of Orthopaedic Surgery, The Third Hospital of Hebei Medical University（河北医科大学第三医院骨科部）；Yau Mathematical Sciences Center, Tsinghua University（清华大学姚期智数学科学中心）；Yanqi Lake Beijing Institute of Mathematical Sciences and Applications（北京燕琦湖数学科学与应用研究院）
DiDE:Direct Injection with Color-Texture DEcoupling for 3D Stylization
Computer Vision Center, Universitat Autònoma de Barcelona（巴塞罗那自治大学计算机视觉中心）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Jilin University（吉林大学）；City University of Hong Kong (Dongguan)（香港城市大学（东莞））
SkelEM: Training-Signal Decoupling of Skeleton and Diffusion for Self-supervised Axial Super-Resolution in Volume Microscopy
School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences, China（中国科学院大学先进交叉学科学院）；State Key Laboratory of Brain Cognition and Brain-inspired Intelligence Technology, Institute of Automation, Chinese Academy of Sciences, China（中国科学院脑认知与脑启发智能技术国家重点实验室）；School of Future Technology, University of Chinese Academy of Sciences, China（中国科学院大学未来技术学院）；School of Artificial Intelligence, University of Chinese Academy of Sciences, China（中国科学院大学人工智能学院）
