Stable Diffusion Models (SDMs) have shown remarkable proficiency in image synthesis. However, their broad application is impeded by their large model sizes and intensive computational requirements, which typically require expensive cloud servers for deployment. On the flip side, while there are many compact models tailored for edge devices that can reduce these demands, they often compromise on semantic integrity and visual quality when compared to full-sized SDMs. To bridge this gap, we introduce Hybrid SD, an innovative, training-free SDMs inference framework designed for edge-cloud collaborative inference. Hybrid SD distributes the early steps of the diffusion process to the large models deployed on cloud servers, enhancing semantic planning. Furthermore, small efficient models deployed on edge devices can be integrated for refining visual details in the later stages. Acknowledging the diversity of edge devices with differing computational and storage capacities, we employ structural pruning to the SDMs U-Net and train a lightweight VAE. Empirical evaluations demonstrate that our compressed models achieve state-of-the-art parameter efficiency (225.8M) on edge devices with competitive image quality. Additionally, Hybrid SD reduces the cloud cost by 66% with edge-cloud collaborative inference.
Cloud-Device Collaborative Learning for Multimodal Large Language Models
National Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University(北京大学计算机学院多媒体信息处理全国重点实验室);Shanghai AI Lab(上海人工智能实验室);Nanjing University(南京大学)
SCORE: SubDistribution-aware Collaborative Knowledge Reinforcing for Cloth-Hybrid Lifelong Person Re-Identification
Wangxuan Institute of Computer Technology, Peking University(北京大学王选计算机研究所);University of Chinese Academy of Sciences(中国科学院大学);Intelligent Science & Technology Academy of CASIC(中国航天科工集团智能科技研究院)
Enhancing Scene Classification in Cloudy Image Scenarios: A Collaborative Transfer Method with Information Regulation Mechanism using Optical Cloud-Covered and SAR Remote Sensing Images
School of Geosciences and Info-Physics, Central South University(中南大学地球科学与信息物理学院);Faculty of Geo-Information Science and Earth Observation (ITC), University of Twente(特文特大学地球信息科学与地球观测学院)
A Survey on Video Analytics in Cloud-Edge-Terminal Collaborative Systems
Fudan University(复旦大学);Duke Kunshan University(昆山杜克大学);University of Toronto(多伦多大学);Imperial College London(伦敦帝国学院);University of California, Davis(加利福尼亚大学戴维斯分校);SMBU;University of Ottawa(渥太华大学)
HybridVLA: Collaborative Diffusion and Autoregression in a Unified Vision-Language-Action Model
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University(多媒体信息处理国家重点实验室,计算机学院,北京大学);Beijing Academy of Artificial Intelligence (BAAI)(北京人工智能研究院);CUHK(香港中文大学)
DUET: A Tuning-Free Device-Cloud Collaborative Parameters Generation Framework for Efficient Device Model Generalization
Zhejiang University(浙江大学);Alibaba Group(阿里巴巴集团);National University of Singapore(新加坡国立大学);Shanghai Institute for Advanced Study of Zhejiang University(浙江大学上海高等研究院);Shanghai AI Laboratory(上海人工智能实验室)