Recent advances in text-to-image generative models provide the ability to generate high-quality images from short text descriptions. These foundation models, when pre-trained on billion-scale datasets, are effective for various downstream tasks with little or no further training. A natural question to ask is how such models may be adapted for image compression. We investigate several techniques in which the pre-trained models can be directly used to implement compression schemes targeting novel low rate regimes. We show how text descriptions can be used in conjunction with side information to generate high-fidelity reconstructions that preserve both semantics and spatial structure of the original. We demonstrate that at very low bit-rates, our method can significantly improve upon learned compressors in terms of perceptual and semantic fidelity, despite no end-to-end training.
Block Modulating Video Compression: An Ultra Low Complexity Image Compression Encoder for Resource Limited Platforms
Boston University(波士顿大学);Chinese Academy of Sciences(中国科学院);University of Chinese Academy of Sciences(中国科学院大学);New Jersey Institute of Technology(新泽西理工学院);Weill Cornell Medicine(威尔康奈尔医学院);Beijing University of Posts and Telecommunications(北京邮电大学);Bell Labs(贝尔实验室);Westlake University(西湖大学)
Generative Latent Coding for Ultra-Low Bitrate Image and Video Compression
MoE Key Laboratory of Brain-inspired Intelligent Perception and Cognition, University of Science and Technology of China(脑启发式智能感知与认知联合实验室,中国科学技术大学);Microsoft Research Asia(微软亚洲研究院)
SDGIC: A Semantic Disambiguation-Guided Generative Image Compression Method for Ultra-Low Bitrates
School of Information and Communications Engineering, Xi’an Jiaotong University(西安交通大学信息与通信工程学院);Department of Electrical and Computer Engineering, Faculty of Applied Science, The University of British Columbia(不列颠哥伦比亚大学应用科学学院电气与计算机工程系)
MISC: Ultra-low Bitrate Image Semantic Compression Driven by Large Multimodal Model
Institute of Image Communication and Network Engineering, Shanghai Jiao Tong University(上海交通大学图像通信与网络工程研究所);John Hopcroft Center, Shanghai Jiao Tong University(上海交通大学约翰·霍普克罗夫特中心);S-Lab, Nanyang Technological University(南洋理工大学S-Lab)
Exploiting Semantic and Pixel Representations for Ultra-Low Bitrate Image Compression
National Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University(人机混合增强智能国家重点实验室,人工智能与机器人研究院,西安交通大学);School of Information and Telecommunication, Xi’an Jiaotong University(信息与电信学院,西安交通大学);Department of Computer Science and Software Engineering, The University of Western Australia(计算机科学与软件工程系,西澳大学)
UltraSketchLLM: Sub-1-Bit LLM Compression via Sketch and Hardware-Friendly Operators
National Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University(国家多媒体信息处理重点实验室,计算机科学学院,北京大学);School of Electronic Engineering and Computer Science, Peking University(电子工程与计算机科学学院,北京大学);Center for Energy-efficient Computing and Applications, Peking University(能效计算与应用中心,北京大学)
In Situ Training of Implicit Neural Compressors for Scientific Simulations via Sketch-Based Regularization
Applied Mathematics, University of Washington, Seattle(华盛顿大学应用数学系,西雅图);Applied Mathematics, University of Colorado, Boulder(科罗拉多大学应用数学系,伯尔德);Ann and H.J. Smead Department of Aerospace Engineering Sciences, University of Colorado, Boulder(科罗拉多大学伯尔德分校安与H.J. 塞梅德航空航天工程科学系)
LadderEdit: Edit-Level Residual Compression for Memory-Efficient Lifelong Editing of LLMs
Yale University(耶鲁大学);Washington University in St. Louis(圣路易斯华盛顿大学);Icahn School of Medicine at Mount Sinai(西奈山伊坎医学院);Arizona State University(亚利桑那州立大学)
Nearest Neighbor Sampling of Point Sets using Rays
The University of Texas at Austin(德克萨斯大学奥斯汀分校);Oden Institute for Computational Engineering and Sciences(奥登计算工程与科学研究所);University of British Columbia(不列颠哥伦比亚大学)
SuperGen: An Efficient Ultra-high-resolution Video Generation System with Sketching and Tiling
Rice University(里士满大学);Carnegie Mellon University(卡内基梅隆大学);University of Illinois Urbana Champaign(伊利诺伊大学厄巴纳-香槟分校);Texas A&M University(德克萨斯农工大学);The University of Texas at Austin(德克萨斯大学奥斯汀分校);Cisco(思科公司);NVIDIA(英伟达公司);University of Washington(华盛顿大学)