With the rise in edge-computing devices, there has been an increasing demand to deploy energy and resource-efficient models. A large body of research has been devoted to developing methods that can reduce the size of the model considerably without affecting the standard metrics such as top-1 accuracy. However, these pruning approaches tend to result in a significant mismatch in other metrics such as fairness across classes and explainability. To combat such misalignment, we propose a novel multi-part loss function inspired by the knowledge-distillation literature. Through extensive experiments, we demonstrate the effectiveness of our approach across different compression algorithms, architectures, tasks as well as datasets. In particular, we obtain up to $4.1\times$ reduction in the number of prediction mismatches between the compressed and reference models, and up to $5.7\times$ in cases where the reference model makes the correct prediction; all while making no changes to the compression algorithm, and minor modifications to the loss function. Furthermore, we demonstrate how inducing simple alignment between the predictions of the models naturally improves the alignment on other metrics including fairness and attributions. Our framework can thus serve as a simple plug-and-play component for compression algorithms in the future.
Going Beyond Feature Similarity: Effective Dataset Distillation based on Class-Aware Conditional Mutual Information
Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳));Peng Cheng Laboratory(鹏城实验室);Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);University of Waterloo(滑铁卢大学)
MR$^2$-Bench: Going Beyond Matching to Reasoning in Multimodal Retrieval
Beijing University of Posts and Telecommunications(北京邮电大学);Beijing Academy of Artificial Intelligence(北京智源人工智能研究院);University of Science and Technology of China(中国科学技术大学);Renmin University of China(中国人民大学);Baidu Inc., China(百度公司)
GOLLIC: Learning Global Context beyond Patches for Lossless High-Resolution Image Compression
Huawei Hong Kong Research Center(华为香港研究中心);The Hong Kong University of Science and Technology(香港科技大学);HKUST Shenzhen-Hong Kong Collaborative Innovation Research Institute(香港科技大学深港协同创新研究院)
Going Beyond U-Net: Assessing Vision Transformers for Semantic Segmentation in Microscopy Image Analysis
Institute of Computer Science, University of Tartu(塔尔图大学计算机科学研究所);Center for Artificial Intelligence, Technical University of Applied Sciences Würzburg-Schweinfurt(维尔茨堡-施韦因富特应用技术大学人工智能中心);STACC