VLMInferSlow：评估大型视觉-语言模型作为服务的效率鲁棒性
VLMInferSlow: Evaluating the Efficiency Robustness of Large Vision-Language Models as a Service
AI总结：
提出VLMInferSlow，一种黑盒方法，通过细粒度效率建模和零阶优化生成对抗图像，评估VLM服务效率鲁棒性，计算成本最高提升128.47%。
摘要：
视觉-语言模型（VLMs）在实际应用中展现出巨大潜力。虽然现有研究主要聚焦于提升其准确性，但效率问题仍未得到充分探索。鉴于许多应用对实时性的需求以及VLM高昂的推理开销，效率鲁棒性是一个关键问题。然而，以往研究在评估效率鲁棒性时基于不切实际的假设，要求访问模型架构和参数——这在机器学习即服务（ML-as-a-service）场景中是不现实的，因为VLM通过推理API部署。为弥补这一空白，我们提出VLMInferSlow，一种在现实黑盒设置下评估VLM效率鲁棒性的新颖方法。VLMInferSlow结合了针对VLM推理定制的细粒度效率建模，并利用零阶优化搜索对抗性示例。实验结果表明，VLMInferSlow生成的对抗性图像具有难以察觉的扰动，可将计算成本最多提升128.47%。我们希望这项研究能提高社区对VLM效率鲁棒性的关注。
英文摘要：
Vision-Language Models (VLMs) have demonstrated great potential in real-world applications. While existing research primarily focuses on improving their accuracy, the efficiency remains underexplored. Given the real-time demands of many applications and the high inference overhead of VLMs, efficiency robustness is a critical issue. However, previous studies evaluate efficiency robustness under unrealistic assumptions, requiring access to the model architecture and parameters -- an impractical scenario in ML-as-a-service settings, where VLMs are deployed via inference APIs. To address this gap, we propose VLMInferSlow, a novel approach for evaluating VLM efficiency robustness in a realistic black-box setting. VLMInferSlow incorporates fine-grained efficiency modeling tailored to VLM inference and leverages zero-order optimization to search for adversarial examples. Experimental results show that VLMInferSlow generates adversarial images with imperceptible perturbations, increasing the computational cost by up to 128.47%. We hope this research raises the community's awareness about the efficiency robustness of VLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
