Many adversarial attacks in NLP perturb inputs to produce visually similar strings ('ergo' $\rightarrow$ '$ε$rgo') which are legible to humans but degrade model performance. Although preserving legibility is a necessary condition for text perturbation, little work has been done to systematically characterize it; instead, legibility is typically loosely enforced via intuitions around the nature and extent of perturbations. Particularly, it is unclear to what extent can inputs be perturbed while preserving legibility, or how to quantify the legibility of a perturbed string. In this work, we address this gap by learning models that predict the legibility of a perturbed string, and rank candidate perturbations based on their legibility. To do so, we collect and release LEGIT, a human-annotated dataset comprising the legibility of visually perturbed text. Using this dataset, we build both text- and vision-based models which achieve up to $0.91$ F1 score in predicting whether an input is legible, and an accuracy of $0.86$ in predicting which of two given perturbations is more legible. Additionally, we discover that legible perturbations from the LEGIT dataset are more effective at lowering the performance of NLP models than best-known attack strategies, suggesting that current models may be vulnerable to a broad range of perturbations beyond what is captured by existing visual attacks. Data, code, and models are available at https://github.com/dvsth/learning-legibility-2023.
Transcending Adversarial Perturbations: Manifold-Aided Adversarial Examples with Legitimate Semantics
College of Information Science and Engineering, Northeastern University(东北大学信息科学与工程学院);Foshan Graduate School, Northeastern University(东北大学佛山研究生院);College of Control Science and Engineering, Zhejiang University(浙江大学控制科学与工程学院)
How Robust is OCR-Reasoning? Evaluating OCR-Reasoning Robustness of Vision-Language Models under Visual Perturbations
School of Artificial Intelligence, Jilin University(吉林大学人工智能学院);Engineering Research Center of Knowledge-Driven Human-Machine Intelligence, MOE, China(教育部知识驱动人机智能工程研究中心);International Center of Future Science, Jilin University(吉林大学未来科学国际合作中心)
Evaluating the Diagnostic Robustness of Vision-Language Models Under Visual and Textual Perturbations
University of Tehran(德黑兰大学);Tehran University of Medical Sciences(德黑兰医科大学);Advanced Diagnostic and Interventional Radiology Research Center (ADIR)(高级诊断与介入放射学研究中心(ADIR))
Mitigating Video Hallucinations via Attention-Guided Representation Perturbation
School of Computer Science and Engineering, Beihang University(北京航空航天大学计算机科学与工程学院);Beijing Institute for General Artificial Intelligence, BIGAI(北京通用人工智能研究院);Sino-French Engineer School, Beihang University(北京航空航天大学中法工程师学院)
Durham University(杜伦大学);Aberystwyth University(阿伯里斯特威斯大学);UC San Diego(加州大学圣迭戈分校);University of Manchester(曼彻斯特大学);Hong Kong University of Science and Technology(香港科技大学);University of Sheffield(谢菲尔德大学)