This work shows that it is possible to fool/attack recent state-of-the-art face detectors which are based on the single-stage networks. Successfully attacking face detectors could be a serious malware vulnerability when deploying a smart surveillance system utilizing face detectors. We show that existing adversarial perturbation methods are not effective to perform such an attack, especially when there are multiple faces in the input image. This is because the adversarial perturbation specifically generated for one face may disrupt the adversarial perturbation for another face. In this paper, we call this problem the Instance Perturbation Interference (IPI) problem. This IPI problem is addressed by studying the relationship between the deep neural network receptive field and the adversarial perturbation. As such, we propose the Localized Instance Perturbation (LIP) that uses adversarial perturbation constrained to the Effective Receptive Field (ERF) of a target to perform the attack. Experiment results show the LIP method massively outperforms existing adversarial perturbation generation methods -- often by a factor of 2 to 10.
Bridge to Non-Barrier Communication: Gloss-Prompted Fine-grained Cued Speech Gesture Generation with Diffusion Model
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Tencent AI Lab(腾讯AI实验室);The Hong Kong University of Science and Technology(香港科技大学)
HyperLips: Hyper Control Lips with High Resolution Decoder for Talking Face Generation
University of Electronic Science and Technology of China(电子科技大学);ChengDu Sobey Digital Technology Co., Ltd(成都索贝数码科技股份有限公司);Peng Cheng Laboratory(鹏城实验室)
Can We Read Speech Beyond the Lips? Rethinking RoI Selection for Deep Visual Speech Recognition
Key Laboratory of Intelligent Information Processing of Chinese Academy of Sciences(中国科学院智能信息处理重点实验室);Institute of Computing Technology, CAS(中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学)
SelfTalk: A Self-Supervised Commutative Training Diagram to Comprehend 3D Talking Faces
Renmin University of China(中国人民大学);Imperial College London(伦敦帝国理工学院);Psyche AI Inc.(Psyche AI公司);Hong Kong University of Science and Technology(香港科技大学);Chinese Academy of Sciences(中国科学院);Tsinghua University(清华大学)
Three-Dimensional Lip Motion Network for Text-Independent Speaker Recognition
College of Intelligence and Computing, Tianjin University(天津大学智能与计算学部);Tianjin Key Laboratory of Cognitive Computing and Application(天津市认知计算与应用重点实验室);Institute of Linguistics, Chinese Academy of Social Sciences(中国社会科学院语言研究所);Shenzhen Research Institute of Big Data, the chinese university of hong kong shenzhen(香港中文大学(深圳)深圳大数据研究院)
GSmoothFace: Generalized Smooth Talking Face Generation via Fine Grained 3D Face Guidance
The Chinese University of Hong Kong (Shenzhen)(香港中文大学(深圳));The Future Network of Intelligence Institute(未来智能网络研究院);School of Science and Engineering(理工学院);Sun Yat-sen University(中山大学);Shenzhen University(深圳大学);Health Science Center(健康科学中心);South China Hospital(华南医院)
Learning Visual Speech Representations via Cross-Modal Distillation and Joint Face-Lip Modeling
School of Artificial Intelligence and Computer Science, Shaanxi Normal University(陕西师范大学人工智能与计算机科学学院);iFLYTEK Research, iFLYTEK Co., Ltd.(科大讯飞股份有限公司);Key Laboratory of Intelligent Computing and Service Technology for Folk SongMinistry of Culture and Tourism(文化和旅游部民歌智能计算与服务技术重点实验室)
Assessing Identity Leakage in Talking Face Generation: Metrics and Evaluation Framework
Karlsruhe Institute of Technology(卡尔斯鲁厄理工学院);KIT Campus Transfer GmbH (KCT)(KIT校园转移公司);Istanbul Technical University(伊斯坦布尔技术大学);Carnegie Mellon University(卡内基梅隆大学)
Seeing What You Said: Talking Face Generation Guided by a Lip Reading Expert
University of Electronic Science and Technology of China(电子科技大学);National University of Singapore(新加坡国立大学);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));University of Science and Technology Beijing(北京科技大学)
IP-Adapter Is All You Need: Towards Fine-Tuning-Free Diffusion-Based Talking Face Generation
Information Engineering University(信息工程大学);Huai’an University(淮安大学);Chongqing University of Post and Telecommunications(重庆邮电大学);Nankai University(南开大学)
LawDNet: Enhanced Audio-Driven Lip Synthesis via Local Affine Warping Deformation
Communication University of China(中国传媒大学);Imperial College London(帝国理工学院);Hong Kong University(香港大学);Queen Mary University of London(伦敦大学玛丽皇后学院);Beijing University of Posts and Telecommunications(北京邮电大学);Beihang University(北京航空航天大学)
Learn2Talk: 3D Talking Face Learns from 2D Talking Face
School of Informatics, Xiamen University(厦门大学信息学院);China Mobile (Hangzhou) Information Technology Co., Ltd.(中国移动(杭州)信息技术有限公司);City University of Hong Kong(香港城市大学)
PC-Talk: Precise Facial Animation Control for Audio-Driven Talking Face Generation
MAIS, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所MAIS部);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Psyche AI.INC(Psyche AI公司);HKUST(香港科技大学);CAIR, HKISI, Chinese Academy of Sciences(中国科学院计算智能研究所);SCSE, FIE, M.U.S.T(M.U.S.T的SCSE、FIE部门)
SPEAK: Speech-Driven Pose and Emotion-Adjustable Talking Head Generation
Ping An Technology(平安科技);Southeast University(东南大学);Sun Yat-sen University(中山大学);Nanjing University of Science and Technology(南京理工大学);East China University of Science and Technology(华东理工大学);University of Shanghai for Science and Technology(上海理工大学)
Makeup like a superstar: Deep Localized Makeup Transfer Network
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);Huazhong University of Science and Technology(华中科技大学);Yunnan Open University(云南开放大学);University of Electronic Science and Technology of China(电子科技大学)
S^3D-NeRF: Single-Shot Speech-Driven Neural Radiance Field for High Fidelity Talking Head Synthesis
School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Alibaba Group(阿里巴巴集团)
SegTalker: Segmentation-based Talking Face Generation with Mask-guided Local Editing
South China University of Technology(华南理工大学);Zhejiang University(浙江大学);Huawei Cloud Computing Technologies Co., Ltd(华为云计算技术有限公司);Peking University(北京大学);Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ)(广东省人工智能与数字经济实验室(深圳))