Propaganda campaigns have long been used to influence public opinion via disseminating biased and/or misleading information. Despite the increasing prevalence of propaganda content on the Internet, few attempts have been made by AI researchers to analyze such content. We introduce the task of multimodal propaganda processing, where the goal is to automatically analyze propaganda content. We believe that this task presents a long-term challenge to AI researchers and that successful processing of propaganda could bring machine understanding one important step closer to human understanding. We discuss the technical challenges associated with this task and outline the steps that need to be taken to address it.
Sofia University “St. Kliment Ohridski”(圣克莱门特·奥里季斯基索菲亚大学);King’s College London(伦敦国王学院);Qatar Computing Research Institute, HBKU(卡塔尔计算研究所(HBKU));Sapienza University of Rome(罗马第一大学);Facebook AI(脸书人工智能研究院);University of Padova(帕多瓦大学)
MOMENTA: A Multimodal Framework for Detecting Harmful Memes and Their Targets
Johns Hopkins University(约翰斯·霍普金斯大学);Indraprastha Institute of Information Technology - Delhi(德里英迪拉普拉斯信息技术学院);Wipro AI Labs(威普罗人工智能实验室);Sofia University(索非亚大学);Qatar Computing Research Institute, HBKU(哈马德·本·哈利法大学卡塔尔计算研究所)
网络模因已经成为传播政治、心理和社会文化思想的有力手段。虽然模因通常是幽默的,但最近几天,用于拖钓、网络欺凌和虐待的有害模因不断升级。检测这样的模因是具有挑战性的,因为它们可能是高度讽刺和神秘的。此外,虽然以前的工作集中在模因的具体方面,如仇恨言论和宣传,但很少有关于一般伤害的工作。在这里,我们的目标是弥合这一差距。我们专注于两个任务:(i)检测有害的模因,(ii)识别它们所针对的社会实体。我们进一步扩展了最近发布的HarMeme数据集,该数据集涵盖了COVID-19,增加了额外的模因和一个新的主题:美国政治。为了解决这些任务,我们提出了MOMENTA(MultiModal framework for detecting harmful MemEs aNd Their tTargets),这是一种新型的多模态深度神经网络,它使用全局和局部视角来检测有害模因。MOMENTA系统地分析了输入模因的本地和全球视角(两种模式),并将其与背景语境联系起来。MOMENTA是可解释和可推广的,我们的实验表明,它优于几个强大的竞争方法。
MemeMQA: Multimodal Question Answering for Memes via Rationale-Based Inferencing
Indraprastha Institute of Information Technology Delhi(因德拉普拉斯塔信息技术学院德里校区);Indian Institute of Technology Delhi(印度理工学院德里校区);Wipro R&D (Lab45)(威普罗研发中心(45实验室));Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学)
提出 MemeMQA 多模态问答框架、MemeMQACorpus 数据集与两阶段 ARSENAL 模型,用于兼具答案和理由的模因危害理解,准确率较最佳基线提升约18%。
Qatar Computing Research Institute, HBKU(哈马德·本·哈利法大学卡塔尔计算研究所);IIT-CNR(意大利国家研究委员会信息处理研究所);IIT Delhi(德里印度理工学院);Sapienza University of Rome(罗马第一大学);Sofia University(索非亚大学);University of Padova(帕多瓦大学);Facebook AI(脸书人工智能);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学)
A Transformer-based representation-learning model with unified processing of multimodal input for clinical diagnostics
The University of Hong Kong(香港大学);West China Hospital, Sichuan University(四川大学华西医院);Deepwise Healthcare(深睿医疗);Zhuhai People’s Hospital and the First Affiliated Hospital of Faculty of Medicine, Macau University of Science and Technology and University Hospital(澳门科技大学医学院附属第一医院、珠海市人民医院及大学医院);Peking University(北京大学);Guangzhou Laboratory(广州实验室);Jinling Hospital, Nanjing University School of Medicine(南京大学医学院附属金陵医院)
Language and Multimodal Models in Sports: A Survey of Datasets and Applications
University of California, Irvine(加利福尼亚大学欧文分校);George Mason University(乔治梅森大学);Meta Platforms, Inc.(元平台公司);Stanford University(斯坦福大学);University of California, Santa Barbara(加利福尼亚大学圣巴巴拉分校);Rice University(莱斯大学)
Research on Optimization of Natural Language Processing Model Based on Multimodal Deep Learning
Washington University in St. Louis(圣路易斯华盛顿大学);University of Southern California(南加州大学);Carnegie Mellon University(卡内基梅隆大学);Johns Hopkins University(约翰霍普金斯大学);Marquette University(马凯特大学);New York University(纽约大学)
Prediction of mortality and resource utilization in critical care: a deep learning approach using multimodal electronic health records with natural language processing techniques
Saw Swee Hock School of Public Health(Saw Swee Hock 公共卫生学院);National University of Singapore(新加坡国立大学);Institute of Data Science(数据科学研究所);Department of Anaesthesiology(麻醉科部门)
Table Understanding and (Multimodal) LLMs: A Cross-Domain Case Study on Scientific vs. Non-Scientific Data
Deutsches Forschungszentrum für Künstliche Intelligenz GmbH (DFKI)(德国人工智能研究中心有限公司);Technische Universität Berlin(柏林技术大学);BIFOLD;Deutsche Telekom(德国电信);Common Crawl Foundation(Common Crawl基金会);Humboldt-Universität zu Berlin(柏林洪堡大学)
SLAM-LLM: A Modular, Open-Source Multimodal Large Language Model Framework and Best Practice for Speech, Language, Audio and Music Processing
X-LANCE Lab, School of Computer Science, MoE Key Lab of Artificial Intelligence Shanghai Jiao Tong University(X-LANCE实验室,计算机科学学院,人工智能教育部重点实验室,上海交通大学);Tongyi Lab, Alibaba Group(通义实验室,阿里巴巴集团);Peng Cheng Laboratory(鹏城实验室);University of Texas at Austin(德克萨斯大学奥斯汀分校);Tianjin University(天津大学);Hong Kong University of Science and Technology(香港科学大学);Queen Mary University of London(伦敦玛丽女王大学);Nanyang Technological University(南洋理工大学);Shanghai Innovation Institute(上海创新研究院)