We present MCQA, a learning-based algorithm for multimodal question answering. MCQA explicitly fuses and aligns the multimodal input (i.e. text, audio, and video), which forms the context for the query (question and answer). Our approach fuses and aligns the question and the answer within this context. Moreover, we use the notion of co-attention to perform cross-modal alignment and multimodal context-query alignment. Our context-query alignment module matches the relevant parts of the multimodal context and the query with each other and aligns them to improve the overall performance. We evaluate the performance of MCQA on Social-IQ, a benchmark dataset for multimodal question answering. We compare the performance of our algorithm with prior methods and observe an accuracy improvement of 4-7%.
Medical multiple-choice question answering (MCQA) is particularly difficult. Questions may describe patient symptoms and ask for the correct diagnosis, which requires domain knowledge and complex reasoning. Standard language modeling pretraining alone is not sufficient to achieve the best results. \citet{jin2020disease} showed that focusing masked language modeling on disease name prediction when using medical encyclopedic paragraphs as input leads to considerable MCQA accuracy improvement. In this work, we show that (1) fine-tuning on generated MCQA dataset outperforms the masked language mod…
Afri-MCQA: Multimodal Cultural Question Answering for African Languages
MBZUAI；AI4Bharat；Indian Institute of Technology, Madras（印度理工学院马德拉斯分校）；Saarland University（萨尔兰大学）；Aston University（阿斯顿大学）；Addis Ababa University（亚的斯亚贝巴大学）；Lesan AI；Friedrich-Alexander University（弗里德里希-亚历山大大学）；University of Pretoria（比勒陀利亚大学）；University of Minneosta -Twin Cities（明尼苏达大学-双城分校）；Lelapa AI；Tshwane University of Technology（茨瓦内技术大学）；Digital Umuganda；University of Botswana（博茨瓦纳大学）；Mila, McGill University & Canada CIFAR AI Chair（Mila，麦吉尔大学及加拿大CIFAR人工智能主席）
College of Electrical Engineering and Computer Science（电机工程学院与计算机科学学院）；National Taiwan University（台湾大学）
In this paper, we study the possibility of almost unsupervised Multiple Choices Question Answering (MCQA). Starting from very basic knowledge, MCQA model knows that some choices have higher probabilities of being correct than the others. The information, though very noisy, guides the training of an MCQA model. The proposed method is shown to outperform the baseline approaches on RACE and even comparable with some supervised learning approaches on MC500.
Nanjing University（南京大学）；The Ohio State University（俄亥俄州立大学）
A trending paradigm for multiple-choice question answering (MCQA) is using a text-to-text framework. By unifying data in different tasks into a single text-to-text format, it trains a generative encoder-decoder model which is both powerful and universal. However, a side effect of twisting a generation target to fit the classification nature of MCQA is the under-utilization of the decoder and the knowledge that can be decoded. To exploit the generation capability and underlying knowledge of a pre-trained encoder-decoder model, in this paper, we propose a generation-enhanced MCQA model named Gen…
Wuhan Textile University（武汉纺织大学）；University of Wollongong（伍伦贡大学）
The task of multiple choice question answering (MCQA) refers to identifying a suitable answer from multiple candidates, by estimating the matching score among the triple of the passage, question and answer. Despite the general research interest in this regard, existing methods decouple the process into several pair-wise or dual matching steps, that limited the ability of assessing cases with multiple evidence sentences. To alleviate this issue, this paper introduces a novel Context-guided Triple Matching algorithm, which is achieved by integrating a Triple Matching (TM) module and a Contrastiv…
Differentiating Choices via Commonality for Multiple-Choice Question Answering
School of Information and Communication Technology, Griffith University（格里菲斯大学信息与通信技术学院）；College of Intelligence and Computing, Tianjin University（天津大学智能与计算学院）
GRAF: Graph Retrieval Augmented by Facts for Romanian Legal Multi-Choice Question Answering
National University of Science and Technology POLITEHNICA Bucharest, Faculty of Automatic Control and Computers（波兰技术大学布加勒斯特分校国家科学与技术大学，自动控制与计算机学院）；Technical University of Munich（慕尼黑技术大学）；Paris 1 Panthéon-Sorbonne University（巴黎第一大学）；University of Bucharest（布加勒斯特大学）
Singapore Management University（新加坡管理大学）；Nanyang Technological University（南洋理工大学）；Institute for Infocomm Research, A*STAR（新加坡科技研究局资讯通信研究院）；Institute of High Performance Computing, A*STAR（新加坡科技研究局高性能计算研究院）
Multiple-Choice Question Answering (MCQA) is a challenging task in machine reading comprehension. The main challenge in MCQA is to extract "evidence" from the given context that supports the correct answer. In the OpenbookQA dataset, the requirement of extracting "evidence" is particularly important due to the mutual independence of sentences in the context. Existing work tackles this problem by annotated evidence or distant supervision with rules which overly rely on human efforts. To address the challenge, we propose a simple yet effective approach termed evidence filtering to model the rela…
The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））
The evaluation of intelligibility for TTS has reached a bottleneck, as existing assessments heavily rely on word-by-word accuracy metrics such as WER, which fail to capture the complexity of real-world speech or reflect human comprehension needs. To address this, we propose Spoken-Passage Multiple-Choice Question Answering, a novel subjective approach evaluating the accuracy of key information in synthesized speech, and release SP-MCQA-Eval, an 8.76-hour news-style benchmark dataset for SP-MCQA evaluation. Our experiments reveal that low WER does not necessarily guarantee high key-information …
Singapore University of Technology and Design（新加坡科技设计大学）；University of Michigan（密歇根大学）
We propose a simple refactoring of multi-choice question answering (MCQA) tasks as a series of binary classifications. The MCQA task is generally performed by scoring each (question, answer) pair normalized over all the pairs, and then selecting the answer from the pair that yield the highest score. For n answer choices, this is equivalent to an n-class classification setup where only one class (true answer) is correct. We instead show that classifying (question, true answer) as positive instances and (question, false answer) as negative instances is significantly more effective across various…
Improving LLM First-Token Predictions in Multiple-Choice Question Answering via Output Prefilling
University of Modena and Reggio Emilia（摩德纳大学与雷焦艾米利亚大学）；University of Pisa（比萨大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Brown University（布朗大学）；Meta
First Token Probability Guided RAG for Telecom Question Answering
Shenzhen Research Institute of Big Data（深圳大数据研究院）；School of Science and Engineering, The Chinese University of Hong Kong (Shenzhen)（香港中文大学（深圳）理工学院）；School of Computer Science and Engineering, Sun Yat-sen University（中山大学计算机科学与工程学院）
Machine Reading Comprehension (MRC) for question answering (QA), which aims to answer a question given the relevant context passages, is an important way to test the ability of intelligence systems to understand human language. Multiple-Choice QA (MCQA) is one of the most difficult tasks in MRC because it often requires more advanced reading comprehension skills such as logical reasoning, summarization, and arithmetic operations, compared to the extractive counterpart where answers are usually spans of text within given passages. Moreover, most existing MCQA datasets are small in size, making …
Correctness Coverage Evaluation for Medical Multiple-Choice Question Answering Based on the Enhanced Conformal Prediction Framework
School of Electronic and Information Engineering, Tongji University（同济大学电子与信息工程学院）；School of Electrical and Information Engineering, Hunan University（湖南大学电气与信息工程学院）；School of Economics and Management, Fuzhou University（福州市经济学与管理学院）
This paper introduces MedMCQA, a new large-scale, Multiple-Choice Question Answering (MCQA) dataset designed to address real-world medical entrance exam questions. More than 194k high-quality AIIMS \& NEET PG entrance exam MCQs covering 2.4k healthcare topics and 21 medical subjects are collected with an average token length of 12.77 and high topical diversity. Each sample contains a question, correct answer(s), and other options which requires a deeper language understanding as it tests the 10+ reasoning abilities of a model across a wide range of medical subjects \& topics. A detailed explan…
IMB: An Italian Medical Benchmark for Question Answering
University of Naples Federico II, Department of Electrical Engineering（那不勒斯费德里科二世大学电子工程系）；Consorzio Interuniversitario Nazionale per l'Informatica (CINI) - ITEM National Lab（国家信息计算联合研究中心（CINI）- ITEM国家实验室）；Northwestern University, Department of Computer Science, McCormick School of Engineering（西北大学计算机科学系，工程学院）
Promptception: How Sensitive Are Large Multimodal Models to Prompts?
Mohamed Bin Zayed University of Artificial Intelligence（莫罕默德·本·扎耶德人工智能大学）；Swiss Federal Institute of Technology Lausanne (EPFL)（洛桑联邦理工学院）；Australian National University（澳大利亚国立大学）
Multimodal Question Answering for Unified Information Extraction
The Ohio State University（俄亥俄州立大学）；College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；School of Engineering, Westlake University（西湖大学工学院）
ProMQA: Question Answering Dataset for Multimodal Procedural Activity Understanding
Language Technologies Institute, Carnegie Mellon University（语言技术研究所，卡内基梅隆大学）；National Institute of Advanced Industrial Science and Technology (AIST)（国家先进工业科学与技术研究院）
Brigham Young University（杨百翰大学）；University of Southern California（南加州大学）
While large language models (LLMs) like GPT-3 have achieved impressive results on multiple choice question answering (MCQA) tasks in the zero, one, and few-shot settings, they generally lag behind the MCQA state of the art (SOTA). MCQA tasks have traditionally been presented to LLMs like cloze tasks. An LLM is conditioned on a question (without the associated answer options) and its chosen option is the one assigned the highest probability after normalization (for length, etc.). A more natural prompting approach is to present the question and answer options to the LLM jointly and have it outpu…
MTBench: A Multimodal Time Series Benchmark for Temporal Reasoning and Question Answering
Yale University, New Haven, CT, USA（耶鲁大学）；McGill University, Montreal, QC, Canada（麦吉尔大学）；University of Texas Rio Grande Valley, TX, USA（德克萨斯大学里奥格兰德谷分校）
Hierarchical Vision-Language Reasoning for Multimodal Multiple-Choice Question Answering
State Key Laboratory for Novel Software Technology, Nanjing University（新型软件技术国家重点实验室，南京大学）；Chongqing University of Posts and Telecommunications（重庆邮电大学）
MemeMQA: Multimodal Question Answering for Memes via Rationale-Based Inferencing
Indraprastha Institute of Information Technology Delhi（因德拉普拉斯塔信息技术学院德里校区）；Indian Institute of Technology Delhi（印度理工学院德里校区）；Wipro R&D (Lab45)（威普罗研发中心（45实验室））；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
提出 MemeMQA 多模态问答框架、MemeMQACorpus 数据集与两阶段 ARSENAL 模型，用于兼具答案和理由的模因危害理解，准确率较最佳基线提升约18%。
