This paper presents a simple and cost-effective method for synthesizing data to train question-answering systems. For training, fine-tuning GPT models is a common practice in resource-rich languages like English, however, it becomes challenging for non-English languages due to the scarcity of sufficient question-answer (QA) pairs. Existing approaches use question and answer generators trained on human-authored QA pairs, which involves substantial human expenses. In contrast, we use an instruct-tuned model to generate QA pairs in a zero-shot or few-shot manner. We conduct experiments to compare various strategies for obtaining QA pairs from the instruct-tuned model. The results demonstrate that a model trained on our proposed synthetic data achieves comparable performance to a model trained on manually curated datasets, without incurring human costs.
Multi-hop Knowledge Base Question Answering(KBQA) aims to find the answer entity in a knowledge base which is several hops from the topic entity mentioned in the question. Existing Retrieval-based approaches first generate instructions from the question and then use them to guide the multi-hop reasoning on the knowledge graph. As the instructions are fixed during the whole reasoning procedure and the knowledge graph is not considered in instruction generation, the model cannot revise its mistake once it predicts an intermediate entity incorrectly. To handle this, we propose KBIGER(Knowledge Ba…
Knowledge Graph Question Answering (KGQA) involves retrieving entities as answers from a Knowledge Graph (KG) using natural language queries. The challenge is to learn to reason over question-relevant KG facts that traverse KG entities and lead to the question answers. To facilitate reasoning, the question is decoded into instructions, which are dense question representations used to guide the KG traversals. However, if the derived instructions do not exactly match the underlying KG information, they may lead to reasoning under irrelevant context. Our method, termed ReaRev, introduces a new wa…
Releasing the CRaQAn (Coreference Resolution in Question-Answering): An open-source dataset and dataset creation methodology using instruction-following models
We provide a solution for elementary science test using instructional materials. We posit that there is a hidden structure that explains the correctness of an answer given the question and instructional materials and present a unified max-margin framework that learns to find these hidden structures (given a corpus of question-answer pairs and instructional materials), and uses what it learns to answer novel elementary science questions. Our evaluation shows that our framework outperforms several strong baselines.
Table Question Answering (TQA) is an important but under-explored task. Most of the existing QA datasets are in unstructured text format and only few of them use tables as the context. To the best of our knowledge, none of TQA datasets exist in the biomedical domain where tables are frequently used to present information. In this paper, we first curate a table question answering dataset, BioTABQA, using 22 templates and the context from a biomedical textbook on differential diagnosis. BioTABQA can not only be used to teach a model how to answer questions from tables but also evaluate how a mod…
Research Training Group AIPHES（AIPHES研究培训组）；Institute for Computational Linguistics（计算语言学研究所）；Heidelberg University（海德堡大学）；Qatar Computing Research Institute, HBKU（哈马德·本·哈利法大学卡塔尔计算研究所）
We describe our system for finding good answers in a community forum, as defined in SemEval-2016, Task 3 on Community Question Answering. Our approach relies on several semantic similarity features based on fine-tuned word embeddings and topics similarities. In the main Subtask C, our primary submission was ranked third, with a MAP of 51.68 and accuracy of 69.94. In Subtask A, our primary submission was also third, with MAP of 77.58 and accuracy of 73.39.
University of Florida（佛罗里达大学）；Adobe Research（奥多比研究院）
Despite the number of currently available datasets on video question answering, there still remains a need for a dataset involving multi-step and non-factoid answers. Moreover, relying on video transcripts remains an under-explored topic. To adequately address this, We propose a new question answering task on instructional videos, because of their verbose and narrative nature. While previous studies on video question answering have focused on generating a short text as an answer, given a question and video clip, our task aims to identify a span of a video segment as an answer which contains in…
PaniniQA: Enhancing Patient Education Through Interactive Question Answering
University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）；Emory University（埃默里大学）；Northeastern University（东北大学）；UMass Chan Medical School（马萨诸塞大学陈医学院）；University of Minnesota（明尼苏达大学）；University of Massachusetts Lowell（马萨诸塞大学洛厄尔分校）
University of Washington（华盛顿大学）；Allen Institute for AI（艾伦人工智能研究所）
We conduct a large empirical evaluation to investigate the landscape of distributional robustness in question answering. Our investigation spans over 350 models and 16 question answering datasets, including a diverse set of architectures, model sizes, and adaptation methods (e.g., fine-tuning, adapter tuning, in-context learning, etc.). We find that, in many cases, model variations do not affect robustness and in-distribution performance alone determines out-of-distribution performance. Moreover, our findings indicate that i) zero-shot and in-context learning methods are more robust to distrib…
Lister Hill National Center for Biomedical Communications（利斯特·希尔国家生物医学通信中心）；National Library of Medicine（国家医学图书馆）；National Institutes of Health（美国国立卫生研究院）
This paper introduces a new challenge and datasets to foster research toward designing systems that can understand medical videos and provide visual answers to natural language questions. We believe medical videos may provide the best possible answers to many first aids, medical emergency, and medical education questions. Toward this, we created the MedVidCL and MedVidQA datasets and introduce the tasks of Medical Video Classification (MVC) and Medical Visual Answer Localization (MVAL), two tasks that focus on cross-modal (medical language and medical video) understanding. The proposed tasks a…
Overview of the NLPCC 2025 Shared Task 4: Multi-modal, Multilingual, and Multi-hop Medical Instructional Video Question Answering Challenge
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（深圳先进技术研究所，中国科学院）；School of Computer Science and Technology, Beijing Institute of Technology（计算机科学与技术学院，北京理工大学）；School of Engineering, Westlake University（工程学院，西湖大学）
Optimizing Medical Question-Answering Systems: A Comparative Study of Fine-Tuned and Zero-Shot Large Language Models with RAG Framework
Department of Electrical Engineering；Computer Science University of Toledo Toledo, USA；Institute of Mathematical Sciences Claremont Graduate University Claremont, USA；Department of Bioengineering University of Toledo Toledo, USA；Department of Computer Science Bowling Green State University Bowling Green, USA
Tel-Aviv University（特拉维夫大学）；Allen Institute for AI（艾伦人工智能研究所）
Large pre-trained language models (LMs) have been shown to perform surprisingly well when fine-tuned on tasks that require commonsense and world knowledge. However, in end-to-end architectures, it is difficult to explain what is the knowledge in the LM that allows it to make a correct prediction. In this work, we propose a model for multi-choice question answering, where a LM-based generator generates a textual hypothesis that is later used by a classifier to answer the question. The hypothesis provides a window into the information used by the fine-tuned LM that can be inspected by humans. A …
GIFT: Guided Fine-Tuning and Transfer for Enhancing Instruction-Tuned Language Models
Southern University of Science and Technology（南方科技大学）；Alibaba Group（阿里巴巴集团）；Shanghai University of Finance and Economics（上海财经大学）；Tsinghua University（清华大学）；Xiamen University（厦门大学）
From Templates to Natural Language: Generalization Challenges in Instruction-Tuned LLMs for Spatial Reasoning
Computational Linguistics, Department of Linguistics University of Potsdam（乌特雷赫特大学语言学系计算语言学部）；German Research Center for Artificial Intelligence (DFKI), Berlin（德国人工智能研究中心（DFKI）柏林）
Building Instruction-Tuning Datasets from Human-Written Instructions with Open-Weight Large Language Models
Department of Computer Science, School of Computing, Institute of Science Tokyo（计算机科学系、计算学院、科学东京研究所）；National Institute of Advanced Industrial Science and Technology（国家先进工业科学与技术研究院）；Research and Development Center for Large Language Models, NII（大型语言模型研究与开发中心、日本科学振兴会）
Evaluating Evidence Grounding Under User Pressure in Instruction-Tuned Language Models
College of Information Sciences and Technology, Pennsylvania State University（宾夕法尼亚州立大学信息科学与技术学院）；Department of Computer Science, Old Dominion University（老 Dominion 大学计算机科学系）
Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；Nara Institute of Science and Technology（奈良先端科学技术大学院大学）
Text structuralization is one of the important fields of natural language processing (NLP) consists of information extraction (IE) and structure formalization. However, current studies of text structuralization suffer from a shortage of manually annotated high-quality datasets from different domains and languages, which require specialized professional knowledge. In addition, most IE methods are designed for a specific type of structured data, e.g., entities, relations, and events, making them hard to generalize to others. In this work, we propose a simple and efficient approach to instruct la…
Merging Triggers, Breaking Backdoors: Defensive Poisoning for Instruction-Tuned Language Models
Graduate School of Artificial Intelligence, POSTECH, Republic of Korea（POSTECH人工智能研究生院，韩国）；Department of Computer Science and Engineering, POSTECH, Republic of Korea（POSTECH计算机科学与工程系，韩国）
