With the considerable growth of linked data, researchers have focused on how to increase the availability of semantic web technologies to provide practical usages for real life systems. Question answering systems are an example of real-life systems that communicate directly with end users, understand user intention and generate answers. End users do not care about the structural query language or the vocabulary of the knowledge base where the point of a problem arises. In this study, a question answering framework that converts Turkish natural language input into SPARQL queries in the geographical domain is proposed. Additionally, a novel Turkish ontology, which covers a 10th grade geography lesson named Spatial Synthesis Turkey, has been developed to be used as a linked data provider. Moreover, a gap in the literature on Turkish question answering systems, which utilizes linked data in the geographical domain, is addressed. A hybrid system architecture that combines natural language processing techniques with linked data technologies to generate answers is also proposed. Further related research areas are suggested.
University of California Santa Barbara（加州大学圣巴巴拉分校）
As an important part of Artificial Intelligence (AI), Question Answering (QA) aims at generating answers to questions phrased in natural language. While there has been substantial progress in open-domain question answering, QA systems are still struggling to answer questions which involve geographic entities or concepts and that require spatial operations. In this paper, we discuss the problem of geographic question answering (GeoQA). We first investigate the reasons why geographic questions are difficult to answer by analyzing challenges of geographic questions. We discuss the uniqueness of g…
University of California, Santa Barbara（加州大学圣巴巴拉分校）；LinkedIn Corporation（领英公司）；SayMosaic Inc.（SayMosaic公司）
Learning knowledge graph (KG) embeddings is an emerging technique for a variety of downstream tasks such as summarization, link prediction, information retrieval, and question answering. However, most existing KG embedding models neglect space and, therefore, do not perform well when applied to (geo)spatial data and tasks. For those models that consider space, most of them primarily rely on some notions of distance. These models suffer from higher computational complexity during training while still losing information beyond the relative distance between entities. In this work, we propose a lo…
With the steady emergence of community question answering (CQA) platforms like Quora, StackExchange, and WikiHow, users now have an unprecedented access to information on various kind of queries and tasks. Moreover, the rapid proliferation and localization of these platforms spanning geographic and linguistic boundaries offer a unique opportunity to study the task requirements and preferences of users in different socio-linguistic groups. In this study, we implement an entity-embedding model trained on a large longitudinal dataset of multi-lingual and task-oriented question-answer pairs to unc…
MapIQ: Evaluating Multimodal Large Language Models for Map Question Answering
School of Computing and Augmented Intelligence（计算与增强智能学院）；Arizona State University（亚利桑那州立大学）；Department of Computer Science（计算机科学系）；International Institute of Information Technology（国际信息科技研究所）
Facebook AI Research（脸书人工智能研究院）；University College London（伦敦大学学院）
Question answering (QA) models have shown rapid progress enabled by the availability of large, high-quality benchmark datasets. Such annotated datasets are difficult and costly to collect, and rarely exist in languages other than English, making training QA systems in other languages challenging. An alternative to building large monolingual training datasets is to develop cross-lingual systems which can transfer to a target language without requiring training data in that language. In order to develop such systems, it is crucial to invest in high quality multilingual evaluation benchmarks to m…
IBM Research（IBM研究院）；Rensselaer Polytechnic Institute（伦斯勒理工学院）；IIT Bombay（印度理工学院孟买分校）
Recent advances in transformers have enabled Table Question Answering (Table QA) systems to achieve high accuracy and SOTA results on open domain datasets like WikiTableQuestions and WikiSQL. Such transformers are frequently pre-trained on open-domain content such as Wikipedia, where they effectively encode questions and corresponding tables from Wikipedia as seen in Table QA dataset. However, web tables in Wikipedia are notably flat in their layout, with the first row as the sole column header. The layout lends to a relational view of tables where each row is a tuple. Whereas, tables in domai…
Large language models (LLMs) have emerged as valuable tools for many natural language understanding tasks. In safety-critical applications such as healthcare, the utility of these models is governed by their ability to generate outputs that are factually accurate and complete. In this work, we present dialog-enabled resolving agents (DERA). DERA is a paradigm made possible by the increased conversational abilities of LLMs, namely GPT-4. It provides a simple, interpretable forum for models to communicate feedback and iteratively improve output. We frame our dialog as a discussion between two ag…
Hierarchical Question-Answering for Driving Scene Understanding Using Vision-Language Models
Center for ICT and Automotive Convergence, Kyungpook National University（信息与汽车融合研究中心，庆北国立大学）；School of Electronic and Electrical Engineering, Kyungpook National University（电子电气工程学院，庆北国立大学）
MAPWise: Evaluating Vision-Language Models for Advanced Map Queries
IIIT Hyderabad（印度信息技术学院海得拉巴分校）；University of Utah（犹他大学）；University of Delaware（特拉华大学）；Arizona State University（亚利桑那州立大学）；University of Pennsylvania（宾夕法尼亚大学）
Foundation Models for Geospatial Reasoning: Assessing Capabilities of Large Language Models in Understanding Geometries and Topological Spatial Relations
GeoDS Lab, Department of Geography, University of Wisconsin-Madison, USA（威斯康星大学麦迪逊分校地理系GeoDS实验室）；Department of Geography and Regional Research, University of Vienna, Austria（维也纳大学地理与区域研究系）；Department of Geography, University of California-Santa Barbara, USA（加州大学圣塔芭芭拉分校地理系）
Applied Artificial Intelligence Institute, Deakin University（迪肯大学应用人工智能研究所）
What does it take to design a machine that learns to answer natural questions about a video? A Video QA system must simultaneously understand language, represent visual content over space-time, and iteratively transform these representations in response to lingual content in the query, and finally arriving at a sensible answer. While recent advances in lingual and visual question answering have enabled sophisticated representations and neural reasoning mechanisms, major challenges in Video QA remain on dynamic grounding of concepts, relations and actions to support the reasoning process. Inspi…
Applying Relation Extraction and Graph Matching to Answering Multiple Choice Questions
Graduate School of Informatics, Kyoto University（京都大学信息科学研究生院）；Center for Innovative Research and Education in Data Science, Kyoto University（京都大学数据科学创新研究与教育中心）
ReGraP-LLaVA: Reasoning enabled Graph-based Personalized Large Language and Vision Assistant
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究所）；School of Engineering, Westlake University（西湖大学工程学院）；University of Minnesota – Twin Cities（明尼苏达大学双城分校）；University of Toronto（多伦多大学）
University of California-Berkeley（加州大学伯克利分校）；NEC Laboratories America（NEC美国实验室）
We present Adaptive Memory Networks (AMN) that processes input-question pairs to dynamically construct a network architecture optimized for lower inference times for Question Answering (QA) tasks. AMN processes the input story to extract entities and stores them in memory banks. Starting from a single bank, as the number of input entities increases, AMN learns to create new banks as the entropy in a single bank becomes too high. Hence, after processing an input-question(s) pair, the resulting network represents a hierarchical structure where entities are stored in different banks, distanced by…
Efficient Onboard Vision-Language Inference in UAV-Enabled Low-Altitude Economy Networks via LLM-Enhanced Optimization
College of Computing and Data Science, Nanyang Technological University, Singapore（计算与数据科学学院、新加坡国立科技大学）；The University of Sydney, Sydney, Australia（悉尼大学、澳大利亚悉尼）；Department of Electrical and Computer Engineering, Western University, London, Canada（电气与计算机工程系、西方大学、加拿大伦敦）；Department of Electrical and Computer Engineering, Sungkyunkwan University, South Korea（电气与计算机工程系、全州大学、韩国）
In the era of loT (Internet of Things) we are surrounded by a plethora of Al enabled devices that can transcribe images, video, audio, and sensors signals into text descriptions. When such transcriptions are captured in activity reports for monitoring, life logging and anomaly detection applications, a user would typically request a summary or ask targeted questions about certain sections of the report they are interested in. Depending on the context and the type of question asked, a question answering (QA) system would need to automatically determine whether the answer covers single-span or m…
Geospatial-Temporal Sensemaking of Remote Sensing Activity Detections with Multimodal Large Language Model
SenSIP Center, School of ECEE, Arizona State University（SenSIP中心，电子与计算机工程学院，亚利桑那州立大学）；Prime Solutions Group Inc（Prime Solutions Group公司）；Intelligence Advanced Research Projects Activity（智能高级研究计划局）
Coupled with the availability of large scale datasets, deep learning architectures have enabled rapid progress on the Question Answering task. However, most of those datasets are in English, and the performances of state-of-the-art multilingual models are significantly lower when evaluated on non-English data. Due to high data collection costs, it is not realistic to obtain annotated data for each language one desires to support. We propose a method to improve the Cross-lingual Question Answering performance without requiring additional annotated data, leveraging Question Generation models to …
AI systems' ability to explain their reasoning is critical to their utility and trustworthiness. Deep neural networks have enabled significant progress on many challenging problems such as visual question answering (VQA). However, most of them are opaque black boxes with limited explanatory capability. This paper presents a novel approach to developing a high-performing VQA system that can elucidate its answers with integrated textual and visual explanations that faithfully reflect important aspects of its underlying reasoning while capturing the style of comprehensible human explanations. Ext…
