We present an open source math-aware Question Answering System based on Ask Platypus. Our system returns as a single mathematical formula for a natural language question in English or Hindi. This formulae originate from the knowledge-base Wikidata. We translate these formulae to computable data by integrating the calculation engine sympy into our system. This way, users can enter numeric values for the variables occurring in the formula. Moreover, the system loads numeric values for constants occurring in the formula from Wikidata. In a user study, our system outperformed a commercial computational mathematical knowledge engine by 13%. However, the performance of our system heavily depends on the size and quality of the formula data available in Wikidata. Since only a few items in Wikidata contained formulae when we started the project, we facilitated the import process by suggesting formula edits to Wikidata editors. With the simple heuristic that the first formula is significant for the article, 80% of the suggestions were correct.
Confident RAG: Enhancing the Performance of LLMs for Mathematics Question Answering through Multi-Embedding and Confidence Scoring
Faculty of Education, The University of Hong Kong(香港大学教育学院);Department of Civil and Environmental Engineering, The Hong Kong University of Science and Technology(香港科学与技术大学土木与环境工程系)
LiveQA: A Question Answering Dataset over Sports Live
Key Laboratory of Computational Linguistics, MOE, Peking University(北京大学计算语言学教育部重点实验室);Graduate School of Informatics, Kyoto University(京都大学信息学研究生院);University of Washington(华盛顿大学)
Exploring Generative Process Reward Modeling for Semi-Structured Data: A Case Study of Table Question Answering
University of Augsburg(奥格斯堡大学);Institut für Maschinelle Sprachverarbeitung, University of Stuttgart(斯图加特大学机械语言处理研究所);Bosch Center for Artificial Intelligence(博世人工智能中心)