Long-context question answering (QA) over literary texts poses significant challenges for modern large language models, particularly in low-resource languages. We address the scarcity of long-context QA resources for Indic languages by introducing LittiChoQA, the largest literary QA dataset to date covering many languages spoken in the Gangetic plains of India. The dataset comprises over 270K automatically generated question-answer pairs with a balanced distribution of factoid and non-factoid questions, generated from naturally authored literary texts collected from the open web. We evaluate multiple multilingual LLMs on non-factoid, abstractive QA, under both full-context and context-shortened settings. Results demonstrate a clear trade-off between performance and efficiency: full-context fine-tuning yields the highest token-level and semantic-level scores, while context shortening substantially improves throughput. Among the evaluated models, Krutrim-2 achieves the strongest performance, obtaining a semantic score of 76.1 with full context. While, in shortened context settings it scores 74.9 with answer paragraph selection and 71.4 with vector-based retrieval. Qualitative evaluations further corroborate these findings.
LiTransProQA: an LLM-based Literary Translation evaluation metric with Professional Question Answering
Natural Language Learning Group (NLLG) School of Business Informatics and Mathematics University of Mannheim（曼海姆大学商业信息学与数学学院自然语言学习组）；University of Aberdeen, Department of Computing Science（阿伯丁大学计算机科学系）；University of Gent, Department of Translation, Interpreting and Communication（根特大学翻译、口译与传播系）；University of Technology Nuremberg (UTN), Department Engineering（纽伦堡技术大学工程系）；Natural Language Learning and Generation (NLLG) Lab（自然语言学习与生成（NLLG）实验室）
Instituto de Física da UFRGS（巴西南大河州联邦大学物理研究所）；Instituto de Informática da UFRGS（巴西南大河州联邦大学信息学研究所）；University of Oxford（牛津大学）；Coventry University（考文垂大学）
