Empowering Persian LLMs for Instruction Following: A Novel Dataset and Training Approach
Isfahan University of Technology(伊斯法罕理工大学);Shahid Bahonar University of Kerman(克尔曼沙希德·巴霍纳尔大学);University of Tehran(德黑兰大学);Hunter College, City University of New York(纽约市立大学亨特学院)
Hojjat Mokhtarabadi, Ziba Zamani, Abbas Maazallahi, Mohammad Hossein Manshaei
经过指令微调的大语言模型已在多个领域展现出遵循人类指令的卓越能力,但在许多低资源语言中,其性能仍存在明显不足。为应对这一挑战,我们首先推出FarsInstruct,这是一个综合性指令数据集,旨在专门提升波斯语(一种在全球范围内使用广泛却代表性不足的语言)大语言模型的指令遵循能力。FarsInstruct涵盖多种任务类型与数据集,每个数据集都包含从简单到复杂的人工编写指令,以及译自Public Pool of Prompts(公共提示池)的指令,确保了丰富的语言与文化代表性。此外,我们提出Co-CoLA框架,用于增强经LoRA(低秩适配)微调的模型的多任务适配能力。通过大量实验分析,本研究证明FarsInstruct数据集结合Co-CoLA框架训练,可有效提升波斯语场景下大语言模型的性能。截至撰写本文时,FarsInstruct包含21个不同数据集下的197个模板,我们计划持续更新该数据集,以进一步提升其适用性。
英文摘要:
Instruction-tuned large language models have demonstrated remarkable capabilities in following human instructions across various domains. However, their proficiency remains notably deficient in many low-resource languages. To address this challenge, we begin by introducing FarsInstruct a comprehensive instruction dataset designed to enhance the instruction following ability of large language models specifically for the Persian language a significant yet underrepresented language globally. FarsInstruct encompasses a wide range of task types and datasets, each containing a mix of straightforward to complex manual written instructions, as well as translations from the Public Pool of Prompts, ensuring a rich linguistic and cultural representation. Furthermore, we introduce Co-CoLA, a framework designed to enhance the multi-task adaptability of LoRA-tuned models. Through extensive experimental analyses, our study showcases the effectiveness of the FarsInstruct dataset coupled with training by the Co-CoLA framework, in improving the performance of large language models within the Persian context. As of the current writing, FarsInstruct comprises 197 templates across 21 distinct datasets, and we intend to update it consistently, thus augmenting its applicability.
PerMedCQA: Benchmarking Large Language Models on Medical Consumer Question Answering in Persian Language
School of Computer Science Institute for Research in Fundamental Sciences(计算机科学学院基础科学研究研究所);School of Electrical and Computer Engineering University of Tehran(电气与计算机工程学院塔里班大学);Department of Computer Science Faculty of Mathematical Sciences Alzahra University(计算机科学系数学科学学院阿扎赫拉大学)
ChemOrch: Empowering LLMs with Chemical Intelligence via Synthetic Instructions
Department of Computer Science and Engineering, University of Notre Dame(诺丁汉大学计算机科学与工程系);MIT(麻省理工学院);CalTech(加州理工学院);MBZUAI(穆斯林人工智能研究所);CMU(卡内基梅隆大学);Department of Chemistry & Biochemistry, University of Notre Dame(诺丁汉大学化学与生物化学系)
PersianMedQA: Evaluating Large Language Models on a Persian-English Bilingual Medical Question Answering Benchmark
School of Electrical and Computer Engineering, University of Tehran(德黑兰大学电气与计算机工程学院);Shahid Beheshti University of Medical Sciences(沙希德·贝赫什提医学科学大学);Institute for Research in Fundamental Sciences (IPM)(基础科学研究所(IPM))
PointLLM: Empowering Large Language Models to Understand Point Clouds
The Chinese University of Hong Kong(香港中文大学);Shanghai AI Laboratory(上海人工智能实验室);Zhejiang University(浙江大学);Centre for Perceptual and Interactive Intelligence(感知与交互智能中心)
MMEvol: Empowering Multimodal Large Language Models with Evol-Instruct
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院);University of Chinese Academy of Sciences(中国科学院大学);Alibaba Group(阿里巴巴集团);Tongji University(同济大学);The University of Sydney(悉尼大学)
VFA: Empowering Multilingual MLLMs via Vision-Free Adaptation
Southern University of Science and Technology(南方科技大学);Microsoft Research Asia(微软亚洲研究院);Shanghai University of Finance and Economics(上海财经大学);Peking University(北京大学)
CityGPT: Empowering Urban Spatial Cognition of Large Language Models
Department of Electronic Engineering, BNRist, Tsinghua University(电子工程系、BNRist、清华大学);School of Electronic and Information Engineering, Beijing Jiaotong University(电子信息工程学院、北京交通大学);Department of Electronic Engineering, Tsinghua University(电子工程系、清华大学);Department of Urban Planning, Tsinghua University(城市规划系、清华大学)
GraphInstruct: Empowering Large Language Models with Graph Understanding and Reasoning Capability
National Engineering Research Center for Big Data Technology and System(大数据技术与系统国家工程研究中心);System Lab, Cluster(系统实验室,集群);Grid Computing Lab, School of Computer Science(网格计算实验室,计算机科学学院);Huazhong University of Science and Technology(华中科技大学);School of Computer Science(计算机科学学院);Microsoft Research Asia, Beijing 100190, China(微软亚洲研究院,北京100190,中国)
MC-GPT: Empowering Vision-and-Language Navigation with Memory Map and Reasoning Chains
School of Intelligent Systems Engineering, Shenzhen Campus of Sun Yat-sen University(中山大学深圳校区智能系统工程学院);School of Computer Science and Engineering(计算机科学与工程学院)
针对基于LLM的VLN方法记忆构建不足、策略单一的问题,本文提出MC-GPT,通过拓扑记忆地图、Navigation Chain of Thoughts及集成流水线增强导航与推理可解释性。
WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instruct
Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);Microsoft Corporation(微软公司);Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院)
WizardMath 通过 RLEIF 方法在不依赖外部 Python 工具的情况下提升大语言模型的数学推理能力,在 GSM8k 和 MATH 基准上超越多个顶级开源及商用模型。
Advancing Tool-Augmented Large Language Models via Meta-Verification and Reflection Learning
State Key Laboratory of Cognitive Intelligence, University of Science and Technology of China(认知智能国家重点实验室,中国科学技术大学);Research Center for Social Computing and Interactive Robotics, Harbin Institute of Technology(社会计算与交互机器人研究中心,哈尔滨工业大学)
GeoX: Geometric Problem Solving Through Unified Formalized Vision-Language Pre-training
Shanghai Jiao Tong University(上海交通大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Fudan University(复旦大学);The Chinese University of Hong Kong(香港中文大学)