The rapid development of Large Language Models (LLMs) demonstrates remarkable multilingual capabilities in natural language processing, attracting global attention in both academia and industry. To mitigate potential discrimination and enhance the overall usability and accessibility for diverse language user groups, it is important for the development of language-fair technology. Despite the breakthroughs of LLMs, the investigation into the multilingual scenario remains insufficient, where a comprehensive survey to summarize recent approaches, developments, limitations, and potential solutions is desirable. To this end, we provide a survey with multiple perspectives on the utilization of LLMs in the multilingual scenario. We first rethink the transitions between previous and current research on pre-trained language models. Then we introduce several perspectives on the multilingualism of LLMs, including training and inference methods, information retrieval, model security, multi-domain with language culture, and usage of datasets. We also discuss the major challenges that arise in these aspects, along with possible solutions. Besides, we highlight future research directions that aim at further enhancing LLMs with multilingualism. The survey aims to help the research community address multilingual problems and provide a comprehensive understanding of the core concepts, key techniques, and latest developments in multilingual natural language processing based on LLMs.
A Comprehensive Survey on Relation Extraction: Recent Advances and New Frontiers
The Chinese University of Hong Kong(香港中文大学);Singapore Management University(新加坡管理大学);Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院);Huazhong University of Science and Technology(华中科技大学);Sun Yat-Sen University(中山大学);Harbin Institute of Technology (Shenzhen)(哈尔滨工业大学(深圳))
Multilingual Encoder Knows more than You Realize: Shared Weights Pretraining for Extremely Low-Resource Languages
Key Laboratory of Ethnic Language Intelligent Analysis and Security Governance of MOE(民族语言智能分析与安全治理重点实验室);Minzu University of China(民族大学);Shanghai Jiao Tong University(上海交通大学)
Advances and Frontiers of LLM-based Issue Resolution in Software Engineering: A Comprehensive Survey
Sun Yat-sen University(中山大学);Hangzhou Normal University(杭州师范大学);Zhejiang University(浙江大学);Huawei Technologies Co, Ltd(华为技术有限公司);Chongqing University(重庆大学)
Exploring the Frontier of Vision-Language Models: A Survey of Current Methodologies and Future Directions
Department of Computer Science and Engineering, IIT Patna, India(印度帕纳大学计算机科学与工程系);Stanford University(斯坦福大学);Amazon AI(亚马逊人工智能);Indian Institute of Technology Patna, India(印度帕纳大学)
Shaping Schema via Language Representation as the Next Frontier for LLM Intelligence Expanding
The Hong Kong University of Science and Technology(香港理工大学);MiLM Plus, Xiaomi Inc(小米公司);Zhongguancun Academy(中关村学院);Hong Kong Baptist University(香港 Baptist大学);The University of Tokyo(东京大学);RIKEN Center for Advanced Intelligence Project(日本理化学研究院高级智能项目中心);Xi’an Jiaotong University(西安交通大学)
Evaluating Frontier LLMs on PhD-Level Mathematical Reasoning: A Benchmark on a Textbook in Theoretical Computer Science about Randomized Algorithms
Wyoming Seminary(怀俄明神学院);San Jose State University(圣何塞州立大学);Guilin University of Electronic Technology(桂林电子科技大学);University of California, Berkeley(加州大学伯克利分校);Northeastern University(东北大学);Guangdong University of Technology(广东工业大学)
Sparks of Artificial General Intelligence(AGI) in Semiconductor Material Science: Early Explorations into the Next Frontier of Generative AI-Assisted Electron Micrograph Analysis