Data plays a fundamental role in training Large Language Models (LLMs). Efficient data management, particularly in formulating a well-suited training dataset, is significant for enhancing model performance and improving training efficiency during pretraining and supervised fine-tuning stages. Despite the considerable importance of data management, the underlying mechanism of current prominent practices are still unknown. Consequently, the exploration of data management has attracted more and more attention among the research community. This survey aims to provide a comprehensive overview of current research in data management within both the pretraining and supervised fine-tuning stages of LLMs, covering various aspects of data management strategy design. Looking into the future, we extrapolate existing challenges and outline promising directions for development in this field. Therefore, this survey serves as a guiding resource for practitioners aspiring to construct powerful LLMs through efficient data management practices. The collection of the latest papers is available at https://github.com/ZigeW/data_management_LLM.
A Survey on Large Language Model Acceleration based on KV Cache Management
Department of Computing, The Hong Kong Polytechnic University(计算系,香港理工大学);Department of Computer Science and Engineering(计算机科学与工程系);Department of Computer Science and Technology(计算机科学与技术系);Department of Computing and Data Science(计算与数据科学系)
Trajectory Data Management and Mining: A Survey from Deep Learning to the LLM Era
Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));The University of Melbourne(墨尔本大学);Zhejiang University(浙江大学);Ocean University of China(中国海洋大学);Beijing Jiaotong University(北京交通大学);Squirrel AI(松鼠AI);Chongqing University(重庆大学);University of Electronic Science and Technology of China(电子科技大学);Hong Kong University of Science and Technology(香港科技大学)
How Indian Dermatologists are Utilizing Artificial Intelligence for Clinical Practice and Workflow Management: A Nationwide Survey with a Special Focus on atopic dermatitis
Charnock Hospital, Kolkata, India(科钦医院,加尔各答,印度);Department of Dermatology, Jagannath Gupta Institute of Medical Sciences and Hospital, Kolkata, India(皮肤科部,贾亚纳塔·古普塔医学科学与医院,加尔各答,印度);Department of Pediatric Dermatology, Institute of Child Health, Kolkata-700017, India(儿科皮肤科部,儿童健康研究所,加尔各答-700017,印度);Department of Dermatology, Venereology, and Leprology, Postgraduate Institute of Medical Education and Research, Chandigarh-160012, India(皮肤科、性病学和麻风病学部,医学教育与研究研究生院,昌迪加尔-160012,印度);Department of Dermatology and STD, University College of Medical Sciences & GTBH, University of Delhi, Delhi-110095, India(皮肤科和性传播疾病部,医学科学大学及GTBH,德里大学,德里-110095,印度);Department of DVL, Sree Balaji Medical College and Hospital, Chennai, India(DVL部,Sree Balaji医学院和医院,钦奈,印度)
Location reference recognition from texts: A survey and comparison
German Aerospace Center (DLR)(德国航空航天中心);University of Zurich(苏黎世大学);University of Heidelberg(海德堡大学);University at Buffalo(布法罗大学);Chongqing University(重庆大学);Norwegian University of Science and Technology(挪威科技大学)