Language Models as Continuous Self-Evolving Data Engineers
School of Computer Science and Engineering, Northeastern University(东北大学计算机科学与工程学院);China Mobile Internet Company Limited(中国移动互联网有限公司);Alibaba Group(阿里巴巴集团)
Peidong Wang, Ming Wang, Zhiming Ma, Xiaocui Yang, Shi Feng, Daling Wang, Yifei Zhang, Kaisong Song
Large Language Models (LLMs) have demonstrated remarkable capabilities on various tasks, while the further evolvement is limited to the lack of high-quality training data. In addition, traditional training approaches rely too much on expert-labeled data, setting a ceiling on the performance of LLMs. To address this issue, we propose a novel paradigm named LANCE (LANguage models as Continuous self-Evolving data engineers) that enables LLMs to train themselves by autonomously generating, cleaning, reviewing, and annotating data with preference information. Our approach demonstrates that LLMs can serve as continuous self-evolving data engineers, significantly reducing the time and cost of the post-training data construction. Through iterative fine-tuning on Qwen2 series models, we validate the effectiveness of LANCE across various tasks, showing that it can maintain high-quality data generation and continuously improve model performance. Across multiple benchmark dimensions, LANCE results in an average score enhancement of 3.64 for Qwen2-7B and 1.75 for Qwen2-7B-Instruct. This training paradigm with autonomous data construction not only reduces the reliance on human experts or external models but also ensures that the data aligns with human preferences, paving the way for the development of future superintelligent systems that can exceed human capabilities. Codes are available at: https://github.com/Control-derek/LANCE.
Few-Shot Learning for Mental Disorder Detection: A Continuous Multi-Prompt Engineering Approach with Medical Knowledge Injection
Hefei University of Technology(合肥工业大学);Iowa State University(爱荷华州立大学);University of Delaware(特拉华大学);University of Rhode Island(罗德岛大学);University of Arizona(亚利桑那大学)
Building Self-Evolving Agents via Experience-Driven Lifelong Learning: A Framework and Benchmark
School of Computer Science and Technology, East China Normal University(东华师范大学计算机科学与技术学院);Shanghai AI Laboratory(上海人工智能实验室);The Chinese University of HongKong(香港中文大学);Fudan University(复旦大学)
ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environments
Center for Research on Intelligent Perception and Computing (CRIPAC), National Laboratory of Pattern Recognition (NLPR)(中国科学院大学未来技术学院、人工智能学院;模式识别国家重点实验室智能感知与计算研究中心);School of Future Technology and School of Artificial Intelligence, University of Chinese Academy of Sciences(北京理工大学);Beijing Institute of Technology(浙江大学);Zhejiang University(澳大利亚国立大学);Australian National University