The field of Math-NLP has witnessed significant growth in recent years, motivated by the desire to expand LLM performance to the learning of non-linguistic notions (numerals, and subsequently, arithmetic reasoning). However, non-linguistic skill injection typically comes at a cost for LLMs: it leads to catastrophic forgetting of core linguistic skills, a consequence that often remains unaddressed in the literature. As Math-NLP has been able to create LLMs that can closely approximate the mathematical skills of a grade-schooler or the arithmetic reasoning skills of a calculator, the practicality of these models fail if they concomitantly shed their linguistic capabilities. In this work, we take a closer look into the phenomena of catastrophic forgetting as it pertains to LLMs and subsequently offer a novel framework for non-linguistic skill injection for LLMs based on information theoretic interventions and skill-specific losses that enable the learning of strict arithmetic reasoning. Our model outperforms the state-of-the-art both on injected non-linguistic skills and on linguistic knowledge retention, and does so with a fraction of the non-linguistic training data (1/4) and zero additional synthetic linguistic training data.
Towards Ontology-Based Descriptions of Conversations with Qualitatively-Defined Concepts
Loria, CNRS, Université de Lorraine, Nancy, France(Loria研究所、法国国家科学研究中心、洛林大学、南希分校);Université Sorbonne Paris Nord, CNRS, Laboratoire d'Informatique de Paris Nord, LIPN, F-93430 Villetaneuse, France(巴黎北大学、法国国家科学研究中心、巴黎北信息技术实验室、LIPN、法国)
ViLMA: A Zero-Shot Benchmark for Linguistic and Temporal Grounding in Video-Language Models
Koç University(科奇大学);KUIS AI Center(KUIS人工智能中心);University of Pisa(比萨大学);Hacettepe University(哈塞特佩大学);Aselsan Research(阿塞尔桑研究);University of Malta(马耳他大学);Heidelberg University(海德堡大学);Amsterdam UMC, University of Amsterdam(阿姆斯特丹大学医学中心);Utrecht University(乌得勒支大学)
Putting GPT-4o to the Sword: A Comprehensive Evaluation of Language, Vision, Speech, and Multimodal Proficiency
University of Guelph(圭尔夫大学);University of North Texas(北德克萨斯大学);Iowa State University(爱荷华州立大学);Zayed University(扎耶德大学);University of Texas at Dallas(德克萨斯大学达拉斯分校);National University of Computer & Emerging Sciences(国立计算机与新兴科学大学)
Unveiling Linguistic Regions in Large Language Models
School of Computer Science, Fudan University(复旦大学计算机科学技术学院);Institute of Modern Languages and Linguistics, Fudan University(复旦大学现代语言与语言学研究所);Shanghai Collaborative Innovation Center of Intelligent Visual Computing(上海智能视觉计算协同创新中心)
Automated Speech Scoring System Under The Lens: Evaluating and interpreting the linguistic cues for language proficiency
Columbia University(哥伦比亚大学);Second Language Testing, Inc.(第二语言测试公司);Indraprastha Institute of Information Technology, Delhi(德里印多尔信息科技学院);SUNY at Buffalo(布法罗纽约州立大学);Adobe Media Data Science Research(奥多比媒体数据科学研究中心)
Empowering Persian LLMs for Instruction Following: A Novel Dataset and Training Approach
Isfahan University of Technology(伊斯法罕理工大学);Shahid Bahonar University of Kerman(克尔曼沙希德·巴霍纳尔大学);University of Tehran(德黑兰大学);Hunter College, City University of New York(纽约市立大学亨特学院)
Beyond Transcription: Unified Audio Schema for Perception-Aware AudioLLMs
Basic Model Technology Center, WeChat AI, Tencent Inc.(腾讯基本模型技术中心、微信AI、腾讯公司);State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University(多媒体信息处理国家重点实验室、计算机学院、北京大学)
Knowledge Distillation and Dataset Distillation of Large Language Models: Emerging Trends, Challenges, and Future Directions
University of Georgia(佐治亚大学);The University of Texas at Arlington(德克萨斯大学阿灵顿分校);Harvard University(哈佛大学);Carnegie Mellon University(卡内基梅隆大学);Vanderbilt University(范德堡大学);Mayo Clinic Arizona(亚利桑那梅奥诊所);Augusta University(奥古斯塔大学)
Almieyar-Oryx-BloomBench: A Bilingual Multimodal Benchmark for Cognitively Informed Evaluation of Vision-Language Models
University of British Columbia(不列颠哥伦比亚大学);Zuse School(Zuse学校);Qatar Computing Research Institute (QCRI)(卡塔尔计算研究所);Hamad Bin Khalifa University(哈马德·本·哈利法大学)
Prompting Multilingual Large Language Models to Generate Code-Mixed Texts: The Case of South East Asian Languages
Brown University(布朗大学);UC Berkeley(加州大学伯克利分校);University of California, Los Angeles(加州大学洛杉矶分校);HKUST(香港科技大学);Bloomberg(彭博);EleutherAI;Samsung R&D Institute Philippines(三星菲律宾研发中心);Stanford University(斯坦福大学);National University of Singapore(新加坡国立大学);VietAI Research(VietAI研究院);University of Potsdam(波茨坦大学);MBZUAI(穆罕默德·本·扎耶德人工智能大学)
Small Models are LLM Knowledge Triggers on Medical Tabular Prediction
College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院);Thrust of Artificial Intelligence, Information Hub, HKUST (GZ)(香港科技大学(广州)信息枢纽人工智能方向);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);The Second Affiliated Hospital Zhejiang University School of Medicine(浙江大学医学院附属第二医院);Zhejiang Key Laboratory of Medical Imaging Artificial Intelligence(浙江省医学影像人工智能重点实验室)