Innovation: An Almost Characterization of Hallucination
School of Technology and Computer Science, Tata Institute of Fundamental Research, Mumbai, Maharashtra - 400 005, India(技术与计算机科学学院,塔塔基础研究机构,孟买,马哈拉施特拉邦 - 400 005, 印度)
Hallucination is a central limitation of large language models (LLMs), and substantial effort has been devoted to understanding and mitigating it. Towards this, Kalai and Vempala (STOC 2024) introduced a probabilistic framework formalizing calibration and hallucination, and showed that, with high probability, calibrated LLMs hallucinate roughly at the rate of the "missing mass", a measure of how incomplete the training data is relative to its source. This raises two fundamental questions: (i) what property of a calibrated LLM makes hallucinations unavoidable? and (ii) can hallucinations be avoided by giving up calibration? We answer these questions by introducing a simpler property we call innovation that measures the tendency of a model to produce outputs outside the training data. We show that innovation is implied by the condition for hallucination identified by Kalai and Vempala, and, further, that it is an almost characterization of hallucination: hallucination implies innovation, and conversely, innovation implies hallucination with high probability. We also provide lower bounds on the hallucination rate based on the "innovation rate", and by relating innovation rate back to missing mass, we obtain new hallucination rate lower bounds based on missing mass that extend the results of Kalai and Vempala.
Transforming the Voice of the Customer: Large Language Models for Identifying Customer Needs
Kellogg School of Management, Northwestern University(西北大学凯洛格管理学院);MIT Management School, Massachusetts Institute of Technology(麻省理工学院斯隆管理学院);Applied Marketing Science, Inc.(应用营销科学公司)
Teaching LLMs String Matching, Backtracking, and Error Recovery to Deduce Bases and Truth Tables for the Combinatorially Exploding Bit Manipulation Puzzles
Large Language Models for Forecasting and Anomaly Detection: A Systematic Literature Review
AT&T Center for Virtualization(AT&T虚拟化中心);Southern Methodist University(南卫理公会大学);The University of Texas at Austin(德克萨斯大学奥斯汀分校);Ohio State University(俄亥俄州立大学);Universidad Internacional Isabel I de Castilla(卡斯蒂利亚伊莎贝尔一世国际大学);University of Florida(佛罗里达大学);Peking University(北京大学);Carnegie Mellon University(卡内基梅隆大学);University of Southern California(南加州大学)
Factuality Challenges in the Era of Large Language Models
University of Copenhagen(哥本哈根大学);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);Korea Advanced Institute of Science and Technology(韩国科学技术院);Indian Institute of Technology Delhi(德里印度理工学院);University of Maryland(马里兰大学);Full Fact;Stanford University(斯坦福大学);University of Southern California(南加州大学);University of Oxford(牛津大学);Meta AI;Carnegie Mellon University(卡内基梅隆大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Indiana University(印第安纳大学);Newtrales;University of Wisconsin(威斯康星大学);Pagella Politica/Facta
AdaTok: Adaptive Token Compression with Object-Aware Representations for Efficient Multimodal LLMs
Institute of Medical Technology, Peking University Health Science Center(北京大学医学部医学技术研究所);Department of Biomedical Engineering, Peking University(北京大学生物医学工程系);National Biomedical Imaging Center, Peking University(北京大学国家生物医学成像中心)