arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.03320cs.CVcs.CL

InternLM-XComposer-2.5：一个支持长上下文输入与输出的通用大型视觉语言模型

InternLM-XComposer-2.5: A Versatile Large Vision Language Model Supporting Long-Contextual Input and Output

The Chinese University of Hong Kong（香港中文大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；SenseTime Group（商汤科技集团）
Pan Zhang, Xiaoyi Dong, Yuhang Zang, Yuhang Cao, Rui Qian, Lin Chen, Qipeng Guo, Haodong Duan, Bin Wang, Linke Ouyang, Songyang Zhang, Wenwei Zhang, Yining Li, … 展开作者

Pan Zhang, Xiaoyi Dong, Yuhang Zang, Yuhang Cao, Rui Qian, Lin Chen, Qipeng Guo, Haodong Duan, Bin Wang, Linke Ouyang, Songyang Zhang, Wenwei Zhang, Yining Li, Yang Gao, Peng Sun, Xinyue Zhang, Wei Li, Jingwen Li, Wenhai Wang, Hang Yan, Conghui He, Xingcheng Zhang, Kai Chen, Jifeng Dai, Yu Qiao, Dahua Lin, Jiaqi Wang

阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

InternLM-XComposer-2.5以7B后端实现GPT-4V级能力，通过24K训练和RoPE外推支持96K长上下文，在超高分辨率理解、细粒度视频理解、多轮多图对话及图文创作上实现突破，并在16个基准上超越开源最先进模型。

摘要：

我们提出了InternLM-XComposer-2.5（IXC-2.5），这是一个支持长上下文输入与输出的通用大型视觉语言模型。IXC-2.5在各种文本-图像理解与创作应用中表现出色，仅以7B LLM后端即达到GPT-4V级别能力。该模型使用24K交错图像-文本上下文训练，并可通过RoPE外推无缝扩展至96K长上下文。这种长上下文能力使IXC-2.5能够胜任需要大量输入和输出上下文的任务。与之前的2.0版本相比，InternLM-XComposer-2.5在视觉-语言理解方面有三项重大升级：（1）超高分辨率理解，（2）细粒度视频理解，以及（3）多轮多图对话。除理解能力外，IXC-2.5还利用额外的LoRA参数扩展了两个引人注目的文本-图像创作应用：（1）构建网页和（2）撰写高质量图文文章。IXC-2.5已在28个基准上评估，在16个基准上超越现有开源最先进模型，并在16项关键任务上超越或接近GPT-4V和Gemini Pro。InternLM-XComposer-2.5已在https://github.com/InternLM/InternLM-XComposer公开提供。

英文摘要：

We present InternLM-XComposer-2.5 (IXC-2.5), a versatile large-vision language model that supports long-contextual input and output. IXC-2.5 excels in various text-image comprehension and composition applications, achieving GPT-4V level capabilities with merely 7B LLM backend. Trained with 24K interleaved image-text contexts, it can seamlessly extend to 96K long contexts via RoPE extrapolation. This long-context capability allows IXC-2.5 to excel in tasks requiring extensive input and output contexts. Compared to its previous 2.0 version, InternLM-XComposer-2.5 features three major upgrades in vision-language comprehension: (1) Ultra-High Resolution Understanding, (2) Fine-Grained Video Understanding, and (3) Multi-Turn Multi-Image Dialogue. In addition to comprehension, IXC-2.5 extends to two compelling applications using extra LoRA parameters for text-image composition: (1) Crafting Webpages and (2) Composing High-Quality Text-Image Articles. IXC-2.5 has been evaluated on 28 benchmarks, outperforming existing open-source state-of-the-art models on 16 benchmarks. It also surpasses or competes closely with GPT-4V and Gemini Pro on 16 key tasks. The InternLM-XComposer-2.5 is publicly available at https://github.com/InternLM/InternLM-XComposer.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑