多模态大模型

R-AVST: Empowering Video-LLMs with Fine-Grained Spatio-Temporal Reasoning in Complex Audio-Visual Scenarios

R-AVST：通过细粒度时空推理增强视频大语言模型以应对复杂音频视觉场景

Lu Zhu, Tiantian Geng, Yangye Chen, Teng Wang, Ping Lu, Feng Zheng

专题命中 音频语音多模态 ：audio-visual(title,abstract)；multimodal(abstract)；分类 cs.CV

AI总结 R-AVST通过细粒度时空推理提升视频大语言模型，构建首个真实世界音频视觉时空推理数据集，并提出AVST-Zero模型增强推理能力。

Comments Accepted by AAAI 2026. Project page: https://github.com/zhlllau/R-AVST

LongCat-Flash-Omni Technical Report

LongCat-Flash-Omni 技术报告

Meituan LongCat Team, Bairui Wang, Bayan, Bin Xiao, Bo Zhang, Bolin Rong, Borun Chen, Chang Wan, Chao Zhang, Chen Huang, Chen Chen, Chen Chen, Chengxu Yang, Chengzuo Yang, Cong Han, Dandan Peng, Delian Ruan, Detai Xin, Disong Wang, Dongchao Yang, Fanfan Liu, Fengjiao Chen, Fengyu Yang, Gan Dong, Gang Huang, Gang Xu, Guanglu Wan, Guoqiang Tan, Guoqiao Yu, Haibo Qiu, Hao Lu, Hongbo Liu, Hongyu Xiang, Jiaheng Wu, Jian Yang, Jiaxing Liu, Jing Huang, Jingang Wang, Jinrui Ding, Juchao Jiang, Jun Kuang, Jun Wang, Junhui Mei, Ke Ding, Kefeng Zhang, Lei Chen, Liang Shi, Limeng Qiao, Liming Zheng, Lin Ma, Liuyang Guo, Liya Ma, Luying Sun, Man Gao, Mengshen Zhu, Miao Cao, Minliang Lin, Nuo Xu, Peng Shi, Qi Zhang, Qian Fang, Qian Wang, Qian Yang, Quanxiu Wang, Rongxiang Weng, Rongxin Guo, Ruoxuan Liang, Senbin Yang, Shanbo Xu, Shanglin Lei, Shengze Ye, Shimin Chen, Shuaiqi Chen, Shujie Hu, Shuo Li, Siqi Yang, Siyu Xu, Siyu Ren, Song Li, Songxiang Liu, Tianhao Bai, Tianye Dai, Wei Hong, Wei Wang, Weixiao Zhao, Wengang Cao, Wenlong Zhu, Wenlong He, Xi Su, Xi Nan, Xiaohan Zhao, Xiaohao Wang, Xiaoyu Zhao, Xiaoyu Wang, Xiaoyu Li, Xin Pan, Xin Chen, Xiusong Sun, Xu Xiang, Xudong Xing, Xuezhi Cao, Xunliang Cai, Yang Yang, Yanli Tan, Yao Yao, Yerui Sun, Yi Chen, Yifan Lu, Yin Gong, Yining Zhang, Yitian Chen, Yiyang Gan, Yuchen Tang, Yuchen Xie, Yueqian Wang, Yuewen Zheng, Yufei Zhang, Yufeng Zhong, Yulei Qian, Yuqi Peng, Yuqian Li, Yuwei Jiang, Zeyang Hu, Zheng Zhang, Zhengkun Tian, Zhiqing Hong, Zhixiong Zeng, Zhuqi Mi, Ziran Li, Ziwen Wang, Ziyi Zhao, Ziyuan Zhuang, Zizhe Zhao

机构 * Meituan LongCat Team（美团LongCat团队） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；audio-visual(abstract)；omni-modal(abstract)；分类 cs.CL、cs.AI、cs.MM

AI总结 LongCat-Flash-Omni 是一个具有5600亿参数的开源多模态模型，通过课程启发式训练策略实现高效实时音频视觉交互，具备强大的多模态和单模能力。

DistTrain: Addressing Model and Data Heterogeneity with Disaggregated Training for Multimodal Large Language Models

DistTrain: 通过解耦训练解决多模态大语言模型中的模型与数据异质性

Zili Zhang, Yinmin Zhong, Yimin Jiang, Hanpeng Hu, Jianjian Sun, Zheng Ge, Yibo Zhu, Daxin Jiang, Xin Jin

专题命中 音频语音多模态 ：multimodal(title,abstract)

AI总结 DistTrain通过解耦模型编排和数据预处理，提升多模态大语言模型的训练效率和资源利用率。

Comments SIGCOMM 2025 (https://dl.acm.org/doi/10.1145/3718958.3750472)

Journal ref SIGCOMM'25: Proceedings of the ACM SIGCOMM 2025 Conference Pages 24-38

Colin: A Multimodal Human-AI Co-Creation Storytelling System To Support Children's Multi-Level Narrative Skills

Colin：一种多模态人机协同创故事叙述系统，以支持儿童的多层级叙述技能

Lyumanshan Ye, Jiandong Jiang, Yuhan Liu, Yihan Ran, Yufan Zhou, Zhao Wang, Yipeng Yu, Pengfei Liu, Danni Chang, Yucheng Jin

专题命中 音频语音多模态 ：multimodal(title)

AI总结 Colin通过语音和视觉模态支持儿童多层级叙述技能发展，通过生成、理解和构建阶段提升儿童的叙事能力。

Layover or Direct Flight: Rethinking Audio-Guided Image Segmentation

转折或直接飞行：重新思考音频引导的图像分割

Joel Alberto Santos, Zongwei Wu, Xavier Alameda-Pineda, Radu Timofte

机构 * Computer Vision Lab, CAIDAS & IFI, University of Würzburg, Germany（计算机视觉实验室、CAIDAS与IFI、乌尔姆大学、德国） ； Inria at Univ. Grenoble Alpes, CNRS, LJK, France（Inria于格勒诺布尔阿尔卑斯大学、CNRS、LJK、法国） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；audio-visual(abstract)；分类 cs.CV

AI总结 本文提出通过直接音频-视觉对齐进行图像分割，无需依赖文本转录，提升了鲁棒性和效率。

STAR-Bench: Probing Deep Spatio-Temporal Reasoning as Audio 4D Intelligence

STAR-Bench：探测深度时空推理作为音频4D智能

Zihan Liu, Zhikang Niu, Qiuyang Xiao, Zhisheng Zheng, Ruoqi Yuan, Yuhang Zang, Yuhang Cao, Xiaoyi Dong, Jianze Liang, Xie Chen, Leilei Sun, Dahua Lin, Jiaqi Wang

机构 * Fudan University（复旦大学） ； Shanghai AI Laboratory（上海人工智能实验室） ； The Chinese University of Hong Kong（香港中文大学） ； Shanghai Innovation Institute（上海创新研究院） ； Beihang University（北京航空航天大学） ； Shanghai Jiao Tong University（上海交通大学） 纠错

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.CL、eess.AS

AI总结 STAR-Bench通过测试音频4D智能，揭示了模型在细粒度感知和推理上的不足，为未来模型发展提供方向。

Comments Homepage: https://internlm.github.io/StarBench/

LFM2 Technical Report

LFM2 技术报告

Alexander Amini, Anna Banaszak, Harold Benoit, Arthur Böök, Tarek Dakhran, Song Duong, Alfred Eng, Fernando Fernandes, Marc Härkönen, Anne Harrington, Ramin Hasani, Saniya Karwa, Yuri Khrustalev, Maxime Labonne, Mathias Lechner, Valentine Lechner, Simon Lee, Zetian Li, Noel Loo, Jacob Marks, Edoardo Mosca, Samuel J. Paech, Paul Pak, Rom N. Parnichkun, Alex Quach, Ryan Rogers, Daniela Rus, Nayan Saxena, Bettina Schlager, Tim Seyde, Jimmy T. H. Smith, Aditya Tadimeti, Neehal Tumma

机构 * Liquid AI 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.AI

AI总结 LFM2 是一种高效部署且具备强大任务能力的液体基础模型家族，通过紧凑的混合主干和优化训练流程，在多种基准测试中表现出色，支持多模态和检索变体。

Balancing Speech Understanding and Generation Using Continual Pre-training for Codec-based Speech LLM

通过持续预训练平衡基于编码器的语音LLM的语音理解和生成

Jiatong Shi, Chunlei Zhang, Jinchuan Tian, Junrui Ni, Hao Zhang, Shinji Watanabe, Dong Yu

专题命中 音频语音多模态 ：cross-modal(abstract)；分类 eess.AS

AI总结 本文提出通过持续预训练框架平衡基于编码器的语音LLM的语音理解和生成，实现端到端的语音翻译系统，无需中间转录和翻译步骤。

Comments Accepted by ASRU2025