arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2601.01792cs.LGcs.AIcs.CLcs.SD
收藏

HyperCLOVA X 8B Omni:首个支持文本、音频和视觉的任何到任何多模态模型

HyperCLOVA X 8B Omni

NAVER Cloud HyperCLOVA X Team(NAVER云HyperCLOVA X团队)

NAVER Cloud HyperCLOVA X Team

AI总结:

HyperCLOVA X 8B Omni是首个支持文本、音频和视觉的任何到任何多模态模型,通过统一的多模态处理实现高效且灵活的多模态交互。

摘要:

本报告中,我们介绍了HyperCLOVA X 8B Omni,这是HyperCLOVA X系列中首个支持文本、音频和视觉作为输入和输出的任何到任何多模态模型。通过将多模态理解和生成整合到单一模型中,而不是分别的模态特定流程,HyperCLOVA X 8B Omni成为了一个8B规模的多路径点,朝着实用的任何到任何多模态助手迈进。总体而言,该模型通过交错的多模态序列上的共享下一个标记预测接口来统一各种模态,同时视觉和音频编码器注入连续嵌入以实现细粒度的理解和基础。实证评估显示,在涵盖文本、音频和视觉的多种输入-输出组合中,该模型在韩语和英语中均表现出与同样规模的模型相竞争的性能。我们预计,HyperCLOVA X 8B Omni的开源权重发布将支持广泛的研究和部署场景。

英文摘要:

In this report, we present HyperCLOVA X 8B Omni, the first any-to-any omnimodal model in the HyperCLOVA X family that supports text, audio, and vision as both inputs and outputs. By consolidating multimodal understanding and generation into a single model rather than separate modality-specific pipelines, HyperCLOVA X 8B Omni serves as an 8B-scale omni-pathfinding point toward practical any-to-any omni assistants. At a high level, the model unifies modalities through a shared next-token prediction interface over an interleaved multimodal sequence, while vision and audio encoders inject continuous embeddings for fine-grained understanding and grounding. Empirical evaluations demonstrate competitive performance against comparably sized models across diverse input-output combinations spanning text, audio, and vision, in both Korean and English. We anticipate that the open-weight release of HyperCLOVA X 8B Omni will support a wide range of research and deployment scenarios.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑