arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2408.13402cs.LG
收藏

LLaVaOLMoBitnet1B:三元大语言模型迈向多模态!

LLaVaOLMoBitnet1B: Ternary LLM goes Multimodal!

Intel(英特尔)

Jainaveen Sundaram, Ravi Iyer

AI总结:

本文提出首个开源三元多模态大语言模型 LLaVaOLMoBitnet1B,可在小算力平台上处理图像和文本输入,并公开训练脚本以推动该方向研究。

摘要:

多模态大语言模型(MM-LLMs)在过去一年取得了显著进展,在各类任务中展现出令人印象深刻的性能。然而,要真正实现人工智能的普及化,模型不仅需要具备强大的能力,还必须能够在大多数人可及的小算力平台上高效运行。作为这一探索的一部分,我们推出了 LLaVaOLMoBitnet1B——首个能够接受图像和文本输入并生成连贯文本回复的三元多模态大语言模型。该模型完全开源,并附有训练脚本,以鼓励该领域的进一步研究。本技术报告重点介绍了训练过程、评估细节、三元模型面临的挑战以及未来的发展机遇。模型链接:https://huggingface.co/IntelLabs/LlavaOLMoBitnet1B

英文摘要:

Multimodal Large Language Models (MM-LLMs) have seen significant advancements in the last year, demonstrating impressive performance across tasks. However, to truly democratize AI, models must exhibit strong capabilities and be able to run efficiently on small compute footprints accessible by most. Part of this quest, we introduce LLaVaOLMoBitnet1B - the first Ternary Multimodal LLM capable of accepting Image(s)+Text inputs to produce coherent textual responses. The model is fully open-sourced along with training scripts to encourage further research in this space. This accompanying technical report highlights the training process, evaluation details, challenges associated with ternary models and future opportunities. Link to the model: https://huggingface.co/IntelLabs/LlavaOLMoBitnet1B

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑