视觉大模型 / VLM - arXivDaily 专题

2501.15368 2025-01-28 cs.CL cs.SD eess.AS 50%

Baichuan-Omni-1.5 Technical Report

Yadong Li, Jun Liu, Tao Zhang, Tao Zhang, Song Chen, Tianpeng Li, Zehuan Li, Lijun Liu, Lingfeng Ming, Guosheng Dong, Da Pan, Chong Li, Yuanbo Fang, Dongdong Kuang, Mingrui Wang, Chenglin Zhu, Youwei Zhang, Hongyu Guo, Fengyu Zhang, Yuran Wang, Bowen Ding, Wei Song, Xu Li, Yuqi Huo, Zheng Liang, Shusen Zhang, Xin Wu, Shuai Zhao, Linchu Xiong, Yozhen Wu, Jiahui Ye, Wenhao Lu, Bowen Li, Yan Zhang, Yaqi Zhou, Xin Chen, Lei Su, Hongda Zhang, Fuzhong Chen, Xuezhen Dong, Na Nie, Zhiying Wu, Bin Xiao, Ting Li, Shunya Dang, Ping Zhang, Yijia Sun, Jincheng Wu, Jinjie Yang, Xionghai Lin, Zhi Ma, Kegeng Wu, Jia li, Aiyuan Yang, Hui Liu, Jianqiang Zhang, Xiaoxi Chen, Guangwei Ai, Wentao Zhang, Yicong Chen, Xiaoqin Huang, Kun Li, Wenjing Luo, Yifei Duan, Lingling Zhu, Ran Xiao, Zhe Su, Jiani Pu, Dian Wang, Xu Jia, Tianyu Zhang, Mengyu Ai, Mang Wang, Yujing Qiao, Lei Zhang, Yanjun Shen, Fan Yang, Miao Zhen, Yijie Zhou, Mingyang Chen, Fei Li, Chenzheng Zhu, Keer Lu, Yaqi Zhao, Hao Liang, Youquan Li, Yanzhao Qin, Linzhuang Sun, Jianhua Xu, Haoze Sun, Mingan Lin, Zenan Zhou, Weipeng Chen

专题命中 VLM训练与架构：MLLM(abstract)

2405.16789 2025-01-22 cs.IR 50%

NoteLLM-2: Multimodal Large Representation Models for Recommendation

Chao Zhang, Haoxin Zhang, Shiwei Wu, Di Wu, Tong Xu, Xiangyu Zhao, Yan Gao, Yao Hu, Enhong Chen

专题命中 VLM训练与架构：multimodal large language model(abstract)

Comments Accepted by KDD'25 ADS track

2501.07295 2025-01-16 cs.RO 50%

GestLLM: Advanced Hand Gesture Interpretation via Large Language Models for Human-Robot Interaction

Oleg Kobzarev, Artem Lykov, Dzmitry Tsetserukou

专题命中 VLM训练与架构：vision-language model(abstract)

2306.00398 2025-01-09 cs.CL 50%

Preference-grounded Token-level Guidance for Language Model Fine-tuning

Shentao Yang, Shujian Zhang, Congying Xia, Yihao Feng, Caiming Xiong, Mingyuan Zhou

专题命中 VLM训练与架构：grounding(abstract)

Comments v2: 37th Conference on Neural Information Processing Systems (NeurIPS 2023); v3: update on scaling up to PPO + LLMs

2405.18653 2024-12-02 cs.CL 50%

Recent Advances of Foundation Language Models-based Continual Learning: A Survey

Yutao Yang, Jie Zhou, Xuanwen Ding, Tianyu Huai, Shunyu Liu, Qin Chen, Yuan Xie, Liang He

专题命中 VLM训练与架构：vision-language model(abstract)

Comments Accepted by ACM Computing Survey

2410.20494 2024-10-29 cs.CL 50%

MatViX: Multimodal Information Extraction from Visually Rich Articles

Ghazal Khalighinejad, Sharon Scott, Ollie Liu, Kelly L. Anderson, Rickard Stureborg, Aman Tyagi, Bhuwan Dhingra

专题命中 VLM训练与架构：vision-language model(abstract)

2407.02067 2024-10-21 cs.CL 50%

Crossroads of Continents: Automated Artifact Extraction for Cultural Adaptation with Large Multimodal Models

Anjishnu Mukherjee, Ziwei Zhu, Antonios Anastasopoulos

专题命中 VLM训练与架构：LLaVA(abstract)

Comments under review

2410.05582 2024-10-10 cs.RO 50%

Gen-Drive: Enhancing Diffusion Generative Driving Policies with Reward Modeling and Reinforcement Learning Fine-tuning

Zhiyu Huang, Xinshuo Weng, Maximilian Igl, Yuxiao Chen, Yulong Cao, Boris Ivanovic, Marco Pavone, Chen Lv

专题命中 VLM训练与架构：VLM(abstract)

2410.03265 2024-10-08 cs.IR cs.SI 50%

Multimodal Point-of-Interest Recommendation

Yuta Kanzawa, Toyotaro Suzumura, Hiroki Kanezashi, Jiawei Yong, Shintaro Fukushima

专题命中 VLM训练与架构：LLaVA(abstract)

2409.14247 2024-10-07 cs.CL cs.HC 50%

Repairs in a Block World: A New Benchmark for Handling User Corrections with Multi-Modal Language Models

Javier Chiyah-Garcia, Alessandro Suglia, Arash Eshghi

专题命中 VLM训练与架构：VLM(abstract)

Comments Accepted to EMNLP'24 Main (Upcoming). Data and code at www.github.com/JChiyah/blockworld-repairs - for Bibtex see https://raw.githubusercontent.com/JChiyah/blockworld-repairs/refs/heads/main/citation.bib

2408.08640 2024-09-26 cs.CL 50%

Math-PUMA: Progressive Upward Multimodal Alignment to Enhance Mathematical Reasoning

Wenwen Zhuang, Xin Huang, Xiantao Zhang, Jin Zeng

专题命中 VLM训练与架构：multimodal large language model(abstract)

2409.03054 2024-09-06 cs.HC 50%

Context-Aware Image Descriptions for Web Accessibility

Ananya Gubbi Mohanbabu, Amy Pavel

专题命中 VLM训练与架构：LLaVA(abstract)

Comments 17 pages, 11 figures

2407.21417 2024-08-01 cs.CL 50%

Dancing in Chains: Reconciling Instruction Following and Faithfulness in Language Models

Zhengxuan Wu, Yuhao Zhang, Peng Qi, Yumo Xu, Rujun Han, Yian Zhang, Jifan Chen, Bonan Min, Zhiheng Huang

专题命中 VLM训练与架构：grounding(abstract)

Comments preprint

2407.17974 2024-07-26 cs.CL 50%

What does Kiki look like? Cross-modal associations between speech sounds and visual shapes in vision-and-language models

Tessa Verhoef, Kiana Shahrasbi, Tom Kouwenhoven

专题命中 VLM训练与架构：VLM(abstract)

Comments Appeared at the 13th edition of the Workshop on Cognitive Modeling and Computational Linguistics (CMCL 2024)

2403.07794 2024-07-04 cs.CL 50%

Fine-tuning Large Language Models with Sequential Instructions

Hanxu Hu, Simon Yu, Pinzhen Chen, Edoardo M. Ponti

专题命中 VLM训练与架构：visual question answering(abstract)

Comments 21pages, 8 figures

2402.16797 2024-06-11 cs.CL 50%

Set the Clock: Temporal Alignment of Pretrained Language Models

Bowen Zhao, Zander Brumbaugh, Yizhong Wang, Hannaneh Hajishirzi, Noah A. Smith

专题命中 VLM训练与架构：grounding(abstract)

Comments Accepted as Findings of ACL 2024. Our code and data is available at https://github.com/yizhongw/llm-temporal-alignment

2404.17779 2024-04-30 cs.CL 50%

Medical Vision-Language Pre-Training for Brain Abnormalities

Masoud Monajatipoor, Zi-Yi Dou, Aichi Chien, Nanyun Peng, Kai-Wei Chang

专题命中 VLM训练与架构：vision-language model(abstract)

2404.10238 2024-04-17 math.NA cs.NA 50%

A novel interpretation of Nesterov's acceleration via variable step-size linear multistep methods

Ryota Nozawa, Shun Sato, Takayasu Matsuo

专题命中 VLM训练与架构：VLM(abstract)

2312.05278 2024-04-15 cs.CL 50%

Lyrics: Boosting Fine-grained Language-Vision Alignment and Comprehension via Semantic-aware Visual Objects

Junyu Lu, Dixiang Zhang, Songxin Zhang, Zejian Xie, Zhuoyang Song, Cong Lin, Jiaxing Zhang, Bingyi Jing, Pingjian Zhang

专题命中 VLM训练与架构：vision language model(abstract)

2311.07463 2024-04-04 cs.CL 50%

MEGAVERSE: Benchmarking Large Language Models Across Languages, Modalities, Models and Tasks

Sanchit Ahuja, Divyanshu Aggarwal, Varun Gumma, Ishaan Watts, Ashutosh Sathe, Millicent Ochieng, Rishav Hada, Prachi Jain, Maxamed Axmed, Kalika Bali, Sunayana Sitaram

专题命中 VLM训练与架构：LLaVA(abstract)

Comments 40 pages, 35 figures and 34 tables

2403.20116 2024-04-01 cs.RO 50%

LeGo-Drive: Language-enhanced Goal-oriented Closed-Loop End-to-End Autonomous Driving

Pranjal Paul, Anant Garg, Tushar Choudhary, Arun Kumar Singh, K. Madhava Krishna

专题命中 VLM训练与架构：vision-language model(abstract)

2403.10496 2024-03-18 cs.RO 50%

Reconfigurable Robot Identification from Motion Data

Yuhang Hu, Yunzhe Wang, Ruibo Liu, Zhou Shen, Hod Lipson

专题命中 VLM训练与架构：vision-language model(abstract)

2402.17936 2024-02-29 cs.CL 50%

Acquiring Linguistic Knowledge from Multimodal Input

Theodor Amariucai, Alex Warstadt

专题命中 VLM训练与架构：grounding(abstract)

Comments in Proceedings of the BabyLM Challenge at the 27th Conference on Computational Natural Language Learning

2402.11297 2024-02-20 cs.CL 50%

MMMModal -- Multi-Images Multi-Audio Multi-turn Multi-Modal

Husein Zolkepli, Aisyah Razak, Kamarul Adha, Ariff Nazhan

专题命中 VLM训练与架构：multimodal large language model(abstract)

2305.07358 2024-02-19 cs.CL 50%

Towards Versatile and Efficient Visual Knowledge Integration into Pre-trained Language Models with Cross-Modal Adapters

Xinyun Zhang, Haochen Tan, Han Wu, Bei Yu

专题命中 VLM训练与架构：vision-language model(abstract)

2312.01592 2024-01-10 cs.CL 50%

Expand BERT Representation with Visual Information via Grounded Language Learning with Multimodal Partial Alignment

Cong-Duy Nguyen, The-Anh Vu-Le, Thong Nguyen, Tho Quan, Luu Anh Tuan

专题命中 VLM训练与架构：grounding(abstract)

2401.01076 2024-01-04 cs.CL 50%

DialCLIP: Empowering CLIP as Multi-Modal Dialog Retriever

Zhichao Yin, Binyuan Hui, Min Yang, Fei Huang, Yongbin Li

专题命中 VLM训练与架构：vision-language model(abstract)

Comments ICASSP 2024

2201.12501 2023-08-01 cs.CL 50%

Does Transliteration Help Multilingual Language Modeling?

Ibraheem Muhammad Moosa, Mahmud Elahi Akhter, Ashfia Binte Habib

专题命中 VLM训练与架构：MLLM(abstract)

Comments In Findings of the Association for Computational Linguistics: EACL 2023

2304.08518 2023-07-12 astro-ph.EP astro-ph.SR 50%

Mapping the Skies of Ultracool Worlds: Detecting Storms and Spots with Extremely Large Telescopes

Michael K. Plummer, Ji Wang

专题命中 VLM训练与架构：VLM(abstract)

Comments Accepted for publication in The Astrophysical Journal, 26 pages, 10 figures, 3 tables

2303.08877 2023-03-17 cs.CL q-bio.NC 50%

ROSE: A Neurocomputational Architecture for Syntax

Elliot Murphy

专题命中 VLM训练与架构：grounding(abstract)

1. VLM训练与架构 5058 篇

Baichuan-Omni-1.5 Technical Report

NoteLLM-2: Multimodal Large Representation Models for Recommendation

GestLLM: Advanced Hand Gesture Interpretation via Large Language Models for Human-Robot Interaction

Preference-grounded Token-level Guidance for Language Model Fine-tuning

Recent Advances of Foundation Language Models-based Continual Learning: A Survey

MatViX: Multimodal Information Extraction from Visually Rich Articles

Crossroads of Continents: Automated Artifact Extraction for Cultural Adaptation with Large Multimodal Models

Gen-Drive: Enhancing Diffusion Generative Driving Policies with Reward Modeling and Reinforcement Learning Fine-tuning

Multimodal Point-of-Interest Recommendation

Repairs in a Block World: A New Benchmark for Handling User Corrections with Multi-Modal Language Models

Math-PUMA: Progressive Upward Multimodal Alignment to Enhance Mathematical Reasoning

Context-Aware Image Descriptions for Web Accessibility

Dancing in Chains: Reconciling Instruction Following and Faithfulness in Language Models

What does Kiki look like? Cross-modal associations between speech sounds and visual shapes in vision-and-language models

Fine-tuning Large Language Models with Sequential Instructions

Set the Clock: Temporal Alignment of Pretrained Language Models

Medical Vision-Language Pre-Training for Brain Abnormalities

A novel interpretation of Nesterov's acceleration via variable step-size linear multistep methods

Lyrics: Boosting Fine-grained Language-Vision Alignment and Comprehension via Semantic-aware Visual Objects

MEGAVERSE: Benchmarking Large Language Models Across Languages, Modalities, Models and Tasks

LeGo-Drive: Language-enhanced Goal-oriented Closed-Loop End-to-End Autonomous Driving

Reconfigurable Robot Identification from Motion Data

Acquiring Linguistic Knowledge from Multimodal Input

MMMModal -- Multi-Images Multi-Audio Multi-turn Multi-Modal

Towards Versatile and Efficient Visual Knowledge Integration into Pre-trained Language Models with Cross-Modal Adapters

Expand BERT Representation with Visual Information via Grounded Language Learning with Multimodal Partial Alignment

DialCLIP: Empowering CLIP as Multi-Modal Dialog Retriever

Does Transliteration Help Multilingual Language Modeling?

Mapping the Skies of Ultracool Worlds: Detecting Storms and Spots with Extremely Large Telescopes

ROSE: A Neurocomputational Architecture for Syntax