多模态大模型

Tur[k]ingBench: A Challenge Benchmark for Web Agents

Kevin Xu, Yeganeh Kordi, Tanay Nayak, Adi Asija, Yizhong Wang, Kate Sanders, Adam Byerly, Jingyu Zhang, Benjamin Van Durme, Daniel Khashabi

专题命中 多模态评测 ：multi-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

LOVA3: Learning to Visual Question Answering, Asking and Assessment

Henry Hengyuan Zhao, Pan Zhou, Difei Gao, Zechen Bai, Mike Zheng Shou

机构 * National University of Singapore（新加坡国立大学） ； Singapore Management University（新加坡管理大学） 纠错

专题命中 多模态评测 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments NeurIPS 2024. The code is available at https://github.com/showlab/LOVA3

Movie2Story: A framework for understanding videos and telling stories in the form of novel text

Kangning Li, Zheyang Jia, Anyu Ying

专题命中 多模态评测 ：multi-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

Detecting and Mitigating Hallucination in Large Vision Language Models via Fine-Grained AI Feedback

Wenyi Xiao, Ziwei Huang, Leilei Gan, Wanggui He, Haoyuan Li, Zhelun Yu, Fangxun Shu, Hao Jiang, Linchao Zhu

专题命中 多模态评测 ：multi-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments repo: https://github.com/Mr-Loevan/HSA-DPO

Investigating Acoustic-Textual Emotional Inconsistency Information for Automatic Depression Detection

Rongfeng Su, Changqing Xu, Xinyi Wu, Feng Xu, Xie Chen, Lan Wangt, Nan Yan

机构 * Guangdong-Hong Kong-Macao Joint Laboratory of Human-Machine Intelligence-Synergy Systems, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院粤港澳人机协同智能系统联合实验室） ； University of Chinese Academy of Sciences（中国科学院大学） ； Good Mood Health Industry Group Co., Ltd（好心情健康产业集团有限公司） ； Shanghai Jiao Tong University（上海交通大学） 纠错

专题命中 多模态评测 ：multimodal(abstract)；分类 cs.CL、cs.AI、eess.AS

G-VEval: A Versatile Metric for Evaluating Image and Video Captions Using GPT-4o

Tony Cheng Tong, Sirui He, Zhiwen Shao, Dit-Yan Yeung

专题命中 多模态评测 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

How Good is ChatGPT at Audiovisual Deepfake Detection: A Comparative Study of ChatGPT, AI Models and Human Perception

Sahibzada Adil Shahzad, Ammarah Hashmi, Yan-Tsung Peng, Yu Tsao, Hsin-Min Wang

机构 * Academia Sinica（中央研究院） ； National Chengchi University（国立政治大学） ； National Tsing Hua University（国立清华大学） 纠错

专题命中 多模态评测 ：multimodal(abstract)；分类 cs.CV、cs.AI、cs.MM

Exploring Diverse Methods in Visual Question Answering

Panfeng Li, Qikai Yang, Xieming Geng, Wenjing Zhou, Zhicheng Ding, Yi Nian

机构 * University of Michigan（密歇根大学） ； University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校） ； Chongqing University（重庆大学） ； Columbia University（哥伦比亚大学） ； University of Chicago（芝加哥大学） 纠错

专题命中 多模态评测 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Accepted by 2024 5th International Conference on Electronic Communication and Artificial Intelligence

Journal ref Proceedings of the 2024 5th International Conference on Electronic Communication and Artificial Intelligence (ICECAI), 2024, pp. 681-685

CVQA: Culturally-diverse Multilingual Visual Question Answering Benchmark

David Romero, Chenyang Lyu, Haryo Akbarianto Wibowo, Teresa Lynn, Injy Hamed, Aditya Nanda Kishore, Aishik Mandal, Alina Dragonetti, Artem Abzaliev, Atnafu Lambebo Tonja, Bontu Fufa Balcha, Chenxi Whitehouse, Christian Salamea, Dan John Velasco, David Ifeoluwa Adelani, David Le Meur, Emilio Villa-Cueva, Fajri Koto, Fauzan Farooqui, Frederico Belcavello, Ganzorig Batnasan, Gisela Vallejo, Grainne Caulfield, Guido Ivetta, Haiyue Song, Henok Biadglign Ademtew, Hernán Maina, Holy Lovenia, Israel Abebe Azime, Jan Christian Blaise Cruz, Jay Gala, Jiahui Geng, Jesus-German Ortiz-Barajas, Jinheon Baek, Jocelyn Dunstan, Laura Alonso Alemany, Kumaranage Ravindu Yasas Nagasinghe, Luciana Benotti, Luis Fernando D'Haro, Marcelo Viridiano, Marcos Estecha-Garitagoitia, Maria Camila Buitrago Cabrera, Mario Rodríguez-Cantelar, Mélanie Jouitteau, Mihail Mihaylov, Mohamed Fazli Mohamed Imam, Muhammad Farid Adilazuarda, Munkhjargal Gochoo, Munkh-Erdene Otgonbold, Naome Etori, Olivier Niyomugisha, Paula Mónica Silva, Pranjal Chitale, Raj Dabre, Rendi Chevi, Ruochen Zhang, Ryandito Diandaru, Samuel Cahyawijaya, Santiago Góngora, Soyeong Jeong, Sukannya Purkayastha, Tatsuki Kuribayashi, Teresa Clifford, Thanmay Jayakumar, Tiago Timponi Torrent, Toqeer Ehsan, Vladimir Araujo, Yova Kementchedjhieva, Zara Burzo, Zheng Wei Lim, Zheng Xin Yong, Oana Ignat, Joan Nwatu, Rada Mihalcea, Thamar Solorio, Alham Fikri Aji

机构 * MBZUAI 纠错

专题命中 多模态评测 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments 38th Conference on Neural Information Processing Systems (NeurIPS 2024) Track on Datasets and Benchmarks

Efficient Learned Query Execution over Text and Tables [Technical Report]

Matthias Urban, Carsten Binnig

专题命中 多模态评测 ：multimodal(abstract)；multi-modal(abstract)

VLKEB: A Large Vision-Language Model Knowledge Editing Benchmark

Han Huang, Haitian Zhong, Tao Yu, Qiang Liu, Shu Wu, Liang Wang, Tieniu Tan

专题命中 多模态评测 ：multi-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments NeurIPS 2024, Datasets and Benchmarks Track

CAMEL-Bench: A Comprehensive Arabic LMM Benchmark

Sara Ghaboura, Ahmed Heakl, Omkar Thawakar, Ali Alharthi, Ines Riahi, Abduljalil Saif, Jorma Laaksonen, Fahad S. Khan, Salman Khan, Rao M. Anwer

专题命中 多模态评测 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments 10 pages, 5 figures, NAACL

Dating ancient manuscripts using radiocarbon and AI-based writing style analysis

Mladen Popović, Maruf A. Dhali, Lambert Schomaker, Johannes van der Plicht, Kaare Lund Rasmussen, Jacopo La Nasa, Ilaria Degano, Maria Perla Colombini, Eibert Tigchelaar

专题命中 多模态评测 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments 16 pages of main article, 103 pages of supplementary materials; the first version of this article is originally prepared in July 2023 after the completion of all the experiments

FineFake: A Knowledge-Enriched Dataset for Fine-Grained Multi-Domain Fake News Detection

Ziyi Zhou, Xiaoming Zhang, Litian Zhang, Jiacheng Liu, Senzhang Wang, Zheng Liu, Xi Zhang, Chaozhuo Li, Philip S. Yu

专题命中 多模态评测 ：multi-modal(abstract)；分类 cs.CL、cs.AI、cs.MM

Benchmarking Vision Language Models for Cultural Understanding

Shravan Nayak, Kanishk Jain, Rabiul Awal, Siva Reddy, Sjoerd van Steenkiste, Lisa Anne Hendricks, Karolina Stańczak, Aishwarya Agrawal

专题命中 多模态评测 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Accepted to EMNLP 2024 Main Conference

ArMeme: Propagandistic Content in Arabic Memes

Firoj Alam, Abul Hasnat, Fatema Ahmed, Md Arid Hasan, Maram Hasanain

专题命中 多模态评测 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments disinformation, misinformation, factuality, harmfulness, fake news, propaganda, multimodality, text, images

MARPLE: A Benchmark for Long-Horizon Inference

Emily Jin, Zhuoyi Huang, Jan-Philipp Fränken, Weiyu Liu, Hannah Cha, Erik Brockbank, Sarah Wu, Ruohan Zhang, Jiajun Wu, Tobias Gerstenberg

专题命中 多模态评测 ：multimodal(abstract)；multi-modal(abstract)

Comments NeurIPS 2024. First two authors contributed equally. Project page: https://marple-benchmark.github.io/

Eureka: Evaluating and Understanding Large Foundation Models

Vidhisha Balachandran, Jingya Chen, Neel Joshi, Besmira Nushi, Hamid Palangi, Eduardo Salinas, Vibhav Vineet, James Woffinden-Luey, Safoora Yousefi

专题命中 多模态评测 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Enhancing Vision Models for Text-Heavy Content Understanding and Interaction

Adithya TG, Adithya SK, Abhinav R Bharadwaj, Abhiram HA, Surabhi Narayan

专题命中 多模态评测 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments 5 pages, 4 figures (including 1 graph)

MultiHateClip: A Multilingual Benchmark Dataset for Hateful Video Detection on YouTube and Bilibili

Han Wang, Tan Rui Yang, Usman Naseem, Roy Ka-Wei Lee

专题命中 多模态评测 ：multimodal(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments 10 pages, 3 figures, ACM Multimedia 2024

MMWorld: Towards Multi-discipline Multi-faceted World Model Evaluation in Videos

Xuehai He, Weixi Feng, Kaizhi Zheng, Yujie Lu, Wanrong Zhu, Jiachen Li, Yue Fan, Jianfeng Wang, Linjie Li, Zhengyuan Yang, Kevin Lin, William Yang Wang, Lijuan Wang, Xin Eric Wang

专题命中 多模态评测 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

How Good (Or Bad) Are LLMs at Detecting Misleading Visualizations?

Leo Yu-Ho Lo, Huamin Qu

专题命中 多模态评测 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments To be presented at IEEE VIS 2024

MuChin: A Chinese Colloquial Description Benchmark for Evaluating Language Models in the Field of Music

Zihao Wang, Shuyu Li, Tao Zhang, Qi Wang, Pengfei Yu, Jinyang Luo, Yan Liu, Ming Xi, Kejun Zhang

专题命中 多模态评测 ：multimodal(abstract)；分类 cs.AI、cs.MM、eess.AS

Comments Accepted by International Joint Conference on Artificial Intelligence 2024 (IJCAI 2024)

REBUS: A Robust Evaluation Benchmark of Understanding Symbols

Andrew Gritsevskiy, Arjun Panickssery, Aaron Kirtland, Derik Kauffman, Hans Gundlach, Irina Gritsevskaya, Joe Cavanagh, Jonathan Chiang, Lydia La Roux, Michelle Hung

专题命中 多模态评测 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments 20 pages, 5 figures. For code, see http://github.com/cvndsh/rebus

SMP Challenge: An Overview and Analysis of Social Media Prediction Challenge

Bo Wu, Peiye Liu, Wen-Huang Cheng, Bei Liu, Zhaoyang Zeng, Jia Wang, Qiushi Huang, Jiebo Luo

专题命中 多模态评测 ：multimodal(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments ACM Multimedia. arXiv admin note: text overlap with arXiv:1910.01795

MileBench: Benchmarking MLLMs in Long Context

Dingjie Song, Shunian Chen, Guiming Hardy Chen, Fei Yu, Xiang Wan, Benyou Wang

专题命中 多模态评测 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments 31 pages, 13 figures, 14 tables; We add results of GPT-4o in this version

Capabilities of Gemini Models in Medicine

Khaled Saab, Tao Tu, Wei-Hung Weng, Ryutaro Tanno, David Stutz, Ellery Wulczyn, Fan Zhang, Tim Strother, Chunjong Park, Elahe Vedadi, Juanma Zambrano Chaves, Szu-Yeu Hu, Mike Schaekermann, Aishwarya Kamath, Yong Cheng, David G. T. Barrett, Cathy Cheung, Basil Mustafa, Anil Palepu, Daniel McDuff, Le Hou, Tomer Golany, Luyang Liu, Jean-baptiste Alayrac, Neil Houlsby, Nenad Tomasev, Jan Freyberg, Charles Lau, Jonas Kemp, Jeremy Lai, Shekoofeh Azizi, Kimberly Kanada, SiWai Man, Kavita Kulkarni, Ruoxi Sun, Siamak Shakeri, Luheng He, Ben Caine, Albert Webson, Natasha Latysheva, Melvin Johnson, Philip Mansfield, Jian Lu, Ehud Rivlin, Jesper Anderson, Bradley Green, Renee Wong, Jonathan Krause, Jonathon Shlens, Ewa Dominowska, S. M. Ali Eslami, Katherine Chou, Claire Cui, Oriol Vinyals, Koray Kavukcuoglu, James Manyika, Jeff Dean, Demis Hassabis, Yossi Matias, Dale Webster, Joelle Barral, Greg Corrado, Christopher Semturs, S. Sara Mahdavi, Juraj Gottweis, Alan Karthikesalingam, Vivek Natarajan

专题命中 多模态评测 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

An Examination of the Compositionality of Large Generative Vision-Language Models

Teli Ma, Rong Li, Junwei Liang

专题命中 多模态评测 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Customizable Perturbation Synthesis for Robust SLAM Benchmarking

Xiaohao Xu, Tianyi Zhang, Sibo Wang, Xiang Li, Yongqi Chen, Ye Li, Bhiksha Raj, Matthew Johnson-Roberson, Xiaonan Huang

专题命中 多模态评测 ：multi-modal(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments 40 pages

VIALM: A Survey and Benchmark of Visually Impaired Assistance with Large Models

Yi Zhao, Yilin Zhang, Rong Xiang, Jing Li, Hillming Li

专题命中 多模态评测 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments under review