多模态大模型 - arXivDaily 专题

2407.15590 2024-07-23 cs.CV 70%

All rivers run into the sea: Unified Modality Brain-like Emotional Central Mechanism

Xinji Mai, Junxiong Lin, Haoran Wang, Zeng Tao, Yan Wang, Shaoqi Yan, Xuan Tong, Jiawen Yu, Boyang Wang, Ziheng Zhou, Qing Zhao, Shuyong Gao, Wenqiang Zhang

专题命中多模态评测：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV

2407.14237 2024-07-22 cs.NE cs.AI cs.DS 70%

Hyper-Heuristics Can Profit From Global Variation Operators

Benjamin Doerr, Johannes F. Lutzeyer

专题命中多模态评测：multimodal(abstract)；multi-modal(abstract)；分类 cs.AI

Comments Continues and significantly extends work presented in the GECCO 2023 conference paper arXiv:2304.10414

2403.13043 2024-07-19 cs.CV 70%

When Do We Not Need Larger Vision Models?

Baifeng Shi, Ziyang Wu, Maolin Mao, Xin Wang, Trevor Darrell

专题命中多模态评测：multimodal(abstract)；MLLM(abstract)；分类 cs.CV

Comments Code: https://github.com/bfshi/scaling_on_scales

2404.05052 2024-07-15 cs.CV 70%

Facial Affective Behavior Analysis with Instruction Tuning

Yifan Li, Anh Dao, Wentao Bao, Zhen Tan, Tianlong Chen, Huan Liu, Yu Kong

专题命中多模态评测：multi-modal(abstract)；MLLM(abstract)；分类 cs.CV

Comments V2.0, project page: https://johnx69.github.io/FABA/

2404.05673 2024-07-12 cs.CV 70%

CoReS: Orchestrating the Dance of Reasoning and Segmentation

Xiaoyi Bao, Siyang Sun, Shuailei Ma, Kecheng Zheng, Yuxin Guo, Guosheng Zhao, Yun Zheng, Xingang Wang

专题命中多模态评测：multi-modal(abstract)；MLLM(abstract)；分类 cs.CV

Comments Accepted at ECCV 2024

2406.14250 2024-07-02 cs.CV cs.HC 70%

E-ANT: A Large-Scale Dataset for Efficient Automatic GUI NavigaTion

Ke Wang, Tianyu Xia, Zhangxuan Gu, Yi Zhao, Shuheng Shen, Changhua Meng, Weiqiang Wang, Ke Xu

专题命中多模态评测：multimodal(abstract)；MLLM(abstract)；分类 cs.CV

Comments 9 pages, 5 figures, Under review

2406.00917 2024-06-04 cs.CV 70%

Alignment-Free RGBT Salient Object Detection: Semantics-guided Asymmetric Correlation Network and A Unified Benchmark

Kunpeng Wang, Danying Lin, Chenglong Li, Zhengzheng Tu, Bin Luo

专题命中多模态评测：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments Accepted by TMM 2024

2405.20363 2024-06-03 cs.CV 70%

LLMGeo: Benchmarking Large Language Models on Image Geolocation In-the-wild

Zhiqiang Wang, Dejia Xu, Rana Muhammad Shahroz Khan, Yanbin Lin, Zhiwen Fan, Xingquan Zhu

专题命中多模态评测：multimodal(abstract)；multi-modal(abstract)；分类 cs.CV

Comments 7 pages, 3 figures, 5 tables, CVPR 2024 Workshop on Computer Vision in the Wild

2306.09278 2024-04-30 cs.CV 70%

Robustness Analysis on Foundational Segmentation Models

Madeline Chantry Schiappa, Shehreen Azad, Sachidanand VS, Yunhao Ge, Ondrej Miksik, Yogesh S. Rawat, Vibhav Vineet

专题命中多模态评测：multimodal(abstract)；multi-modal(abstract)；分类 cs.CV

Comments This benchmark along with the code and datasets is available at: https://tinyurl.com/fm-robust. Accepted at CVPRW 2024

2403.20194 2024-04-26 cs.MM 70%

ConvBench: A Multi-Turn Conversation Evaluation Benchmark with Hierarchical Capability for Large Vision-Language Models

Shuo Liu, Kaining Ying, Hao Zhang, Yue Yang, Yuqi Lin, Tianle Zhang, Chuanhao Li, Yu Qiao, Ping Luo, Wenqi Shao, Kaipeng Zhang

专题命中多模态评测：multimodal(abstract)；multi-modal(abstract)；分类 cs.MM

2401.11775 2024-01-24 cs.CV 70%

Collaborative Position Reasoning Network for Referring Image Segmentation

Jianjian Cao, Beiya Dai, Yulin Li, Xiameng Qin, Jingdong Wang

专题命中多模态评测：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

2311.06815 2023-12-27 cs.CL 70%

Evaluation of GPT-4 for chest X-ray impression generation: A reader study on performance and perception

Sebastian Ziegelmayer, Alexander W. Marka, Nicolas Lenhart, Nadja Nehls, Stefan Reischl, Felix Harder, Andreas Sauter, Marcus Makowski, Markus Graf, Joshua Gawlitza

专题命中多模态评测：multimodal(abstract)；multimodal foundation model(abstract)；分类 cs.CL

Journal ref J Med Internet Res 2023;25:e50865

2312.12418 2023-12-20 cs.CV 70%

LASA: Instance Reconstruction from Real Scans using A Large-scale Aligned Shape Annotation Dataset

Haolin Liu, Chongjie Ye, Yinyu Nie, Yingfan He, Xiaoguang Han

专题命中多模态评测：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments homepage: https://gap-lab-cuhk-sz.github.io/LASA/

2312.11829 2023-12-20 cs.CV 70%

RadOcc: Learning Cross-Modality Occupancy Knowledge through Rendering Assisted Distillation

Haiming Zhang, Xu Yan, Dongfeng Bai, Jiantao Gao, Pan Wang, Bingbing Liu, Shuguang Cui, Zhen Li

专题命中多模态评测：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments Accepted by AAAI 2024

2311.17122 2023-11-30 cs.CV 70%

Large Model Based Referring Camouflaged Object Detection

Shupeng Cheng, Ge-Peng Ji, Pengda Qin, Deng-Ping Fan, Bowen Zhou, Peng Xu

专题命中多模态评测：multimodal(abstract)；MLLM(abstract)；分类 cs.CV

2311.16479 2023-11-29 cs.CV 70%

Mitigating Hallucination in Visual Language Models with Visual Supervision

Zhiyang Chen, Yousong Zhu, Yufei Zhan, Zhaowen Li, Chaoyang Zhao, Jinqiao Wang, Ming Tang

专题命中多模态评测：multi-modal(abstract)；image-text(abstract)；分类 cs.CV

2305.12011 2023-11-09 cs.CV eess.IV 70%

Boosting Crop Classification by Hierarchically Fusing Satellite, Rotational, and Contextual Data

Valentin Barriere, Martin Claverie, Maja Schneider, Guido Lemoine, Raphaël d'Andrimont

专题命中多模态评测：multimodal(abstract)；multi-modal(abstract)；分类 cs.CV

Comments submitted to Remote Sensing of Environment, special issue Deep Learning for Time Series. Version 2

2310.02569 2023-10-18 cs.CV 70%

ReForm-Eval: Evaluating Large Vision Language Models via Unified Re-Formulation of Task-Oriented Benchmarks

Zejun Li, Ye Wang, Mengfei Du, Qingwen Liu, Binhao Wu, Jiwen Zhang, Chengxing Zhou, Zhihao Fan, Jie Fu, Jingjing Chen, Xuanjing Huang, Zhongyu Wei

专题命中多模态评测：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments 38 pages, 11 figures, 24 tables

2310.08487 2023-10-13 cs.CL 70%

GraphextQA: A Benchmark for Evaluating Graph-Enhanced Large Language Models

Yuanchun Shen, Ruotong Liao, Zhen Han, Yunpu Ma, Volker Tresp

专题命中多模态评测：multimodal(abstract)；multi-modal(abstract)；分类 cs.CL

2309.01339 2023-09-06 cs.CL cs.AI cs.CV cs.MM 70%

UniSA: Unified Generative Framework for Sentiment Analysis

Zaijing Li, Ting-En Lin, Yuchuan Wu, Meng Liu, Fengxiao Tang, Ming Zhao, Yongbin Li

专题命中多模态评测：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Accepted to ACM MM 2023

2204.01952 2023-07-19 cs.CV 70%

Towards On-Board Panoptic Segmentation of Multispectral Satellite Images

Tharindu Fernando, Clinton Fookes, Harshala Gammulle, Simon Denman, Sridha Sridharan

专题命中多模态评测：multimodal(abstract)；multi-modal(abstract)；分类 cs.CV

2209.03102 2023-03-06 cs.CV 70%

MSMDFusion: Fusing LiDAR and Camera at Multiple Scales with Multi-Depth Seeds for 3D Object Detection

Yang Jiao, Zequn Jie, Shaoxiang Chen, Jingjing Chen, Lin Ma, Yu-Gang Jiang

专题命中多模态评测：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments Accepted by CVPR 2023

2303.01480 2023-03-03 cs.CV 70%

Delivering Arbitrary-Modal Semantic Segmentation

Jiaming Zhang, Ruiping Liu, Hao Shi, Kailun Yang, Simon Reiß, Kunyu Peng, Haodong Fu, Kaiwei Wang, Rainer Stiefelhagen

专题命中多模态评测：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments Accepted by CVPR 2023. Dataset and our code are at: https://jamycheung.github.io/DELIVER.html

2211.02256 2022-11-07 eess.IV cs.CV 70%

ISA-Net: Improved spatial attention network for PET-CT tumor segmentation

Zhengyong Huang, Sijuan Zou, Guoshuai Wang, Zixiang Chen, Hao Shen, Haiyan Wang, Na Zhang, Lu Zhang, Fan Yang, Haining Wangg, Dong Liang, Tianye Niu, Xiaohua Zhuc, Zhanli Hua

专题命中多模态评测：multimodal(abstract)；multi-modal(abstract)；分类 cs.CV

2109.05569 2022-10-25 cs.CV 70%

MovieCuts: A New Dataset and Benchmark for Cut Type Recognition

Alejandro Pardo, Fabian Caba Heilbron, Juan León Alcázar, Ali Thabet, Bernard Ghanem

专题命中多模态评测：multi-modal(abstract)；audio-visual(abstract)；分类 cs.CV

Comments Paper's website: https://www.alejandropardo.net/publication/moviecuts/

Journal ref ECCV 2022

2209.02604 2022-09-07 cs.MM cs.AI cs.CV cs.SD eess.AS 70%

Make Acoustic and Visual Cues Matter: CH-SIMS v2.0 Dataset and AV-Mixup Consistent Module

Yihe Liu, Ziqi Yuan, Huisheng Mao, Zhiyun Liang, Wanqiuyue Yang, Yuanzhe Qiu, Tie Cheng, Xiaoteng Li, Hua Xu, Kai Gao

专题命中多模态评测：multimodal(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments 16pages, 7 figures, accepted by ICMI 2022

2203.11130 2022-08-02 cs.LG cs.AI cs.CL cs.CV cs.MM 70%

PACS: A Dataset for Physical Audiovisual CommonSense Reasoning

Samuel Yu, Peter Wu, Paul Pu Liang, Ruslan Salakhutdinov, Louis-Philippe Morency

专题命中多模态评测：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments ECCV 2022, 51 pages, 23 figures, 4 tables

2206.04053 2022-06-10 cs.LG cs.AI 70%

Unsupervised Knowledge Adaptation for Passenger Demand Forecasting

Can Li, Lei Bai, Wei Liu, Lina Yao, S Travis Waller

专题命中多模态评测：multimodal(abstract)；cross-modal(abstract)；分类 cs.AI

2205.05738 2022-05-13 cs.CL cs.AI cs.CV cs.CY cs.MM 70%

DISARM: Detecting the Victims Targeted by Harmful Memes

Shivam Sharma, Md. Shad Akhtar, Preslav Nakov, Tanmoy Chakraborty

专题命中多模态评测：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Accepted at NAACL 2022 (Findings)

2205.04404 2022-05-10 cs.CL cs.AI cs.CV cs.MM cs.SI 70%

TeamX@DravidianLangTech-ACL2022: A Comparative Analysis for Troll-Based Meme Classification

Rabindra Nath Nandi, Firoj Alam, Preslav Nakov

专题命中多模态评测：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Accepted at DravidianLangTech-ACL2022 (Colocated with ACL-2022). disinformation, misinformation, factuality, harmfulness, fake news, propaganda, multimodality, text, images, videos, network structure, temporality

1. 多模态评测 9205 篇

All rivers run into the sea: Unified Modality Brain-like Emotional Central Mechanism

Hyper-Heuristics Can Profit From Global Variation Operators

When Do We Not Need Larger Vision Models?

Facial Affective Behavior Analysis with Instruction Tuning

CoReS: Orchestrating the Dance of Reasoning and Segmentation

E-ANT: A Large-Scale Dataset for Efficient Automatic GUI NavigaTion

Alignment-Free RGBT Salient Object Detection: Semantics-guided Asymmetric Correlation Network and A Unified Benchmark

LLMGeo: Benchmarking Large Language Models on Image Geolocation In-the-wild

Robustness Analysis on Foundational Segmentation Models

ConvBench: A Multi-Turn Conversation Evaluation Benchmark with Hierarchical Capability for Large Vision-Language Models

Collaborative Position Reasoning Network for Referring Image Segmentation

Evaluation of GPT-4 for chest X-ray impression generation: A reader study on performance and perception

LASA: Instance Reconstruction from Real Scans using A Large-scale Aligned Shape Annotation Dataset

RadOcc: Learning Cross-Modality Occupancy Knowledge through Rendering Assisted Distillation

Large Model Based Referring Camouflaged Object Detection

Mitigating Hallucination in Visual Language Models with Visual Supervision

Boosting Crop Classification by Hierarchically Fusing Satellite, Rotational, and Contextual Data

ReForm-Eval: Evaluating Large Vision Language Models via Unified Re-Formulation of Task-Oriented Benchmarks

GraphextQA: A Benchmark for Evaluating Graph-Enhanced Large Language Models

UniSA: Unified Generative Framework for Sentiment Analysis

Towards On-Board Panoptic Segmentation of Multispectral Satellite Images

MSMDFusion: Fusing LiDAR and Camera at Multiple Scales with Multi-Depth Seeds for 3D Object Detection

Delivering Arbitrary-Modal Semantic Segmentation

ISA-Net: Improved spatial attention network for PET-CT tumor segmentation

MovieCuts: A New Dataset and Benchmark for Cut Type Recognition

Make Acoustic and Visual Cues Matter: CH-SIMS v2.0 Dataset and AV-Mixup Consistent Module

PACS: A Dataset for Physical Audiovisual CommonSense Reasoning

Unsupervised Knowledge Adaptation for Passenger Demand Forecasting

DISARM: Detecting the Victims Targeted by Harmful Memes

TeamX@DravidianLangTech-ACL2022: A Comparative Analysis for Troll-Based Meme Classification