多模态大模型

A Benchmark Dataset for Multimodal Prediction of Enzymatic Function Coupling DNA Sequences and Natural Language

Yuchen Zhang, Ratish Kumar Chandrakant Jha, Soumya Bharadwaj, Vatsal Sanjaykumar Thakkar, Adrienne Hoarfrost, Jin Sun

专题命中 多模态评测 ：multimodal(title)；multi-modal(abstract)

The Effects of Selected Object Features on a Pick-and-Place Task: a Human Multimodal Dataset

Linda Lastrico, Valerio Belcamino, Alessandro Carfì, Alessia Vignolo, Alessandra Sciutti, Fulvio Mastrogiovanni, Francesco Rea

专题命中 多模态评测 ：multimodal(title,abstract)

Comments Camera ready version. Full paper available in open-access at https://doi.org/10.1177/02783649231210965 Dataset available on Kaggle (DOI: 10.34740/KAGGLE/DS/2319925, https://www.kaggle.com/datasets/alessandrocarf/effects-of-object-characteristics-on-manipulations)

Journal ref The International Journal of Robotics Research. 2024;43(1):98-109.

MER 2024: Semi-Supervised Learning, Noise Robustness, and Open-Vocabulary Multimodal Emotion Recognition

Zheng Lian, Haiyang Sun, Licai Sun, Zhuofan Wen, Siyuan Zhang, Shun Chen, Hao Gu, Jinming Zhao, Ziyang Ma, Xie Chen, Jiangyan Yi, Rui Liu, Kele Xu, Bin Liu, Erik Cambria, Guoying Zhao, Björn W. Schuller, Jianhua Tao

专题命中 多模态评测 ：multimodal(title,abstract)

A Mobility Equity Metric for Multi-Modal Intelligent Transportation Systems

Heeseung Bang, Aditya Dave, Filippos N. Tzortzoglou, Andreas A. Malikopoulos

专题命中 多模态评测 ：multi-modal(title,abstract)

Comments 6 pages, 7 figures

Journal ref 17th IFAC Symposium on Control in Transportation Systems, Ayia Napa, Cyprus, 2024

AffectGPT: Dataset and Framework for Explainable Multimodal Emotion Recognition

Zheng Lian, Haiyang Sun, Licai Sun, Jiangyan Yi, Bin Liu, Jianhua Tao

专题命中 多模态评测 ：multimodal(title,abstract)

ERR@HRI 2024 Challenge: Multimodal Detection of Errors and Failures in Human-Robot Interactions

Micol Spitale, Maria Teresa Parreira, Maia Stiber, Minja Axelsson, Neval Kara, Garima Kankariya, Chien-Ming Huang, Malte Jung, Wendy Ju, Hatice Gunes

专题命中 多模态评测 ：multimodal(title,abstract)

Multimodal Machine Learning for Automated Assessment of Attention-Related Processes during Learning

Babette Bühler

专题命中 多模态评测 ：multimodal(title,abstract)

Comments Doctoral thesis

Certainly Uncertain: A Benchmark and Metric for Multimodal Epistemic and Aleatoric Awareness

Khyathi Raghavi Chandu, Linjie Li, Anas Awadalla, Ximing Lu, Jae Sung Park, Jack Hessel, Lijuan Wang, Yejin Choi

专题命中 多模态评测 ：multimodal(title)；分类 cs.CV、cs.CL、cs.AI

Comments 26 pages

Active Sensing Strategy: Multi-Modal, Multi-Robot Source Localization and Mapping in Real-World Settings with Fixed One-Way Switching

Vu Phi Tran, Asanka G. Perera, Matthew A. Garratt, Kathryn Kasmarik, Sreenatha G. Anavatti

专题命中 多模态评测 ：multi-modal(title,abstract)

Hinge-Wasserstein: Estimating Multimodal Aleatoric Uncertainty in Regression Tasks

Ziliang Xiong, Arvi Jonnarth, Abdelrahman Eldesokey, Joakim Johnander, Bastian Wandt, Per-Erik Forssen

专题命中 多模态评测 ：multimodal(title,abstract)

Multimodal Radiomics Model for Predicting Gold Nanoparticles Accumulation in Mouse Tumors

Jiajia Tang, Jie Zhang, Jiulou Zhang, Yuxia Tang, Hao Ni, Shouju Wang

专题命中 多模态评测 ：multimodal(title,abstract)

Multimodal LLMs Struggle with Basic Visual Network Analysis: a VNA Benchmark

Evan M. Williams, Kathleen M. Carley

专题命中 多模态评测 ：multimodal(title)；分类 cs.CV、cs.CL、cs.AI

Comments 11 pages, 3 figures

MOCAS: A Multimodal Dataset for Objective Cognitive Workload Assessment on Simultaneous Tasks

Wonse Jo, Ruiqi Wang, Su Sun, Revanth Krishna Senthilkumaran, Daniel Foti, Byung-Cheol Min

专题命中 多模态评测 ：multimodal(title,abstract)

Comments 18 pages, 10 figures, 9 tables, Accepted IEEE transaction Affective Computing;

Leveraging Generative AI for Extracting Process Models from Multimodal Documents

Marvin Voelter, Raheleh Hadian, Timotheus Kampik, Marius Breitmayer, Manfred Reichert

专题命中 多模态评测 ：multimodal(title)；multi-modal(abstract)

From Tissue Plane to Organ World: A Benchmark Dataset for Multimodal Biomedical Image Registration using Deep Co-Attention Networks

Yifeng Wang, Weipeng Li, Thomas Pearce, Haohan Wang

专题命中 多模态评测 ：multimodal(title,abstract)

Touch100k: A Large-Scale Touch-Language-Vision Dataset for Touch-Centric Multimodal Representation

Ning Cheng, Changhao Guan, Jing Gao, Weihao Wang, You Li, Fandong Meng, Jie Zhou, Bin Fang, Jinan Xu, Wenjuan Han

专题命中 多模态评测 ：multimodal(title,abstract)

Automated Assessment and Adaptive Multimodal Formative Feedback Improves Psychomotor Skills Training Outcomes in Quadrotor Teleoperation

Emily Jensen, Sriram Sankaranarayanan, Bradley Hayes

专题命中 多模态评测 ：multimodal(title,abstract)

Comments Under review at Human-Agent Interaction 2024 conference

Shopping Queries Image Dataset (SQID): An Image-Enriched ESCI Dataset for Exploring Multimodal Learning in Product Search

Marie Al Ghossein, Ching-Wei Chen, Jason Tang

专题命中 多模态评测 ：multimodal(title,abstract)

M$^4$oE: A Foundation Model for Medical Multimodal Image Segmentation with Mixture of Experts

Yufeng Jiang, Yiqing Shen

专题命中 多模态评测 ：multimodal(title,abstract)

The Dark Side of Dataset Scaling: Evaluating Racial Classification in Multimodal Models

Abeba Birhane, Sepehr Dehdashtian, Vinay Uday Prabhu, Vishnu Boddeti

专题命中 多模态评测 ：multimodal(title,abstract)

Comments To appear in the proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency (FAccT 24), June 3 to 6, 2024, Rio de Janeiro, Brazil. arXiv admin note: text overlap with arXiv:2306.13141

Dataset and Models for Item Recommendation Using Multi-Modal User Interactions

Simone Borg Bruun, Krisztian Balog, Maria Maistro

专题命中 多模态评测 ：multi-modal(title,abstract)

Avoiding Redundant Restarts in Multimodal Global Optimization

Jacob de Nobel, Diederick Vermetten, Anna V. Kononova, Ofer M. Shir, Thomas Bäck

专题命中 多模态评测 ：multimodal(title,abstract)

Transformer-Based Local Feature Matching for Multimodal Image Registration

Remi Delaunay, Ruisi Zhang, Filipe C. Pedrosa, Navid Feizi, Dianne Sacco, Rajni Patel, Jayender Jagadeesan

专题命中 多模态评测 ：multimodal(title,abstract)

Comments Accepted to SPIE Medical Imaging 2024

Multimodal Physical Fitness Monitoring (PFM) Framework Based on TimeMAE-PFM in Wearable Scenarios

Junjie Zhang, Zheming Zhang, Huachen Xiang, Yangquan Tan, Linnan Huo, Fengyi Wang

专题命中 多模态评测 ：multimodal(title)；multi-modal(abstract)

Comments 5 pages, 6 figures

Neuro-Inspired Information-Theoretic Hierarchical Perception for Multimodal Learning

Xiongye Xiao, Gengshuo Liu, Gaurav Gupta, Defu Cao, Shixuan Li, Yaxing Li, Tianqing Fang, Mingxi Cheng, Paul Bogdan

专题命中 多模态评测 ：multimodal(title,abstract)

Comments Accepted by ICLR 2024. Camera Ready Version

MERBench: A Unified Evaluation Benchmark for Multimodal Emotion Recognition

Zheng Lian, Licai Sun, Yong Ren, Hao Gu, Haiyang Sun, Lan Chen, Bin Liu, Jianhua Tao

专题命中 多模态评测 ：multimodal(title,abstract)

Self-Adjusting Evolutionary Algorithms Are Slow on Multimodal Landscapes

Johannes Lengler, Konstantin Sturm

专题命中 多模态评测 ：multimodal(title)；multi-modal(abstract)

Imbalance Knowledge-Driven Multi-modal Network for Land-Cover Semantic Segmentation Using Images and LiDAR Point Clouds

Yameng Wang, Yi Wan, Yongjun Zhang, Bin Zhang, Zhi Gao

专题命中 多模态评测 ：multi-modal(title,abstract)

LimSim++: A Closed-Loop Platform for Deploying Multimodal LLMs in Autonomous Driving

Daocheng Fu, Wenjie Lei, Licheng Wen, Pinlong Cai, Song Mao, Min Dou, Botian Shi, Yu Qiao

专题命中 多模态评测 ：multimodal(title,abstract)

Comments Accepted by 35th IEEE Intelligent Vehicles Symposium (IV 2024)

HealthPrism: A Visual Analytics System for Exploring Children's Physical and Mental Health Profiles with Multimodal Data

Zhihan Jiang, Handi Chen, Rui Zhou, Jing Deng, Xinchen Zhang, Running Zhao, Cong Xie, Yifang Wang, Edith C. H. Ngai

专题命中 多模态评测 ：multimodal(title,abstract)

Comments fixed some typos, 11 pages, 6 figures, Accepted by IEEE VIS23