多模态大模型

A Heterogeneous Graph Neural Network Fusing Functional and Structural Connectivity for MCI Diagnosis

Feiyu Yin, Yu Lei, Siyuan Dai, Wenwen Zeng, Guoqing Wu, Liang Zhan, Jinhua Yu

专题命中 多模态生成 ：cross-modal(abstract)；分类 cs.CV、cs.AI

AURORA: Navigating UI Tarpits via Automated Neural Screen Understanding

Safwat Ali Khan, Wenyu Wang, Yiran Ren, Bin Zhu, Jiangfan Shi, Alyssa McGowan, Wing Lam, Kevin Moran

专题命中 多模态生成 ：multi-modal(abstract)；分类 cs.CV、cs.CL

Comments Published at 17th IEEE International Conference on Software Testing, Verification and Validation (ICST) 2024, 12 pages

Aligning Text-to-Image Diffusion Models with Reward Backpropagation

Mihir Prabhudesai, Anirudh Goyal, Deepak Pathak, Katerina Fragkiadaki

专题命中 多模态生成 ：image-text(abstract)；分类 cs.CV、cs.AI

Comments This paper is subsumed by a later paper of ours: arXiv:2407.08737

Language Models and Cycle Consistency for Self-Reflective Machine Translation

Jianqiao Wangni

专题命中 多模态生成 ：any-to-any(abstract)；分类 cs.CL、cs.AI

ACE: All-round Creator and Editor Following Instructions via Diffusion Transformer

Zhen Han, Zeyinzi Jiang, Yulin Pan, Jingfeng Zhang, Chaojie Mao, Chenwei Xie, Yu Liu, Jingren Zhou

专题命中 多模态生成 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Slight Corruption in Pre-training Data Makes Better Diffusion Models

Hao Chen, Yujin Han, Diganta Misra, Xiang Li, Kai Hu, Difan Zou, Masashi Sugiyama, Jindong Wang, Bhiksha Raj

专题命中 多模态生成 ：image-text(abstract)；分类 cs.CV、cs.AI

Comments NeurIPS 2024 Spotlight

Non-rigid Relative Placement through 3D Dense Diffusion

Eric Cai, Octavian Donca, Ben Eisner, David Held

专题命中 多模态生成 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Conference on Robot Learning (CoRL), 2024

Towards Visual Text Design Transfer Across Languages

Yejin Choi, Jiwan Chung, Sumin Shim, Giyeong Oh, Youngjae Yu

专题命中 多模态生成 ：multimodal(abstract)；分类 cs.CV、cs.AI

Neural Fields in Robotics: A Survey

Muhammad Zubair Irshad, Mauro Comi, Yen-Chen Lin, Nick Heppert, Abhinav Valada, Rares Ambrus, Zsolt Kira, Jonathan Tremblay

专题命中 多模态生成 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments 20 pages, 20 figures. Project Page: https://robonerf.github.io

Multi-Track MusicLDM: Towards Versatile Music Generation with Latent Diffusion Model

Tornike Karchkhadze, Mohammad Rasool Izadi, Ke Chen, Gerard Assayag, Shlomo Dubnov

专题命中 多模态生成 ：cross-modal(abstract)；分类 cs.MM、eess.AS

DaLPSR: Leverage Degradation-Aligned Language Prompt for Real-World Image Super-Resolution

Aiwen Jiang, Zhi Wei, Long Peng, Feiqiang Liu, Wenbo Li, Mingwen Wang

专题命中 多模态生成 ：multimodal(abstract)；分类 cs.CV、cs.CL

Generative AI and Its Impact on Personalized Intelligent Tutoring Systems

Subhankar Maity, Aniket Deroy

专题命中 多模态生成 ：multimodal(abstract)；分类 cs.CL、cs.AI

Comments Scientific Report (Under Review)

Modeling News Interactions and Influence for Financial Market Prediction

Mengyu Wang, Shay B. Cohen, Tiejun Ma

专题命中 多模态生成 ：multi-modal(abstract)；分类 cs.CL、cs.AI

Comments Accepted by EMNLP 2024

The Ingredients for Robotic Diffusion Transformers

Sudeep Dasari, Oier Mees, Sebastian Zhao, Mohan Kumar Srirama, Sergey Levine

专题命中 多模态生成 ：multi-modal(abstract)；分类 cs.CV、cs.AI

ECIS-VQG: Generation of Entity-centric Information-seeking Questions from Videos

Arpan Phukan, Manish Gupta, Asif Ekbal

专题命中 多模态生成 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments Accepted in EMNLP 2024, https://openreview.net/forum?id=CriKOn01dI

ViT3D Alignment of LLaMA3: 3D Medical Image Report Generation

Siyou Li, Beining Xu, Yihao Luo, Dong Nie, Le Zhang

专题命中 多模态生成 ：multimodal(abstract)；分类 cs.CV、cs.AI

ObjectCompose: Evaluating Resilience of Vision-Based Models on Object-to-Background Compositional Changes

Hashmat Shadab Malik, Muhammad Huzaifa, Muzammal Naseer, Salman Khan, Fahad Shahbaz Khan

专题命中 多模态生成 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Journal ref Asian Conference on Computer Vision - 2024

See Detail Say Clear: Towards Brain CT Report Generation via Pathological Clue-driven Representation Learning

Chengxin Zheng, Junzhong Ji, Yanzhao Shi, Xiaodan Zhang, Liangqiong Qu

专题命中 多模态生成 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Comments Our work has been accepted by EMNLP2024 findings

DC3DO: Diffusion Classifier for 3D Objects

Nursena Koprucu, Meher Shashwat Nigam, Shicheng Xu, Biruk Abere, Gabriele Dominici, Andrew Rodriguez, Sharvaree Vadgama, Berfin Inal, Alberto Tono

专题命中 多模态生成 ：multimodal(abstract)；分类 cs.CV、cs.AI

CadVLM: Bridging Language and Vision in the Generation of Parametric CAD Sketches

Sifan Wu, Amir Khasahmadi, Mor Katz, Pradeep Kumar Jayaraman, Yewen Pu, Karl Willis, Bang Liu

专题命中 多模态生成 ：multimodal(abstract)；分类 cs.CV、cs.AI

Are LLMs Ready for Real-World Materials Discovery?

Santiago Miret, N M Anoop Krishnan

专题命中 多模态生成 ：multi-modal(abstract)；分类 cs.CL、cs.AI

MAIRA-2: Grounded Radiology Report Generation

Shruthi Bannur, Kenza Bouzid, Daniel C. Castro, Anton Schwaighofer, Anja Thieme, Sam Bond-Taylor, Maximilian Ilse, Fernando Pérez-García, Valentina Salvatelli, Harshita Sharma, Felix Meissen, Mercy Ranjit, Shaury Srivastav, Julia Gong, Noel C. F. Codella, Fabian Falck, Ozan Oktay, Matthew P. Lungren, Maria Teodora Wetscherek, Javier Alvarez-Valle, Stephanie L. Hyland

专题命中 多模态生成 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments 72 pages, 21 figures. v2 updates the model and adds results on the PadChest-GR dataset

The Era of Foundation Models in Medical Imaging is Approaching : A Scoping Review of the Clinical Value of Large-Scale Generative AI Applications in Radiology

Inwoo Seo, Eunkyoung Bae, Joo-Young Jeon, Young-Sang Yoon, Jiho Cha

专题命中 多模态生成 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments 25 pages,3 figures, 4 tables, submitted to NPJ imaging

Sparks of Artificial General Intelligence(AGI) in Semiconductor Material Science: Early Explorations into the Next Frontier of Generative AI-Assisted Electron Micrograph Analysis

Sakhinana Sagar Srinivas, Geethan Sannidhi, Sreeja Gangasani, Chidaksh Ravuru, Venkataramana Runkana

专题命中 多模态生成 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Published at Deployable AI (DAI) Workshop at AAAI-2024

ChatSUMO: Large Language Model for Automating Traffic Scenario Generation in Simulation of Urban MObility

Shuyang Li, Talha Azfar, Ruimin Ke

专题命中 多模态生成 ：multi-modal(abstract)；分类 cs.CL、cs.AI

Improving Text-guided Object Inpainting with Semantic Pre-inpainting

Yifu Chen, Jingwen Chen, Yingwei Pan, Yehao Li, Ting Yao, Zhineng Chen, Tao Mei

专题命中 多模态生成 ：multi-modal(abstract)；分类 cs.CV、cs.MM

Comments ECCV 2024. Source code is available at https://github.com/Nnn-s/CATdiffusion

UGAD: Universal Generative AI Detector utilizing Frequency Fingerprints

Inzamamul Alam, Muhammad Shahid Muneer, Simon S. Woo

专题命中 多模态生成 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Qwen2 Technical Report

An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jianxin Yang, Jin Xu, Jingren Zhou, Jinze Bai, Jinzheng He, Junyang Lin, Kai Dang, Keming Lu, Keqin Chen, Kexin Yang, Mei Li, Mingfeng Xue, Na Ni, Pei Zhang, Peng Wang, Ru Peng, Rui Men, Ruize Gao, Runji Lin, Shijie Wang, Shuai Bai, Sinan Tan, Tianhang Zhu, Tianhao Li, Tianyu Liu, Wenbin Ge, Xiaodong Deng, Xiaohuan Zhou, Xingzhang Ren, Xinyu Zhang, Xipin Wei, Xuancheng Ren, Xuejing Liu, Yang Fan, Yang Yao, Yichang Zhang, Yu Wan, Yunfei Chu, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, Zhifang Guo, Zhihao Fan

专题命中 多模态生成 ：multimodal(abstract)；分类 cs.CL、cs.AI

Comments 26 pages, 1 figure

SwiftBrush v2: Make Your One-step Diffusion Model Better Than Its Teacher

Trung Dao, Thuan Hoang Nguyen, Thanh Le, Duc Vu, Khoi Nguyen, Cuong Pham, Anh Tran

专题命中 多模态生成 ：image-text(abstract)；分类 cs.CV、cs.AI

Comments Accepted to ECCV'24

JPEG-LM: LLMs as Image Generators with Canonical Codec Representations

Xiaochuang Han, Marjan Ghazvininejad, Pang Wei Koh, Yulia Tsvetkov

专题命中 多模态生成 ：multi-modal(abstract)；分类 cs.CV、cs.CL