Subject-driven image generation aims to synthesize new images that preserve the identity of the given subject while following textual instructions. Existing approaches often encode text and reference images separately. This limits cross-modal reasoning abilities and causes copy-paste artifacts. Recent frameworks that connect multimodal models and diffusion models improve instruction following, but largely overlook identity preservation. To address these limitations, we condition diffusion models on Multimodal Large Language Models (MLLMs) that jointly encode text and reference images, and augment it with VAE-based identity conditioning. A novel Dual Layer Aggregation (DLA) module is designed to aggregate multi-level MLLM features for optimal conditioning, and a multi-stage denoising strategy is applied to progressively balance the semantic information from MLLM and fine-detail identity from VAE during inference. Extensive experiments demonstrate that our approach harmonizes multimodal understanding with identity preservation, mitigates copy-paste issues, and achieves superior performance regarding human preference on subject-driven image generation. Our project website is available at https://zsh2000.github.io/squeeze-mllm-subject-gen/.
MACS: Modality-Aware Capacity Scaling for Efficient Multimodal MoE Inference
School of Software, Tsinghua University, Beijing, China（清华大学软件学院，北京，中国）；TJUNLP Lab, School of Computer Science and Technology, Tianjin University, China（天津大学计算机科学与技术学院，中国）；School of New Media and Communication, Tianjin University, China（天津大学新媒体与传播学院，中国）
MMToM-QA: Multimodal Theory of Mind Question Answering
New York University（纽约大学）；Harvard University（哈佛大学）；Massachusetts Institute of Technology（麻省理工学院）；UC San Diego（加州大学圣迭戈分校）；University of Virginia（弗吉尼亚大学）；Johns Hopkins University（约翰斯·霍普金斯大学）
Data often are formed of multiple modalities, which jointly describe the observed phenomena. Modeling the joint distribution of multimodal data requires larger expressive power to capture high-level concepts and provide better data representations. However, multimodal generative models based on variational inference are limited due to the lack of flexibility of the approximate posterior, which is obtained by searching within a known parametric family of distributions. We introduce a method that improves the representational capacity of multimodal variational methods using normalizing flows. It…
Large Multimodal Model-Based Environment-Aware Mobility Management
Department of Electrical and Computer Engineering and the Institute of New Media and Communications, Seoul National University（电气电子工程系和新媒体与通讯研究所，首尔国立大学）；Information Systems Technology and Design (ISTD) pillar, Singapore University of Technology and Design（信息与系统技术与设计（ISTD）部门，新加坡科技设计大学）；Computer, Electrical and Mathematical Sciences and Engineering Division, King Abdullah University of Science and Technology（计算机、电子与数学科学与工程系，卡塔尔科学与技术大学）
Evaluating GPT-5 as a Multimodal Clinical Reasoner: A Landscape Commentary
Department of Radiation Oncology, Winship Cancer Institute, Emory University School of Medicine（放射肿瘤科，Winship癌症研究所，埃默里大学医学院）；Department of Biomedical Engineering, Georgia Institute of Technology（生物医学工程系，佐治亚理工学院）
We address the problem of one-to-many mappings in supervised learning, where a single instance has many different solutions of possibly equal cost. The framework of conditional variational autoencoders describes a class of methods to tackle such structured-prediction tasks by means of latent variables. We propose to incentivise informative latent representations for increasing the generalisation capacity of conditional variational autoencoders. To this end, we modify the latent variable model by defining the likelihood as a function of the latent variable only and introduce an expressive multi…
Multimodal Federated Learning via Contrastive Representation Ensemble
Institute for AI Industry Research, Tsinghua University（清华大学人工智能产业研究院）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；University of Waterloo（滑铁卢大学）；Carnegie Mellon University（卡内基梅隆大学）
AdaptiveEmbed: Sample-Adaptive Multi-Vector Representation for Multimodal Retrieval
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）
Allen Institute for Artificial Intelligence（艾伦人工智能研究所）；Seoul National University（首尔国立大学）；University of Washington（华盛顿大学）
Large language models readily adapt to novel settings, even without task-specific training data. Can their zero-shot capacity be extended to multimodal inputs? In this work, we propose ESPER which extends language-only zero-shot models to unseen multimodal tasks, like image and audio captioning. Our key novelty is to use reinforcement learning to align multimodal inputs to language model generations without direct supervision: for example, in the image case our reward optimization relies only on cosine similarity derived from CLIP, and thus requires no additional explicitly paired (image, capt…
NMAM Institute of Technology（NMAM 理工学院）；MSR Institute of Technology（MSR 理工学院）
Multimodal medical image fusion helps to increase efficiency in medical diagnosis. This paper presents multimodal medical image fusion by selecting relevant features using Principle Component Analysis (PCA) and Particle Swarm Optimization techniques (PSO). DTCWT is used for decomposition of the images into low and high frequency coefficients. Fusion rules such as combination of minimum, maximum and simple averaging are applied to approximate and detailed coefficients. The fused image is reconstructed by inverse DTCWT. Performance metrics are evaluated and it shows that DTCWT-PCA performs bette…
Multimodal few-shot learning is challenging due to the large domain gap between vision and language modalities. Existing methods are trying to communicate visual concepts as prompts to frozen language models, but rely on hand-engineered task induction to reduce the hypothesis space. To make the whole process learnable, we introduce a multimodal meta-learning approach. Specifically, our approach decomposes the training of the model into a set of related multimodal few-shot tasks. We define a meta-mapper network, acting as a meta-learner, to efficiently bridge frozen large-scale vision and langu…
The University of Edinburgh（爱丁堡大学）；Imperial College, London（伦敦帝国学院）；University of Surrey（萨里大学）；Queen Mary University of London（伦敦玛丽女王大学）；Microsoft Research, Cambridge（微软研究院剑桥分部）
School of Computer, Information and Mathematical Sciences（计算机、信息与数学科学学院）
In this thesis, a multimodal biometric, secure encrypted data and encrypted biometric encoded into the QR code-based biometric-passport authentication method is proposed for national security applications. Firstly, using the Extended Profile - Local Binary Patterns (EP-LBP), a Canny edge detector, and the Scale Invariant Feature Transform (SIFT) algorithm with Image File Information (IMFINFO) process, the facial mark size recognition is initially achieved. Secondly, by using the Active Shape Model (ASM) into Active Appearance Model (AAM) to follow the hand and infusion the hand geometry charac…
